For best performance and cost reduction, Wavefront supports limits. Some limits are recommendations–if your environment exceeds the limits, you’ll see significant performance issues. Other limits result in an error if you exceed the limit.
Concurrent Query Limits
Starting with release 2020.22x, Wavefront enforces the following concurrent query limits. These limits are subject to change without notice.
Per Customer Concurrent Query Limit
Wavefront enforces a limit on concurrent queries for each customer cluster. The default is 1000. If you are getting repeated errors that your cluster is exceeding this limit, contact us.
The following error results if your environment exceeds this limit:
HTTP 429 TOO_MANY_REQUESTS Customer concurrent query limit exceeded. Please try again later. Contact email@example.com for help.
Per User Concurrent Query Limit
Starting with release 2020.22.x, Wavefront enforces a limit on per-user concurrent queries. The default limit is 100. Contact us if you believe that the setting doesn’t make sense for one of your users (for example, one of your service accounts) and we’ll discuss options with you.
The following error results if one of the users exceeds this limit:
HTTP 429 TOO_MANY_REQUESTS “User concurrent query limit exceeded. Please try again later. Contact firstname.lastname@example.org for help.”
Default Customer-Specific Limits
You can start with Wavefront either as part of a free trial or as a new customer. In both cases, a set of out-of-the-box limits applies to that customer account. You can contact our customer success team to request changes. In some cases, a change might involve additional costs.
|Metric length limit||Maximum number of characters for a metric name.||256|
|Histogram length limit||Maximum number of characters for a histogram name.
The maximum number of characters for a histogram name before release 2020-22.x was 128.
|Span length limit||Maximum number of characters for a span name.||128|
|Host length limit||Maximum number of characters for a source name.||128|
|Annotations count limit||Maximum number of point tags associated with a metric.||20|
|Annotations key length limit||Maximum number of characters in a point tag key.||64|
|Annotations value length limit||Maximum number of characters in a point tag value.||255|
|Counter length limit||Maximum number of characters in a counter metric.||256|
|Span annotations count limit||Maximum number of point tags associated with a span.||20|
|Span annotations key length limit||Maximum number of characters associated with a span point tag key.||128|
|Span annotations value length limit||Maximum number of characters associated with a span point tag value.||128|
|Span topology processing Ttl||10|
|Span topology dimensions||Dimensions associated with a span. Defaults to "application" "cluster" "shard"||128|
|Span logs size limit||Maximum size of a span log.||32768|
Wavefront Data Format Best Practices
Follow best practices to avoid hitting query limits and for improved query execution speed and meaningful results.
- Make the metrics the most stable part of your data:
- Do not include source names in the metric name. Wavefront captures sources separately.
- Do not include data or timestamps in the metric name. Each point has an associated timestamp.
- Aim for a metric hierarchy:
- Partition the top level of the metric hierarchy by including at least one dot.
- Organize metric names in a meaningful hierarchy from most general to most specific (i.e.
- For best performance, keep the number of distinct time series per metric and host to under 1000.
See Wavefront Data Naming for more best practices.
You can examine what’s going on with your cluster in several ways: