Learn how to sample your trace data and see them in Wavefront using sampling policies.

Wavefront intelligent sampling reduces the volume of ingested traces. The goal of intelligent sampling is to retain traces that are likely to be informative. But sometimes intelligent sampling discards traces that you want to keep. You can create a sampling policy to fine-tune intelligent sampling and let Wavefront know that you want to keep certain spans. Creating a sampling policy affects your costs as you store more data within Wavefront.

You can create a sampling policy, edit, delete, restore, deactivate, and see the version history of the policy you created:

This diagrams shows the user flow from creating a policy, to edititng, deleting, restoring, deactivation, and seeing teh version history of a policy.

Let’s look at how you can create a sampling policy and get familiar with the policy expressions.

Create a Sampling Policy

To create a sampling policy:

  1. In your web browser, go to your Wavefront cluster and log in.
  2. From the taskbar, click Applications > Sampling Policies.
  3. Click Create Policy and enter the details:
    UI Element Description
    Policy Name The name of the policy. The policy name is unique, and you cannot create another policy with the same name.
    Description Add a description to explain what the policy does. It helps administrators understand why the policy is in place.
    Policy Expression Define a policy expression to let Wavefront know about the spans you want to keep. This expression needs to be in the YAML format. See Add a Policy Expression for details.
    Sampling percentage Define the percentage of spans you want to retain by entering a value between 0 and 100. For example, 80 indicates that you want to keep 80% of the spans that meet the policy expression.
    Status The policy is in effect when it is in the active state. Once the policy is active, it affects your costs as you store more data within Wavefront.

    To see the number of spans stored per second after a sampling policy is created, see Track Volume of Trace Data Stored in Wavefront.
  4. Click Create Policy. the screenshot shows the create policy form with all the UI elements that were described in the table above.

Now, you see the policy you created listed under sampling policies.

Add a Policy Expression

When you create a sampling policy, you need to define a policy expression that specifies the spans you want to keep. This expression needs to be in the YAML format.

Let’s look at a policy expression that asks Wavefront to store traces if the application name is beachshirts.

Attribute Description
Key A source needs to be defined between {{}}.
For example, you can pass the following values and any span tag key as a source:
{{spanName}}, {{sourceName}}, {{startMillis}}, {{duration}}, and more.
Value Surround the value with double quotes ("").
Operation Policy expressions can contain operations that are cases sensitive, and operations that are not case sensitive:

Operations that are case sensitive: These operations match only if the case is the same. For example, {{application}} = "beachshirts" is equal to beachshirts but not beachShirts.
  • =, >, <, <=, >=, !=
  • equals, startsWith, contains, endsWith, matches
  • and, or
  • not
  • in
Operations that are not case sensitive: These operations match even if the case is not the same.
  • equalsIgnoreCase
  • startsWithIgnoreCase
  • endsWithIgnoreCase
  • containsIgnoreCase
  • matchesIgnoreCase

Advanced Example

The following policy expression creates a sampling policy to keep all spans from the beachshirts application where the operation starts with Shopping, the source contains prod, and the status code is 400 or 404.

{{application}} = "beachshirts" and 
{{spanName}} startsWith "Shopping." and 
{{sourceName}} contains "prod" and 
{{http.status_code}} in ("400", "404")

Edit a Sampling Policy

To edit a sampling policy:

  1. Click the vertical ellipsis next to the policy name and click Edit. the screenshot described above, with the edit option highlighted with a red box.

  2. Update the details, and click Update Policy.

Delete a Sampling Policy

To delete a sampling policy, click the vertical ellipsis next to the policy name and click Delete.

To see the policies you deleted, click View Deleted Policies.

Restore a Sampling Policy

To restore a deleted policy:

  1. Click View Deleted Policies.
  2. Click the vertical ellipsis next to the policy name and click Restore.

Deactivate or Activate Sampling Policy

Deactivate a Sampling Policy

To deactivate a sampling policy, Click the vertical ellipsis next to the policy name and click Deactivate.

When the policy is deactivated, it changes to an inactive state. By default, you don’t see inactive policies. Enable the Show Inactive toggle to see the deactivate policies in the sampling policies list.

Example: Shows the how inactive sampling policies toggle is enabled and as a result how you see the inactive sampling policy in the list of policies.

Activate a Sampling Policy

To activate a sampling policy:

  1. Enable the Show Inactive toggle to see the deactivate policies.
  2. Click the vertical ellipsis next to the inactive policy name, and click Activate.

Learn More

To learn more about the Wavefront’s trace data sampling strategies, see Trace Sampling.