Explore traces and spans from the Traces Browser

The Traces Browser that is part of Tanzu Observability by Wavefront lets you explore the context and the details of your application’s traces.


This videovideo camera icon highlights the Traces Browser features and settings:

Explore the Traces Browser

See the Traces Browser:

  • Option 1:
    1. In your web browser, go to your Wavefront instance and log in.
    2. From the toolbar, select Applications > Traces.
  • Option 2:
    1. In your web browser, go to your Wavefront instance and log in.
    2. Drill down to the Traces Browser from the Service Dashboard, application map, table view, or grid view.

explore trace browser

From the Traces Browser, you can:

  • Query for traces and view the query results in the traces list.
  • Select a trace in the list and:
  • Examine a trace’s percentile indicator to see how the trace’s duration compares to the durations of the other listed traces. You can toggle the panel size for the traces list, service map, or trace details.
  • Export traces by clicking import tracing icon, save the JSON file, and view them later using Offline Traces.
  • Easily analyze your traces hierarchy and RED metrics by clicking the expand or restore icon.
    • Click the expand icon to expand the RED metrics and view them next to the trace list, and click the same icon again to restore to the default view. partial expand red metrics
    • Click the expand icon to expand the trace hierarchy and view them next to the trace list, and click the same icon again to restore to the default view. partial expand trace hierachy
  • Use the link icon icon to click to get the link to get a link and share what you’re seeing right now (NON-LIVE display) with other users.

Query for a List of Traces

In the Traces Browser, you can use the query builder or query editor to query for traces that include spans for a particular operation.

You can view the results in a traces list.

explore traces list

Here’s how to get started:

  1. Add an Operation or Trace ID. See Get Started with Trace Queries for details.
  2. Click Add Filter and specify a filter to limit the results.
  3. (Optional) Advanced users can use Query Editor to limit the scope even further.

You can use the trace list to:

  • Sort the returned traces according to different criteria.
  • Click a trace to view its context in the service map panel and its details in the trace details panel.
  • View a trace’s percentile indicator to see how the trace’s duration compares to the durations of the other listed traces – and potentially find outliers.

Save and Manage Searches

Because narrowing down a search in the Traces Browser can be time consuming, you can

  • Save a search. Saved searches are available to all members of your organization.
  • Access and delete your saved searches
  • Access saved searches from everyone in your organization.

After you’ve performed a search on the Application Status page, you can save your search.

  1. Put in the search parameters, such as applications or services, add filters, and click Search.
  2. Click the bookmark icon, specify a name for your search, and click Save.
Bookmark icon brings up save search dialog

Access or Delete Saved Searches

The Lists button gives you access to your own saved searches and to saved searches from others in your organization.

1. Click the Lists button to the left of the query. List button
2. In the dialog that appears:
  • Click My Recent Searches for a list of your recent searches.
  • Click My Saved Searches for a complete list of your searches.
  • Click All Saved Searches for a list of searches in your organization.

3. Click a saved search to select it.

List button
4. To delete a saved search from your list:
  1. Select the list.
  2. Click Manage
  3. Select Delete from the ellipsis icon to the left of that saved search.
  4. Click Delete and confirm.
Elipsis icon and delete option

Investigate the Service Map for a Trace

In the Traces Browser, use the service map to investigate the services that contribute spans to a selected trace.

explore service map

In the service map panel, you can:

  • View the service dependencies, and follow the flow of request calls from service to service.
  • Click on a service to display the RED metrics that reflect the health of the service:
    • Request count, error count, and trace duration at the 95th percentile, over all traces (root spans) that originate in the service.
    • Line charts that indicate the general contour of the RED metrics.
    • Fine-tune the time window of the RED metrics charts, and see events.
  • Navigate to the selected service’s dashboard for more service details.
  • Scroll the service map to zoom in or out, and re-center the service map.
  • Look for nodes that are grayed out. These represent services that contribute spans to at least one trace in the list, but not to the trace that is currently selected.

Examine Trace Details

In the Traces Browser, use the trace details panel to examine the spans that belong to a selected trace. Some of these spans may represent operations executed by other services.

trace details

In the trace details panel, you can:

  • Examine the call hierarchy for a selected trace.
  • Click any span to expand and see more detail.
  • Explore the trace’s critical path. This is an end-to-end sequence of blocking spans, where each blocking span must complete before the trace can complete.

Drill Down Into Spans and View Metrics and Span Logs

In the trace details panel, you can view details about a particular span in the trace. If that span came from another service, you can navigate to the dashboard for that service.

trace span details

Span details include:

  • Application tags. These are the application, service, cluster, and shard, as selected by the trace query.
  • Other tags, including the trace ID.
  • A clickable link to the corresponding Operation Dashboard that lets you examine the RED metrics associated with the call.

If your spans have been instrumented to show span logs, you can click Logs to fetch the logs and drill down to examine them. We don’t show span logs by default for better browser performance.

A Closer Look at Critical Paths

The trace details panel uses an orange line to show the critical path through a trace. You can think of the critical path as the end-to-end combination of spans that are the most blocking. These spans represent the sequence of operations that must complete before the trace itself can complete.

Analyzing the critical path of a trace can help you determine which operations took the most time, and can help you decide which operations to try to optimize.

We use the following rules to determine which spans to include in a critical path (in order of applicability):

  1. Ignore asynchronous spans (spans tagged with followFrom).
  2. Ignore spans that end before their parent starts.
  3. If a child span continues after its parent, ignore that continuation period.
  4. Choose longer spans over shorter siblings.
  5. Choose later spans over earlier spans.
  6. Choose child spans instead of their parent spans.