Explore traces and spans from the Wavefront UI

In the Traces Browser, you can explore the context and the details of your application’s traces.

Explore the Traces Browser

See the Traces Browser:

  • Option 1:
    1. In your web browser, go to your Wavefront cluster and log in.
    2. From the taskbar, click Applications > Traces.
  • Option 2: Drill down to the Traces Browser from the Service Dashboard, application map, table view, or grid view.

explore trace browser

From the Traces Browser, you can:

  • Query for traces and view the query results in the traces list.
  • Select a trace in the list and:
  • Examine a trace’s percentile indicator to see how the trace’s duration compares to the durations of the other listed traces. You can toggle the panel size for the traces list, service map, or trace details.
  • Export traces by clicking import tracing icon, save the JSON file, and view them later using Offline Traces.
  • Easily analyze your traces hierarchy and RED metrics by clicking the expand or restore icon.
    • Click the expand icon to expand the RED metrics and view them next to the trace list, and click the same icon again to restore to the default view. partial expand red metrics
    • Click the expand icon to expand the trace hierarchy and view them next to the trace list, and click the same icon again to restore to the default view. partial expand trace hierachy
  • Use icon to click to get the link to get a link and share what you’re seeing right now (NON-LIVE display) with other users.

Query for a List of Traces

In the Traces Browser, you can use the query builder or query editor to query for traces that include spans for a particular operation.

You can view the results in a traces list.

explore traces list

Here’s how to get started:

  1. Add an Operation or Trace ID. See Get Started with Trace Queries for details.
  2. Click Add Filter and specify a filter to limit the results.
  3. (Optional) Advanced users can use Query Editor to limit the scope even further.

You can use the trace list to:

  • Sort the returned traces according to different criteria.
  • Click a trace to view its context in the service map panel and its details in the trace details panel.
  • View a trace’s percentile indicator to see how the trace’s duration compares to the durations of the other listed traces – and potentially find outliers.

Investigate the Service Map for a Trace

In the Traces Browser, use the service map to investigate the services that contribute spans to a selected trace.

explore service map

In the service map panel, you can:

  • View the service dependencies, and follow the flow of request calls from service to service.
  • Click on a service to display the RED metrics that reflect the health of the service:
    • Request count, error count, and trace duration at the 95th percentile, over all traces (root spans) that originate in the service.
    • Line charts that indicate the general contour of the RED metrics.
    • Fine-tune the time window of the RED metrics charts, and see events.
  • Navigate to the selected service’s dashboard for more service details.
  • Scroll the service map to zoom in or out, and re-center the service map.
  • Look for nodes that are grayed out. These represent services that contribute spans to at least one trace in the list, but not to the trace that is currently selected.

Watch this video to see how a service map can help you pinpoint a performance bottleneck in a microservices architecture:

introduction to service maps

Examine Trace Details

In the Traces Browser, use the trace details panel to examine the spans that belong to a selected trace. Some of these spans may represent operations executed by other services.

trace details

In the trace details panel, you can:

  • Examine the call hierarchy for a selected trace.
  • Click any span to expand and see more detail.
  • Explore the trace’s critical path. This is an end-to-end sequence of blocking spans, where each blocking span must complete before the trace can complete.

Drill Down Into Spans and View Metrics and Span Logs

In the trace details panel, you can view details about a particular span in the trace. If that span came from another service, you can navigate to the dashboard for that service.

trace span details

Span details include:

  • Application tags. These are the application, service, cluster, and shard, as selected by the trace query.
  • Other tags, including the trace ID.
  • A clickable link to the corresponding dashboard that lets you examine the metrics associated with the call.

If your spans have been instrumented to show span logs, you can click Logs to fetch the logs and drill down to examine them. We don’t show span logs by default for better browser performance.

A Closer Look at Critical Paths

The trace details panel uses an orange line to show the critical path through a trace. You can think of the critical path as the end-to-end combination of spans that are the most blocking. These spans represent the sequence of operations that must complete before the trace itself can complete.

Analyzing the critical path of a trace can help you determine which operations took the most time, and can help you decide which operations to try to optimize.

We use the following rules to determine which spans to include in a critical path (in order of applicability):

  1. Ignore asynchronous spans (spans tagged with followFrom).
  2. Ignore spans that end before their parent starts.
  3. If a child span continues after its parent, ignore that continuation period.
  4. Choose longer spans over shorter siblings.
  5. Choose later spans over earlier spans.
  6. Choose child spans instead of their parent spans.