Moving Window Functions
The moving window functions in Wavefront allow you to calculate continuous aggregation over sliding windows. Wavefront offers many moving functions that allow you to create moving windows on various aggregation such as
count(), etc. These functions create continuous moving or sliding windows which can extremely useful to look at moving trends.
The chart below for instance shows the 24 hour moving average of the points received. As you can see it is a continuous moving average of the past 24 hours.
But what if you wanted to only look at the daily average for each day and display only that average across the day. Essentially what you want to create is fixed size time windows and record your metrics in these windows.
These windows are more commonly referred as tumbling windows. Tumbling windows are a series of fixed size, non-overlapping, and contiguous time intervals.
Moving Windows Versus Tumbling Windows
Before we look at how to create and use these tumbling windows for different use cases, let’s take a look at an example to understand thoroughly the concepts of moving and tumbling windows.
The following diagrams illustrate a stream with a series of points being reported and how they are mapped into 5-second moving windows versus 5-second tumbling windows.
The moving window slides ahead one second at a time and you can see how new points get added at the front and the older points keep falling out as the window moves ahead. So these windows overlap each other, whereas tumbling windows are fixed windows that do not overlap one another and the set of points are exclusive for each window.
If you look closely, these three tumbling windows are actually part of the moving windows and every 5th moving window forms this 5-second tumbling window. So in theory to create tumbling window of
n duration (5 seconds in this example) all you need to do is sample every nth moving window (5th window) sliding ahead at in the same unit of window duration (seconds in this example).
Going back to our example from top to look at the daily average rate, we can use the same concept and sample every
mavg(24h,$rate) at the end of each day.
You can do that using the time functions. In this case, we will use the
hour() function to sample the average at every midnight. This would give you the average of last 24 hours which is essentially the daily average and we display it across the day using the missing data function
In the chart above, the green circles show the sampled data and you can use
next() to display it across the entire day.
Below are more examples of tumbling windows.
Count the unique number of metrics per day starting at 0 every midnight at PDT
This chart shows all the how many metrics are reported into Wavefront from the Slack application.
This example is very similar to the earlier example. However, instead of sampling the moving count exactly at midnight, this expression takes into account any missing or delayed data and picks the first value of the 30 minutes past midnight and then finally the
next() function simply displays this value for the entire day.
Get a 50th percentile of every 5 minute bucket
Here is sample data being reported every 1 minute.
A 5-minute moving percentile will give you the 50th percentile point in the last 5 minutes and display it at every minute interval.
But what if you wanted to find a 50th percentile of points in fixed 5-minute window. You could use the similar logic as we saw in the earlier examples. Here we are using
align(5m,...) to sample, instead of the time functions followed by
next(), to display the 50th percentile for the entire 5-minute window.
This expression is plotting the raw data overlaid with the 5-minute tumble window for the 50th percentile. The orange line is the 50th percentile of points in every 5-minute fixed window. And you see how it coincides with the middle-valued point in that time interval.
Thus any of the moving functions can be converted into fixed or tumbling windows by correct sampling of the moving window for which you should use the time functions and/or