Overview
The cumulative distribution chart allows the user to see the cumulative distribution of values for a specific metric represented as percentages of the whole. It is conceptually similar to the histogram, but range "bucket" is replaced with a percentage value. The data is then graphed as a line.
CDF Chart Advantages
The benefit of the CDF chart is that it helps show what percentage of the data is performing outside of expectations, without potentially misleading wtih summary data such as “averages” or “medians”. Thus, the point at which your CDF tapers is a critical attribute to keep an eye on.
In the below visual, we can clearly see that about 5% of the performance data are at an abnormally high level. This enables you to determine whether the number of abnormal results is acceptable. Based on this chart, you might ask “If 5% of the users visiting our site are seeing poor performance, is that okay”?
The visualization of the distribution of data across a metric provides an understanding as to how the data is spread across the population. Depending on the reference point along the x-axis (the percentage) it shows what percentage of test runs has what performance. It allows you to understand what percentage of test runs falls between two data points for a given metric. Finally, on Catchpoint charts, you can use breakdowns to compare distributions, either by the shape of the chart or by the actual data/results.
Taking the above chart, the following claims can be easily made:
- Roughly 50% of all test runs were below 4s
- Seattle (green) had slower performance than Miami (blue), it is only for about 5% of the test runs that the difference is extremely large.
- Because the slope of each line is fairly flat until about the 90th percentile, performance is pretty consistent for 90% of the test runs, but the jump at 90-100 shows that 10% of the population really takes a performance hit. Potentially, this could be an area for improvement.
Creating CDF Charts
To create these charts in the portal, please click on the CDF Chart type button as shown below. It is available for all Chart Types.
This sets the Chart Dimension as “Percentile” which is fixed from 0 to 100%.
It can be generated using any of the standard metrics as with any other chart. Below is an example of a CDF chart first by Test and then by Node, for two Metrics.
