Experience Score

  • 4 minute(s) read
Prev Next

Experience Score Overview

Experience Score is a composite metric which grades the overall digital experience of individual tests or groups of tests on a scale of 0-100. Its purpose is to provide a quick, intuitive summary of the impact of test performance on user experience across different test types. When viewed in a Custom Dashboard using the Gauge Visualization, Experience Scores can be shown for individual tests, or they can be aggregated in various ways, such as by Folder, Geography, Label, etc.

How Experience Score is Calculated

Experience Score is calculated by three different methods, depending on the test type, as explained below.

Playwright, Puppeteer, Chrome (New), Mobile (New), and Edge Web Tests

For these test types, the Experience Score is calculated as follows:

First Contentful Paint

FCP measures how long the browser takes to render the first piece of DOM content after a user navigates to the page.

  • Good (Green) = 0.9 seconds or less
  • Needs Improvement (Orange) = 1.6 seconds or less and greater than 0.9 seconds
  • Poor (Red) = greater than 1.6 seconds

Total Blocking Time

TBT measures the total amount of time a page is blocked from responding to user input, such as mouse clicks, screen taps, or keyboard presses.

  • Good (Green) = 150 milliseconds or less
  • Needs Improvement (Orange) = 350 milliseconds or less and greater than 150 milliseconds
  • Poor (Red) = greater than 350 milliseconds

Largest Contentful Paint

LCP measures when the largest content element in the viewport is rendered on the screen. This approximates when the main content of the page becomes visible to users.

  • Good (Green) = 1.2 seconds or less
  • Needs Improvement (Orange) = 2.4 seconds or less and greater than 1.2 seconds
  • Poor (Red) = greater than 2.4 seconds

Cumulative Layout Shift

CLS measures the largest burst of layout shift scores for all unexpected layout shifts that occur during the entire lifecycle of a page. A layout shift occurs when a visible element changes its position from one rendered frame to the next.

A burst of layout shifts, known as a session window, occurs when one or more individual layout shifts happen in rapid succession, with less than 1 second between each shift and a maximum total window duration of 5 seconds.

  • Good (Green) = 0.1 or less
  • Needs Improvement (Orange) = 0.25 or less and greater than 0.1
  • Poor (Red) = greater than 0.25

Apdex Experience

Apdex is the only metric for which we do not use the Chrome calculation.

Apdex uses two test time thresholds to divide user experience into three bands:

  • Satisfied (Green)
  • Tolerating (Orange)
  • Frustrated (Red)

Catchpoint supports these thresholds in Test Properties.

We calculate Apdex using the following formula, which yields an index value from 0 to 1. Catchpoint currently supports this calculation.
apdex_experience_ score.png

Note: We must count test failures in the Frustrated range, regardless of how long the test takes to fail (for example, quick failures are not counted as Satisfied).

To produce a value from 0 to 100, we calculate the Experience Score as follows:

experience_score1.png

Examples:

  • All tests perform in the Satisfied zone: Application Score = 100
  • 50% of tests are in the Satisfied zone and 50% are in the Tolerating zone: Application Score = 75
  • 50% of tests are in the Satisfied zone and 50% are in the Frustrated zone or fail: Application Score = 50

The experience score is a weighted average of the metric scores. Metrics with higher weights have a greater impact on the overall experience score.

We weight each metric as follows:

Metric Weight
First Contentful Paint 10%
Apdex Experience 10%
Largest Contentful Paint 25%
Total Blocking Time 30%
Cumulative Layout Shift 25%

To calculate the total experience score, we use the following formula:

score_example.png

Example:
Experience score = (85 × 0.1) + (95 × 0.1) + (75 × 0.25) + (90 × 0.3) + (80 × 0.25) = 83.75

The overall score is 83.75 out of 100, indicating a fair but improvable user experience.

This document explains the 0–100 scoring scale and how to interpret the scores in the following table:

Score Range Rating
90 to 100 Good
50 to 89 Needs improvement
0 to 49 Poor

Ping and Traceroute Tests

The Experience Score for Ping and Traceroute tests is generated using the Packet Loss and Latency (Ping Round Trip) metrics.

First, we calculate two values, Packet Loss Score and Latency Score, as follows:

Packet Loss Score is determined by inverting the percent value of the Packet Loss metric (i.e. subtracting it from 100).

Packet Loss = 0%   ⇒   Packet Loss Score = 100
Packet Loss = 20%  ⇒   Packet Loss Score = 80
Packet Loss = 65%  ⇒   Packet Loss Score = 35

Latency Score is determined from the Ping Round Trip (ms) metric as follows:

Ping RTT < 20ms          ⇒  Latency Score = 100
20ms ≤ Ping RTT ≤ 100ms  ⇒  Latency Score scales linearly from 100 to 0
Ping RTT > 100ms         ⇒  Latency Score = 0

Test Experience Score is calculated by averaging Packet Loss Score and Latency Score, giving Packet Loss Score double weighting as per this equation:

Test Experience Score = ( 2x Packet Loss Score + Latency Score ) / 3

All Other Test Types

For all other test types, Experience Score = Apdex * 100. Apdex is an industry-standard performance index. It is calculated by applying thresholds to key performance-related metrics to determine whether a user is likely to be satisfied, tolerating, or frustrated with the user experience. Catchpoint tests have pre-set values for these thresholds, but you can custom-configure them to your needs.

For full details on Apdex, see: What is Apdex and how to use it? (catchpoint.com) and http://www.apdex.org/