Certain Web Vitals metrics, including Largest Contentful Paint (LCP), Time to Interactive (TTI), and Visually Complete (VC), may display unexpected values when testing a Single Page Application (SPA) or other ajax-based webpage with a multi-step Transaction Test.
For example, there are instances where a multi-step Transaction Test includes steps where the browser does not load and render a new webpage, but actions are performed which cause new content to load within the current page. This is known as "soft navigation”, or AJAX, and is quite common in Single Page Applications as well as traditional websites. The Web Vitals metrics displayed for soft navigation steps may be significantly different than expected, e.g. a larger value than the overall test time for the entire Transaction Test.
This type of discrepancy is due to how these metrics are defined and can be calculated with the Chromium API, as they were not originally designed to support soft navigation use-cases. As Google states in this FAQ,
“Each of the Core Web Vitals metrics is measured relative to the current, top-level page navigation. If a page dynamically loads new content and updates the URL of the page in the address bar, it will have no effect on how the Core Web Vitals metrics are measured. Metric values are not reset, and the URL associated with each metric measurement is the URL the user navigated to that initiated the page load.”
This is not an issue that Catchpoint can address directly, as the Web Vitals metrics are calculated based on Google’s methodology and the Chromium API, and as a result are subject to the limitations of their design and browser implementation. Our near-term plan is to stop capturing LCP, TTI, and VC for soft navigation steps and instead populate them with a NULL value, in order to avoid confusingly high values for the entire Transaction Test in these cases. We intend to make this change early in 2024. This change will not impact any hard navigation steps (where an entire new webpage is loaded.)
Fortunately, there are efforts underway at Google to address this and to provide support for Single Page Applications with Web Vitals: https://developer.chrome.com/blog/soft-navigations-experiment. From this article:
“…(T)he popular Single Page Application architecture has never been fully supported by the Core Web Vitals metrics…
…The soft navigations experiment is an exciting approach to how the Core Web Vitals initiative might evolve to measure a common pattern on the modern web that is currently missing from our metrics. While this experiment is in its early days yet—and there is much still to do—making the progress available so far to the broader web community to experiment with is an important step…
…At present, this soft navigation experiment is exactly that—an experiment… …We are very excited at the possibility of this experiment, but cannot offer guarantees on whether or when this will replace the current measurements.”
We will continue to monitor this experiment in the hopes that there will be a way to address this issue in the future. We are also following developments withing the broader web performance community, including open-source projects like Playwright, to see if any alternatives arise that we can implement.