Catchpoint - Benchmark Feature

  • 1 minute(s) read
Prev Next

What Happens When We Talk About Benchmark Feature

Website-Side Behaviour

image.png

  • Benchmark endpoints load within an iframe and initiate requests after the page’s onload event.
  • Requests are delayed by approximately 1 second post-onload to avoid impacting user experience.
  • Requests are made sequentially, not in parallel.
  • Currently, we test across 7 CDNs:
    • Amazon
    • Azure
    • CDN1 (Akamai)
    • Fastly
    • Cloudflare
    • Google
    • CDN Networks

image.png

🌐 CDN-Side Configuration

Benchmark tests use specific URLs managed by Catchpoint to evaluate CDN performance.

🔹 1 Packet Object URLs

These are lightweight requests used to measure latency:

https://lpulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://kpulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://jpulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://apulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://xpulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://cpulse.perflib.com/ipulse/ad/cp.com/creative/1p.gif
https://ctchpnt.akamaized.net/ipulse/ad/cp.com/creative/1p.gif

🔹 100k Packet Object URLs

Used to measure throughput and larger payload handling:

https://lpulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://kpulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://jpulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://apulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://xpulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://cpulse.perflib.com/ipulse/ad/cp.com/creative/100.jpg
https://ctchpnt.akamaized.net/ipulse/ad/cp.com/creative/100.jpg

  • They all are configured to go to the same origin server, and in all of them we try to do the following:

    • Cache the assets as long as possible through HTTP headers, and whatever cache settings each CDN has
    • Enable serve from cache, and do not validate with origin where possible
    • Enable specific HTTP headers to allow our JavaScript to gather data

⚙️ CDN Configuration Goals

  • Maximise caching via HTTP headers and CDN-specific settings.
  • Serve from cache without origin validation where possible.
  • Enable HTTP headers to support JavaScript-based data collection.

⚠️ Note: Despite these configurations, there's a ~0.2% to 0.4% chance that a CDN edge server may not have the asset cached and will call the origin. To maintain benchmark integrity, such cases are excluded from results.

🚀 Recent Benchmark Feature Updates

  • Option to deploy benchmarks without Catchpoint RUM tag.
  • Added Throughput as a new metric.
  • Added ability to report HTTP Version.
  • Improved test execution and reporting to minimise user impact.

🔒 Private Benchmarks

image.png

Private benchmarks allow you to run custom tests from real user devices, comparing performance across CDNs and origin servers.

Benefits:

  • Evaluate delivery infrastructure from real-world user perspectives.
  • Identify regional performance differences.
  • Determine the best-performing CDN or origin strategy.

Private benchmark results are excluded from the public dataset.