Running Instant Tests

Catchpoint’s WebPageTest Instant test is based on our popular WebPageTest.org platform, and enables you to perform the same types of tests instantly from within the Catchpoint Portal. The WebPageTest Instant Test has most of the same configuration options as the website, so if you are familiar with that platform, it should look pretty familiar. This guide will walk you through submitting a test and interpreting the results.

Running an Instant Test

Follow these steps to run a basic performance test using default settings.

1. Log into the Catchpoint Portal
2. In the main menu on the left, select Instant Tests
3. Click New Test
4. Input the URL of the web page you want to test in the Enter Website URL field
5. In the Device field, select the desktop browser or mobile device type that you want to run the test.
6. Select the Location (node) to run the test from
   a. Either input the name of the location in the Location field
   -or-
   b. Click List, then locate and select the desired location
7. Click Run Test to execute the test. The results will be available once the test completes, which may take several minutes depending on the website you are testing and the settings selected.

Advanced Options

When running an instant Web Page test, there are many options available. This section explains what each option does.

Test Parameters

• Connection – specifies the type and speed of internet connection you want the test to emulate.
• Browser Dimension – when testing using a Desktop browser, this setting lets you specify the dimensions of the browser window used to test the web page.
• Number of Runs – the number of test runs to execute for this Instant Test. By executing multiple test runs, you will get a wider sampling of web page performance.
• Repeat View – causes the page to be loaded twice during each Test Run.
• Tests Performed – choose Site Performance, Lighthouse, or both.
  • Site Performance - captures detailed site performance metrics including Core Web Vitals.
  • Lighthouse - Google's open-source initiative for auditing the performance of any web page.
• Label – you can assign one or more labels to the test, which makes it easier to find specific test results later.

Advanced

• Test Duration
  • Min test Duration – specifies how long the test will take at minimum. This way, even if the initial page load has completed, the test will continue to watch for any additional activity up to the minimum duration.
  • Stop Test at Document Complete – when selected, the test will terminate at Document Complete, regardless of the minimum duration setting.
• Capture Settings
  • Capture Full Size Video - when enabled, test results will include a full-size video of the browser rendering the webpage.
  • Capture Network Packet Trace (tcpdump) - when enabled, test results will include a tcpdump.
  • Save Response Bodies for Text Resources- when enabled, response bodies will be displayed under Request Details in Waterfall.
• User Agent String
  • Preserve Original UA String – save a copy of the original user agent string, which provides information to the website about the connecting device and browser
  • Append to UA String – specify additional text to append to the user agent string
  • Custom String - specify a fully customized user agent string
• Image Support
  • Disable AVIF image support - do not load AVIF formatted images
  • Disable WEBP image support - do not load WEBP formatted images
• Other Settings
  • Ignore SSL Certificate Errors - when loading the webpage, disregard any SSL certificate errors
• Custom Headers - include a custom header with all requests generated by the browser during the test
• Inject Script - include a custom javascript which will run after the page loads

Chromium

• Chromium Test Parameters
  • Run Carbon Control- when selected, your website's carbon footprint will measured.
  • Use Chrome dev tools traffic shaping - can be used to emulate different internet connection characteristics. We recommend using the settings available in the Advanced tab.
  • Host Resolver Rules - one or more hostnames and the IP address(es) you want them to resolve to. Settings here will bypass DNS lookup for the specified hosts.
  • Command Line - additional Chromium command line switch(es) to customize test behavior.
• Capture Settings
  • Capture Dev Tools Timeline - when enabled, test results will include the Chrome DevTools timeline trace.
  • Capture Network Log - when enabled, test results will include a HAR file listing all network requests.
• Trace Settings
  • Capture Chrome Trace - when enabled, test results will include Chrome Trace.
    • Trace Categories - a listing of the trace categories to include in the Trace.
  • Enable V8 Sampling Profiler - when enabled, the log file will include a sampling of javascript activity that occurred during the test. This results in much larger trace files.
  • Capture V8 Runtime Call Stats - when enabled, test results will include certain low-level V8 metrics which aren’t currently exposed in DevTools.

Script

• Script - Enable/Disable including a test script
• Script includes sensitive data - select this option to omit the script from test results
• Discard All HTTP Headers - select to remove all HTTP headers from test results

Block

• Block Requests Containing URL Substrings - specify a list of substrings and the test will block requests containing any of them
• Block Domains Full Host Name - specify a list of host names and the test will block requests to any specified hosts

SPOF

• Hosts to fail - use this setting to simulate failure of specified domains. All requests for the domains will be rerouted to blackhole.webpagetest.org which will silently drop all requests.

Custom

WebPageTest can execute arbitrary javascript at the end of a test to collect custom metrics. These can be defined statically in the server configuration or be specified at runtime on a per-test basis.

The javascript should be written as if it were executing inside of a function call and return the custom metric at the end. Here is an example that finds the meta viewport for the page and extracts it:

var viewport = undefined;
var metaTags=document.getElementsByTagName("meta");
for (var i = 0; i < metaTags.length; i++) {
    if (metaTags[i].getAttribute("name") == "viewport") {
        viewport = metaTags[i].getAttribute("content");
        break;
    }
}
return viewport;

Supported Browsers

• Internet Explorer
• Chrome
• Firefox

Things to watch out for

• Asynchronous operations are supported by returning a Promise, but it must resolve after 30 seconds.
• Custom metrics live in the same namespace as the built-in metrics and can override built-in metrics if they have the same name.
• Metric names should be simple alpha-numeric and probably without spaces.

Specifying custom metrics at test time

The text input box for the metrics takes the metrics definitions in the form of an ini file for simplicity:

[metric-name]
<code>

[metric-2-name]
<metric 2 code>

Here is an example that collects 3 different metrics (2 numerical and one string):

[iframe-count]
return document.getElementsByTagName("iframe").length;

[script-tag-count]
return document.getElementsByTagName("script").length;

[meta-viewport]
var viewport = undefined;
var metaTags=document.getElementsByTagName("meta");
for (var i = 0; i < metaTags.length; i++) {
    if (metaTags[i].getAttribute("name") == "viewport") {
        viewport = metaTags[i].getAttribute("content");
        break;
    }
}
return viewport;