HTTP Transaction Anatomy

  • 4 minute(s) read
Prev Next

The majority of the World Wide Web is built on top of the Hypertext Transfer Protocol (HTTP). Understanding how HTTP works will help in interpreting the data collected by Catchpoint’s monitoring service and diagnosing performance issues.

HTTP data rides above the Transmission Control Protocol (TCP), which breaks down large data requests and responses into smaller pieces, called packets. When a client starts a dialogue with a server, a TCP connection is opened between the two and HTTP data is transferred. When the dialogue is complete, the connection closes. All data in the HTTP protocol is expressed in ASCII text.

Basic HTTP Transaction

A basic HTTP transaction is composed of five parts:
http.png

  1. DNS Lookup: Client resolves the domain name to an IP address
    1. The client sends a query to the DNS server (usually that of the internet service provider)
    2. DNS server responds with IP address associated with that hostname
  2. Connect: The client establishes a TCP connection with the server using the resolved server IP. This is referred to as the ‘three-way handshake’
    1. The client sends an SYN (synchronize) packet.
    2. A web server sends SYN-ACK (synchronize acknowledge) packet.
    3. Client answers with ACK (acknowledge) packet
  3. Send: Client sends HTTP request
  4. Wait: The client waits for the server to respond to the HTTP request
    1. The web server processes the request, finds the resource, and sends the response back.
    2. The client receives the first byte of the first packet from the web server, which contains the HTTP Response headers and content.
  5. Load: Client loads content contained within the response
    1. The web server sends a second TCP segment with the PSH (Push) flag set. This indicates that it wants to push data to the client without buffering. This provides faster transfer because buffer-processing is bypassed.
    2. The client sends ACK. The client sends ACK for every two segments it receives from the host.
    3. The web server sends a third TCP segment with an HTTP_Continue, alerting the client that the server is continuing sending data.
  6. Close: The client sends a FIN packet to close the TCP connection to indicate the connection is no longer needed.

Other Types of HTTP Transactions**

Serial HTTP Transaction

Multiple requests are issued sequentially to a server. Each request establishes its own new connection. This method rarely occurs nowadays because modern browsers support parallel connections.
request.png

Request 1

  1. DNS Lookup: The client tries to resolve the domain name for the request.
    1. The client sends a DNS Query to the ISP DNS server.
    2. DNS server responds with the IP address for the requested host
  2. Connect: Client establishes TCP Connection (1) with the IP address
    1. The client sends an SYN packet.
    2. A web server sends an SYN-ACK packet.
    3. The client sends an ACK packet, concluding the three-way TCP connection establishment.
  3. Send: The client sends the HTTP request to the webserver.
  4. Wait: The client waits for the server to respond to the request.
  5. Load: The client loads the content of the response.

Request II

  1. Connect: Client establishes TCP Connection (2) with the IP address of hostname.com
  2. Send: Client sends the HTTP request to the webserver
  3. Wait: The client waits for the server to respond to the request.
  4. Load: The client loads the content of the response.

And so-on for additional requests...

Persistent HTTP Transaction

Browser/ HTTP client utilizes the same connection for different object requests that are made to the same hostname. HTTP 1.1 supports this natively while HTTP 1.0 requires Keep-Alive flags in the HTTP header.

resp.png

  1. DNS Lookup: The client tries to resolve the domain name.
    1. The client sends a DNS Query to the local ISP DNS server.
    2. DNS server responds with the IP address to hostname.com.
  2. Connect: Client establishes TCP Connection (1) with the IP address of hostname.com
    1. The client sends an SYN packet.
    2. A web server sends SYN-ACK packet
    3. The client sends an ACK packet, concluding a three-way TCP connection establishment.
  3. Send: The client sends the HTTP request to the webserver.
  4. Wait: The client waits for the server to respond to the request.
  5. Load: The client loads the content of the response.
  6. Send: Client sends additional HTTP request to the webserver
  7. Wait: The client waits for the server to respond to the request.
  8. Load: The client loads the content of the response.
  9. Repeat Send, Wait, Load for additional requests...

Parallel HTTP Transaction:

HTTP 1.1 protocol allows browsers to open multiple connections and perform requests in parallel. The following example shows a browser loading an HTML page from hostname.com that contains two image requests to hostname.com and two requests to adserver.com

Once the first request to the base URL is completed the subsequent requests are performed in parallel:

Here is a list of the default maximum number of connections per host that can be open in parallel by various browsers. The limits increase over time as newer browsers are released. The more parallel connections, the faster the webpage renders however studies have shown that there might be limits to it – having it at 10 would not necessarily yield better results.

To learn more about this topic, we highly recommend “HTTP: The Definitive Guide” by David Gourley and Brian Totty.