Traceroute - Explicit Congestion Notification (ECN) Support

Introduction

The Explicit Congestion Notification (ECN) field has taken on new importance due to Low Latency, Low Loss, and Scalable throughput (L4S) technology designed for extremely latency-sensitive applications (such as cloud games and cloud-rendered VR/AR). Conventionally, TCP/IP networks signal congestion by dropping packets. ECN is a mechanism where an ECN-aware router may, instead of dropping a packet, set a flag in the IP header to signal impending congestion. The receiver with ECN-capable transport protocol will inform the sender regarding the impending congestion. Once the sender receives this message, it triggers the congestion avoidance algorithm in place and notifies the receiver.

Summary of ECN Operation (RFC 3168)

TCP (Transmission Control Protocol) endpoints need to agree to use ECN exchanging by setting the appropriate flags during the TCP handshake.

1. Once ECN is agreed, the TCP sender determines whether each data packet should be treated with ECN by setting its ECN field in the IP header (2 bits):
   • 00: Not ECN-Capable Transport / Not-ECT (the packet should not be treated with ECN.)
   • 01 or 10: the packet should be treated with ECN.
     • 01 is known as ECT(0)
     • 10 is known as ECT(1)
2. When congestion is experienced, the router sets ECN to 11 CE (Congestion Experienced) and forwards the packet to the destination.
3. When the destination receives a packet with ECN equal to 11 CE (Congestion Experienced) it sets the ECE (ECN Echo) flag into the TCP ACK to notify the sender of the the congestion, and the need to slow down the transmission rate.
4. When the sender receives a TCP ACK with ECE (ECN Echo), it reacts like the data packet has been dropped (so triggering the congestion avoidance algorithm in place) and notifies the receiver that this action was taken by setting the TCP CWR (Congestion Window Reduce) flag into the next data packet it sends.

The ECN mechanism involves two levels, TCP (Transmission Control Protocol) and IP (Internet Protocol).

ECN in TCP (Transmission Control Protocol)

This uses two bits in the TCP header for setting up an ECN-enabled connection and endpoint-to-endpoint signaling: the ECN-Echo (ECE) and Congestion Window Reduce (CWR) flags. As these are two bits, there are four possible combinations:

These flags are handled appropriately by the Catchpoint Agent when ECN is enabled in a test's Advanced Settings. They are used during the TCP handshake to check for ECN-enabled endpoints and they are communicated between the Sender and Receiver to handle congestion.

However, what happens when the Sender and Receiver support ECN, and an intermeidate router needs to mark the packets instead of dropping them? Routers cannot access the TCP header; they can only access the IP header of the packet. This is why there is an ECN implementation in IP header as well.

ECN in IP (Internet Protocol)

This uses two bits in the IP header, so again there are four possible combinations.

The IP-level ECN codepoints enable our Traceroute test to simulate various scenarios. Using ECT(0) or ECT(1) from the sender simulates that it is from an ECN-capable endpoint. Routers treat the ECT(0) and ECT(1) codepoints as equivalent. With these two codepoints we can check whether the network experiences congestion along the path by detecting that the ECN flag was changed to CE. The data of ECN Change is available in Records page and Smartboard hop by hop visualization in the Network tab.

Support for ECN at IP level can be enabled for all Traceroute monitors: ICMP, UCP, TCP, InSession, and QUIC.

Accurate ECN (AccECN)

What we have discussed so far is called Classic ECN and will be referred to as Classic ECN going forward in this article. One weakness of Classic ECN is the feedback mechanism, which lets the sender know that some amount of congestion occurred during, but not how many packets were marked CE by intermediate routers. So, for have 1 RTT (Round Trip Time) we can have one congestion notification, but we do not know how many CE markings have been received for a period. This provides sufficient notification in most cases, but for new TCP mechanisms like Congestion Exposure (ConEx), Data Center TCP (DCTCP), and Low Latency, Low Loss, and Scalable Throughput (L4S), which require low loss and low latency, improvements were needed. The new enhancement to the Classic ECN is Accurate ECN with improved feedback mechanism (Accurate ECN RFC Draft). With AccurateECN, there is a counter implemented so that the sender will know if it is little or a lot of congestion.

The table below shows the options available depending on the capabilities of Sender (A) and Receiver (B):

Accurate ECN check is possible with the TCP Traceroute test and enabled by default with QUIC Traceroute. A TCP traceroute can be set to run with either Classic or Accurate ECN. Accurate ECN will fall back to Classic ECN in case the destination does not support it.

ECN in the Catchpoint Portal

Configuring ECN

Use Control Center to set either IP or TCP level ECN checks. On the Test Properties page, click on Add Advanced Settings to set ECN.

The IP and TCP level support is as follows:

• TCP-level ECN support – TCP Traceroute.
• IP-level ECN support – ICMP, UDP, TCP, InSession, QUIC traceroute.

At TCP level, we can either set Classic ECN or Accurate ECN. Accurate ECN has fallback to Classic ECN.

Available IP-level options are as follows.

Consuming ECN Data

Once a traceroute test is set to capture ECN data it can be consumed from Records Page and Smartboard.

ECN in TCP

On the Records page, the ECN at TCP level details can be found in two places. On the top, the summary card summarizes the ECN check for the run. Then the traceroute table displays the ECN flags that were set on the destination hop. The ECN flags displayed on the destination hop of the traceroute are for the ECN at TCP level.

The ECN summary card can have one of these possible messages.

The possible list of Classic ECN and Accurate ECN flags on the destination hop can be found in Table 1 and 3 above respectively.

ECN in IP

The ECN in IP can be consumed from Records page and Smartboard.

On the records page, either on the hop by hop visual or traceroute data table we can see ECN changes. The hop by hop visual will highlight ECN values if it changes or resets along the network path. The traceroute table will list the ECN value observed on each hop.

Similarly, Smartboard hop-by-hop visual on the Network tab will show the ECN value changes. Also, the changes will be available as an observation so that you can easily filter them out.

Abbreviations

• ECN - Explicit Congestion Notification
• ECT – ECN Capable transport
• ECE - ECN-Echo
• CWR - Congestion Window Reduced
• CE - Congestion Experienced
• AE – Accurate ECN