ACKs and window updates may affect the sender

While doing high-speed TCP communication, there will be a large amount of packets going in the opposite direction. These packets are of two types (almost without exception); ACK for received data and window update packets.

The window update packets are sent when the receiving application have read data from the socket buffer and informs the other end that it can send more data.

ACKs are sent to acknowledge received data. ACKs are usually sent for each second received segment to tell the sender that it can free the segments from its send window.

As a calculation example, assume a 1Gbit communication between two hosts using an MTU of 1500 bytes. At full speed the sending host would put ~82200 packets/second on the wire, which would correspond to ~41100 ACK packets/second received. To this must be added the window update packets, the amount of them are very implementation and application dependent but measurements with ttcp as the application shows that between 5000-10000 packets/second are window updates with a receive window of 32768 bytes is used. This measurements also show that ~47800 packets/second are transmitted in the opposite direction.  Note that the amount of window update packets may be piggy-backed on ACK packets (or vice versa) so it is difficult to give an exact number.

If ethernet is used as the transmission media then the minimum MTU is 64 byte, and in this case 64*8*47800=24473600 bits/second or around 25 Mbit/second would be needed in the return direction. While this may not be a significantly large number compared to the amount of data going in the opposite direction, the packet count is not ignorable.  It's the number of packets that puts heavier load on the receiver and the network equipment there between.

One simple solution to this is to just avoid sending acks for each second received segment, which would instead cause the acks to be sent either when the next window update packet goes out or when the TCP ACK clock times out (200 ms). Tests show that doing this reduces the return packet rate in the above example to about 10K packets/second, or 5 Mbit/second in return traffic. It is not investigated how this may affect non-bulk data flow or other types of traffic with small window sizes.

While the above usually requires changes to the IP stack, a change that only affects the receiving process is to increase its receive buffer size.  This will decrease the count of window updates that are sent back to the transmitting machine.

Anders Magnusson (