TCP behavior and the network
In one line
TCP throughput is governed by window / RTT / loss, so the network's latency and drop behaviour directly cap application speed.
The mechanics that matter
- Windowing + bandwidth-delay product (BDP): max in-flight = window; throughput ~= window / RTT. Long-fat networks (high BW x high RTT) need window scaling or they leave capacity unused.
- Congestion control: slow start, then congestion avoidance. CUBIC (loss-based, the common default) backs off on drops; BBR (model-based) targets bandwidth/RTT and tolerates random loss better.
- Loss is expensive: a dropped segment triggers retransmit and a congestion-window cut. On lossy or far links this crushes throughput - which is why you don't tail-drop TCP indiscriminately.
Design implications
- Size buffers and QoS so TCP isn't starved or bufferbloated (see Buffering, microbursts, and congestion signaling and QoS design - models and strategy).
- Prefer ECN over drops where supported - signal congestion without forcing retransmits.
- For high-BDP paths, ensure window scaling; for storage/replication, account for RTT in throughput planning.
Throughput vs goodput
Line rate, throughput, and goodput are three different numbers. Throughput counts bytes per second on the wire; goodput counts only error-free, useful application-layer bytes - headers, acknowledgments, and above all retransmissions inflate throughput while goodput stands still (you can even raise throughput by disabling compression while the user experience gets worse). The design consequence: state performance requirements and capacity plans in goodput, and budget the gap between wire speed and useful delivery - protocol overhead plus the retransmission rate the loss profile implies (Bandwidth and capacity planning applies the same discipline to AI fabrics, where incast loss taxes goodput hardest).
IPv6 / dual-stack note
TCP behaviour is identical over v6; just mind MTU/MSS so PMTUD issues don't cause silent stalls (MTU, fragmentation, and PMTUD).
Related
- Buffering, microbursts, and congestion signaling
- QoS design - models and strategy
- Factors affecting convergence
Spaced repetition
TCP throughput is approximately [...], so a high-bandwidth high-latency path needs [...] to fill the pipe.
TCP throughput is approximately window size / RTT, so a high-bandwidth high-latency path needs window scaling to fill the pipe.
Why is indiscriminately tail-dropping TCP harmful?
Each drop forces a retransmit and cuts the congestion window, collapsing throughput - signalling via ECN avoids the retransmit.
CUBIC vs BBR in a phrase: CUBIC is [...]; BBR is [...].
CUBIC vs BBR in a phrase: CUBIC is loss-based (backs off on drops); BBR is model-based (targets bandwidth/RTT, tolerates random loss).
[...] counts only error-free application-layer bytes per second - overhead and retransmissions raise throughput without raising it, so requirements and capacity plans should be stated in goodput.
Goodput counts only error-free application-layer bytes per second - overhead and retransmissions raise throughput without raising it, so requirements and capacity plans should be stated in goodput.
Sources
- Cisco Press, End-to-End QoS Network Design; TCP congestion-control literature.