Networking for System Design: TCP, UDP, HTTP/2, HTTP/3 and WebSockets

TCP gives order and reliability at a cost; UDP gives neither. Learn head-of-line blocking, slow start, HTTP/1.1 vs HTTP/2 vs HTTP/3 measured on a lossy network, connection pools and ports, and when to use polling, SSE or WebSockets.

Every design choice about how services talk to each other sits on a handful of network facts. TCP delivers bytes in order, so one lost packet can hold up everything behind it. A new connection starts slowly and speeds up. HTTP/2 puts many requests on one connection, and HTTP/3 moves them onto QUIC over UDP. A browser that needs live updates can poll, hold a request open, or keep a connection open.

Part 1 followed one request end to end. This part explains why the transport underneath behaves the way it does. We also ran HTTP/1.1, HTTP/2 and HTTP/3 in a lab with deliberate packet loss, and measured head-of-line blocking directly.

Try this first

A page needs 20 files of 50 KB each, from a server 50 ms away (one round trip). Four clients fetch them:

1. HTTP/1.1 over one connection, one file at a time.
2. HTTP/1.1 over six connections, which is roughly what browsers do.
3. HTTP/2 over one TCP connection, all 20 at once.
4. HTTP/3 over one QUIC connection, all 20 at once.

Rank them from fastest to slowest on a clean network. Then rank them again when 3 of every 100 packets from the server are lost. Write both rankings down. We measured all eight cases, and the answer is in the section on the lab.

The layers a designer deals with

The internet’s base layer, IP, moves packets between addresses on a best-effort basis. Packets can be lost, duplicated, delayed or reordered, and IP doesn’t fix any of that. The transport protocols on top decide what an application gets:

Protocol	What the application gets	Built on
TCP	A reliable, ordered stream of bytes, with flow and congestion control	IP
UDP	Individual datagrams, with no delivery or ordering guarantee	IP
TLS	Encryption and authentication for a TCP stream	TCP
QUIC	Reliable, ordered streams, several per connection, with TLS 1.3 built in	UDP
HTTP/1.1 and HTTP/2	Requests and responses	TLS over TCP
HTTP/3	Requests and responses	QUIC

Most of what follows depends on what happens when a packet is lost.

TCP: reliable and ordered, and what that costs

TCP turns lost, reordered packets into a clean stream of bytes. The receiver acknowledges what it has. The sender resends what wasn’t acknowledged. The receiver puts bytes back in order before handing them to the application.

That last part has a cost. TCP can only hand the application bytes in order. If segment 5 is lost and segments 6 to 20 arrive, the application gets nothing past segment 4 until segment 5’s resend arrives. This is head-of-line blocking: one missing piece holds up everything queued behind it.

For one file on one connection, that’s just a delay. For many requests sharing one connection, it’s a delay for all of them, even the ones whose data already arrived.

How long a loss costs

TCP notices loss in two main ways (RFC 5681 and RFC 8985):

• Quickly, from the packets that follow. When later segments arrive, the receiver keeps acknowledging the last in-order byte, and the sender resends the gap. This usually costs about one round trip.
• By probing, when nothing follows. If the last packets of a response are lost, no later packets arrive to reveal the gap. RACK-TLP (RFC 8985) sends a probe after a short timeout, by default two smoothed round-trip times, to trigger the acknowledgement that exposes the loss.
• Slowly, by timeout. If that fails too, the sender waits for its retransmission timer. RFC 6298 says that if the computed timeout “is less than 1 second, then the RTO SHOULD be rounded up to 1 second”. Linux uses a lower minimum of 200 ms, and starts a new connection with a 1-second timeout. Either way, a timeout costs several round trips on a fast path, and it cuts the sending rate hard.

A loss also tells TCP the network may be congested, so it cuts its sending rate. That slows the whole connection, not only the stream that lost data.

Slow start: why a new connection is slow even after the handshake

TCP doesn’t know how much the network can carry, so it starts cautiously and speeds up. It keeps a congestion window: how many segments it may have in flight without acknowledgement.

• The start: RFC 6928 allows an initial window of up to 10 segments, about 14.6 KB with 1,460-byte segments. Linux uses 10 (TCP_INIT_CWND). RFC 6928 is published as Experimental, but 10 is the common default.
• The growth: during slow start, the window grows by one segment for each segment acknowledged (RFC 5681). If every segment is acknowledged, the window doubles each round trip: 10, 20, 40, 80 and so on.
• The end: slow start ends when the window reaches a threshold (ssthresh) or at the first sign of congestion, when the window is cut. Linux’s default algorithm, CUBIC, can also leave slow start early when round-trip times start to rise.

Move the slider to see how many round trips a response needs on a new connection, with no loss.

Response size	Round trips of data on a new connection
14 KB	1
50 KB	3
100 KB	3
500 KB	6
1 MB	7

So at a 50 ms round trip, a 1 MB response on a brand-new connection needs about 350 ms of round trips after the handshakes. With an unlimited window it would need one, so about 300 ms of that is the window growing.

A connection that has just carried data has a larger window, so the same response arrives in fewer round trips. But the window only stays large while the connection is busy. RFC 5681 says a TCP that has been idle for longer than its retransmission timeout should shrink the window again. Linux does this by default (tcp_slow_start_after_idle), so a pooled connection that sat idle drops back towards the initial window. Reuse still saves the handshakes either way.

The first round trip carries only the initial window, about 14.6 KB, and on a new TLS connection the server’s certificate shares it. That’s why keeping the most important part of a response small is a known optimisation.

TIME-WAIT and running out of ports

When a TCP connection closes, the side that closes first keeps a record of it in the TIME-WAIT state for a while (RFC 9293). This stops delayed packets from an old connection being mistaken for a new one with the same addresses and ports.

On Linux, TIME-WAIT lasts 60 seconds (TCP_TIMEWAIT_LEN), and the default range of local ports for outgoing connections is 32768 to 60999, which is 28,232 ports. A connection is identified by its source and destination address and port. TIME-WAIT is held by whichever side closes the connection first. So if a client opens connections to one destination address and port and closes each one itself, it can sustain only about:

28,232 ports ÷ 60 seconds ≈ 471 new connections per second

A service that opens a new connection for every call to another service, at more than that rate to one address, runs out of local ports and starts failing to connect. If the server closes first instead, for example because of its idle timeout, TIME-WAIT sits on the server and uses none of the client’s ports. On Linux, ss -tan state time-wait lists the connections waiting. Linux can reuse TIME-WAIT ports for new outgoing connections when it’s safe (tcp_tw_reuse), but its default only does that for loopback traffic. An older option, tcp_tw_recycle, was removed from the kernel in 2017 because it broke connections.

The better fix is not to open that many connections: reuse them through a connection pool. Part 1’s common mistakes showed how in C#, Java, Go and Rust.

UDP: no guarantees, on purpose

UDP adds almost nothing to IP. Its header is 8 bytes: source port, destination port, length and checksum (RFC 768, which is three pages long). There’s no connection, no acknowledgement, no resending, no ordering, and no congestion control. A datagram may arrive once, more than once, or not at all, in any order: RFC 8085 notes that “UDP also does not protect against datagram duplication”.

That sounds worse, and for most data it is. It’s the right choice when:

• Old data is useless. In a voice call, a video call or a game, a sound or position from half a second ago isn’t worth waiting for. Waiting for a resend would make things worse.
• The exchange is one small question and one small answer. DNS queries have used UDP for decades. DNS also needs TCP when an answer is too big or UDP fails, and RFC 7766 makes TCP support “a REQUIRED part of a full DNS protocol implementation”.
• You want to build your own reliability on top. That’s exactly what QUIC does.

An application sending a lot of UDP traffic has to avoid overwhelming the network itself, because nothing below it will. RFC 8085, the guidelines for UDP use, says such an application “SHOULD control the rate at which it sends UDP datagrams”.

Explain it like I’m ten

TCP is like a phone call where you read a story to a friend, line by line. If a line gets garbled, your friend says “say that again”, and you wait until they have it before going on. Nothing is ever missed or out of order, but one bad line holds up the rest of the story.

UDP is like throwing postcards over a wall. Most arrive. Some don’t. Some arrive in the wrong order. Nobody tells you which. That’s useless for a story, but fine for shouting the score of a game every few seconds: if one card is lost, the next one has the new score anyway.

The precise version

TCP provides a reliable, ordered byte stream using sequence numbers, acknowledgements and retransmission, and it adjusts its sending rate with congestion control. UDP provides unreliable, unordered datagrams with an 8-byte header, and leaves reliability and rate control to the application.

Where the analogy breaks: a TCP sender doesn’t wait for each line to be confirmed before sending the next. It sends many segments at once, up to its window, and resends only what’s missing. The story is held up for the listener, who can’t use the later lines until the missing one arrives.

HTTP/1.1: one request at a time per connection

HTTP/1.1 connections are persistent by default, so one connection can carry many requests, but only one after another. The client sends a request, waits for the whole response, then sends the next.

HTTP/1.1 does allow pipelining: sending several requests without waiting. But the server “MUST send the corresponding responses in the same order that the requests were received” (RFC 9112), so a slow first response blocks the rest. This is head-of-line blocking at the HTTP level, and pipelining was never widely used by browsers.

So browsers open several connections to the same host, and send one request on each. RFC 9112 no longer sets a number. It says a client “ought to limit the number of simultaneous open connections that it maintains to a given server”. Browsers commonly use about six per host over HTTP/1.1. Every connection pays its own handshakes and its own slow start.

HTTP/2: many streams on one TCP connection

HTTP/2 (RFC 9113) keeps HTTP’s meaning and changes how it’s carried:

• Streams: each request and response is a stream, and frames from many streams are interleaved on one connection. Twenty requests no longer need twenty connections, or a queue.
• Binary framing: messages are split into typed frames instead of text lines.
• Header compression: HPACK (RFC 7541) avoids resending the same headers, such as cookies and user agents, on every request.
• Flow control per stream: a slow reader of one stream doesn’t have to stop the others at the HTTP layer. There’s also a window for the whole connection, so a stream nobody reads can still use it up if it’s sized too small.
• Server push, which let a server send responses before they were asked for, is still in the specification. It was hard to use well, and Chrome disabled it by default from Chrome 106.

One TCP connection also means one handshake, one congestion window to grow, and less competition between connections. RFC 9113 is careful about what it didn’t fix: “TCP head-of-line blocking is not addressed by this protocol.”

The streams are independent in HTTP/2, but the bytes of all of them travel in one ordered TCP stream. One lost packet holds up every stream behind it. Step through the model, then switch between protocols.

In words: three streams, A, B and C, each send four packets, taking turns. Stream B’s first packet is lost, and its resend arrives 6.5 time units later.

• HTTP/2 over one TCP connection: A’s and C’s packets that arrive after the lost one can’t be handed to the application until the resend fills the gap. All three streams stall.
• HTTP/3 over QUIC: QUIC keeps order per stream, so A and C are delivered as they arrive. Only stream B waits.
• HTTP/1.1 over three connections: each connection keeps its own order, so again only B waits. The cost is three handshakes and three windows to grow.

HTTP/3 and QUIC: streams that don’t block each other

QUIC (RFC 9000) is a transport protocol that runs over UDP and does what TCP does, and more:

• Independent streams. RFC 9000 says: “When a packet loss occurs, only streams with data in that packet are blocked waiting for a retransmission to be received, while other streams can continue making progress.”
• TLS 1.3 built in. The transport and cryptographic handshakes are combined, so a new connection needs one round trip in the common case, instead of TCP’s plus TLS’s two. Returning clients can use 0-RTT, with the replay caveats from Part 1.
• Better loss detection. Every packet gets a new, increasing packet number, even a retransmission. RFC 9002 explains this removes any “ambiguity” about which copy an acknowledgement refers to, which TCP has to work around.
• Connection migration. A connection is identified by connection IDs, not by addresses and ports, so it can survive a phone switching from Wi-Fi to mobile data (RFC 9000, section 9). That needs support in the client, the server and any load balancer, which has to route packets by connection ID. It isn’t universal: Microsoft’s documentation says HttpClient and Kestrel don’t support network transitions in .NET 7.
• More of the header is encrypted. Even the packet number is protected (RFC 9001), which leaves middleboxes less to inspect or interfere with.

HTTP/3 (RFC 9114) is HTTP carried over QUIC. It uses QPACK (RFC 9204) for header compression, because HPACK, in RFC 9204’s words, “would induce head-of-line blocking” in HTTP/3.

QUIC has costs too:

• Some networks block UDP. RFC 9308 cites 2016 measurements showing “between 3% […] and 5% […] of networks block all UDP traffic”, so clients must be able to fall back to HTTP/2 over TCP.
• Clients have to discover it. A server advertises HTTP/3 in an Alt-Svc response header (RFC 7838) or an HTTPS DNS record (RFC 9460), so the first visit often starts over TCP. When we checked on 15 September 2026, www.cloudflare.com and www.google.com sent alt-svc: h3=":443", and www.wikipedia.org sent no such header.
• It runs in user space. QUIC is usually implemented in a library inside the application, not in the operating system’s kernel. That can cost more CPU per byte than kernel TCP.

Measured: HTTP/1.1, HTTP/2 and HTTP/3 on a lossy network

The head-of-line figure above is a model. To see what real protocol stacks do, we built a small lab with Docker containers on a private network:

• Server: Caddy 2.11.4, serving over HTTP/1.1, HTTP/2 and HTTP/3. Its HTTP/3 uses the quic-go library, v0.59.1, and its TCP is the Linux kernel’s, with CUBIC congestion control.
• Network: Linux’s tc netem delays every packet the server sends by 50 ms, and drops a set share of them: 0%, 1% or 3%.
• Clients: curl 8.14.1 for page loads, and a small Go program for the head-of-line test.

Getting the network right took three attempts, and the lessons are part of the result:

• Offloads: the network interface first had segmentation offloads switched on, so netem dropped whole bundles of packets at once, and each protocol saw a different loss pattern. We switched them off, along with quic-go’s own packet batching.
• Queue size: netem’s default queue of 1,000 packets overflowed under QUIC’s bursts, adding drops we hadn’t asked for. We raised it.
• Counting: every run records netem’s drop counter and the server’s TCP retransmission counters, so we know what “3% loss” really meant. At the 3% setting, TCP retransmitted 3.1% of its segments and QUIC lost 3.05% of its packets. But even with offloads off, netem usually dropped TCP segments two at a time. So TCP met only about half as many separate losses as QUIC, 1.6% of its packets against 3.05%, and a burst of two costs one window cut. That favours the TCP setups.

Each experiment ran 30 times at each loss rate, with the protocols taking turns. We only call a difference real when a bootstrap 95% confidence interval for the difference in medians excludes zero.

Experiment 1: head-of-line blocking, measured directly

The Go client opens one connection and warms it with a tiny request, so the handshakes aren’t timed. Then it starts a 20 MB download, waits 300 ms, and sends 19 small requests, 10 KB each, on the same connection, one every 60 ms. It times each small request, and cancels the download once they’re all done. HTTP/2 uses Go’s standard library, and HTTP/3 uses quic-go, the same library as the server.

If a lost packet of the big download stalls the whole connection, the small requests stall with it. Move the slider to change the loss rate.

How long each small request took:

	0% loss	1% loss	3% loss
HTTP/2: median	52 ms	728 ms	1,969 ms
HTTP/2: 90th percentile	104 ms	3,257 ms	6,731 ms
HTTP/3: median	52 ms	92 ms	150 ms
HTTP/3: 90th percentile	53 ms	143 ms	287 ms

With no loss, both protocols answered the typical small request in one 50 ms round trip, so neither server nor client was slow on its own. The download often finished before the last small requests went out, and HTTP/2’s slowest tenth already took about two round trips.

With loss, the small requests on HTTP/2 waited behind the big download on the shared TCP byte stream: behind its lost packets, and behind its bytes already queued in the server’s send buffer. At 1% loss their median was 8 times HTTP/3’s, and at 3% it was 13 times. That’s TCP head-of-line blocking, measured. And it happened even though TCP met fewer separate losses than QUIC.

On HTTP/3 the small requests slowed too, to about 3 times their no-loss time at 3%. Probably they sometimes lost their own packets, and they share the connection’s congestion window. But they didn’t wait for the big stream. Both differences are clear: the confidence intervals for the difference in each run’s median exclude zero.

Experiment 2: loading a page

Each run is one new curl process fetching 20 files of 50 KB each, so it pays its own handshakes, like a first page load. We timed the whole curl run. Four setups:

1. HTTP/1.1 over one connection, one file at a time.
2. HTTP/1.1 over six connections.
3. HTTP/2 over one connection.
4. HTTP/3 over one connection.

The median page time:

	0% loss	1% loss	3% loss
HTTP/1.1, 1 connection	1,257 ms	2,043 ms	3,591 ms
HTTP/1.1, 6 connections	441 ms	543 ms	769 ms
HTTP/2, 1 connection	486 ms	760 ms	1,910 ms
HTTP/3, 1 connection	375 ms	968 ms	3,459 ms

On a clean network, every difference is clear. HTTP/3 was fastest, and one HTTP/1.1 connection was by far the slowest:

• One HTTP/1.1 connection sends 20 requests one after another, so once its window has grown it pays a round trip per file: about 24 round trips in total.
• HTTP/2 sends all 20 requests at once, but grows one congestion window for 1 MB of data: about 9 round trips, including the handshakes.
• Six HTTP/1.1 connections each grow their own window: about 8 round trips.
• HTTP/3 saves a handshake round trip, and quic-go starts with a larger window: 32 packets of 1,280 bytes, about 41 KB, where Linux TCP starts with about 14.6 KB. So part of HTTP/3’s lead here comes from quic-go’s default starting window, which is larger than RFC 9002’s suggestion of about 14.7 KB, not from the protocol.

A slow-start model predicts 1,200, 400 and 450 ms for the three TCP setups. Leaving out curl’s own start-up time, about 21 ms, the measurements were 1,236, 420 and 464 ms. For HTTP/3 the model predicts 300 ms against a measured 353 ms, about one round trip more. We haven’t pinned that gap down. One candidate is that quic-go paces its packets, spreading them out over a round trip instead of sending each window in a burst, and the model leaves pacing out. Another is how the curl client acknowledges packets and grants flow control. The checks script compares all four.

With loss, six HTTP/1.1 connections were clearly the fastest, at 543 ms and 769 ms. There are two reasons:

• A loss affects only one of the six connections. The other five carry on, so there’s no head-of-line blocking between them.
• Congestion control is per connection. A loss cuts the window of one connection, not all six. Together, six connections back off less than one connection would.

That second reason is part of why HTTP/2 and HTTP/3 expect one connection per server: RFC 9113 says clients SHOULD NOT open more than one HTTP/2 connection to the same host and port. Six connections take up to six times the network share of one, at the expense of everyone else on the same links. And they pay six handshakes and use six sets of server resources.

Between HTTP/2 and HTTP/3, there was no clear winner under loss. HTTP/2’s medians were lower, at 760 ms against 968 ms at 1% loss and 1,910 ms against 3,459 ms at 3%. But the runs varied widely, and the confidence intervals for both differences include zero. Two things muddy it further. TCP met fewer, burstier losses than QUIC here, which may favour HTTP/2. And curl’s HTTP/3 support in this build uses OpenSSL’s QUIC implementation, which curl itself labels experimental, so the client may be part of the story.

What the two experiments say together

• Head-of-line blocking is real, and HTTP/3 removes it. Experiment 1 shows it plainly.
• Removing it doesn’t guarantee a faster page. When all objects are small and finish together, as in experiment 2, loss recovery and congestion control decide the total time, and those depend on the implementation.
• So choose by measuring your own traffic with your own stack, not by a protocol’s reputation. HTTP/3 helps most where one connection carries requests that shouldn’t wait for each other, on networks that lose packets.

The limits of this lab

• One server stack and two clients: Caddy with quic-go and Linux TCP, curl for pages, and a Go program for the head-of-line test. Other implementations recover from loss differently.
• Different page-load clients: HTTP/1.1 and HTTP/2 used curl’s mature TCP code, and HTTP/3 used OpenSSL’s experimental QUIC.
• Random loss, not congestion. netem added delay and loss but no bandwidth limit, so congestion control only ever reacted to random drops, never to a full queue. Real networks lose packets when queues fill, often in bursts.
• Loss wasn’t identical across protocols. TCP’s drops mostly came in pairs, QUIC’s singly. The head-of-line runs also show TCP retransmissions with no netem drops at 0% loss, which we haven’t explained.
• One 50 ms path, all on one machine, with delay and loss only on the server’s packets. Requests and client handshake packets were never lost, and acknowledgements travelled back instantly.
• Settings servers don’t use. Offloads and quic-go’s packet batching were off, to make the loss fair.
• Not a browser. Browsers prioritise requests, usually use one connection to an HTTP/2 or HTTP/3 server, and load files of very different sizes.
• 30 runs per case, and many comparisons. Enough for the clear differences above, not enough to rank HTTP/2 and HTTP/3 page loads under loss. A difference that only just clears its interval deserves less trust than the large ones.

So, back to the prediction from the start. On a clean network the order was HTTP/3, six connections, HTTP/2, then one connection. At 3% loss, six connections came first. HTTP/2, HTTP/3 and one connection came after, and 30 runs couldn’t order them with confidence: HTTP/2 was ahead of one connection by a margin that only just cleared its interval, and the other two differences didn’t clear it.

Keep-alive, pools and idle timeouts

Long-lived connections are good for speed, but something along the path always wants to close idle ones. Three different “keep-alives” get confused:

• HTTP persistent connections (HTTP keep-alive): reusing one connection for many requests. HTTP/1.1 does this by default.
• TCP keepalive: small probe packets on an idle connection, to check the other side is still there. RFC 1122 says keepalives “MUST default to off”, and that the interval “MUST default to no less than two hours”. Runtimes vary: Go’s dialer, for example, sends them every 15 seconds by default. Either way, TCP keepalive only covers one TCP connection. Through a proxy or load balancer it keeps the hop to the proxy alive, not the path to the server, and it says nothing about whether the application is healthy.
• Application heartbeats: messages your protocol sends on purpose, such as WebSocket ping and pong frames, or SSE comment lines.

Everything in the middle has idle timeouts:

• NAT devices: RFC 5382 requires an established TCP connection’s idle timeout to be at least 2 hours 4 minutes, and RFC 4787 requires at least 2 minutes for UDP. Real devices vary.
• Load balancers: AWS’s Application Load Balancer closes a connection idle for 60 seconds by default. AWS recommends making your application’s own idle timeout longer than the load balancer’s. Otherwise the load balancer may send a request on a connection the application has just closed. The client then gets a 502 Bad Gateway.

One rule covers all of these: the side that reuses an idle connection must give up on it first. A client’s connection pool, or a load balancer sending to its backends, should close idle connections before the server at the other end does. The server, which only accepts requests on those connections, should keep them open longest.

A connection pool holds open connections for reuse. The settings that matter:

• Maximum connections per destination: too low and requests queue, too high and you overload the server.
• Idle timeout: close pooled connections before the server or load balancer at the other end does, following the rule above. A request sent on a connection the other side just closed fails.
• Maximum lifetime: retire connections after a while, so DNS changes and new servers are picked up. That’s what .NET’s PooledConnectionLifetime does, as Part 1 showed.

Real-time updates: polling, long polling, SSE and WebSockets

HTTP is request and response: the client asks, the server answers. When the server has news the client didn’t ask for, such as a chat message, a price change or a finished job, there are four common ways to get it there. Choose a pattern and watch the same six events reach the browser.

The model has six events over 30 seconds and a 50 ms network trip each way:

Pattern	Requests	Slowest delivery
Polling every 5 s	6, and 2 came back with nothing	4.1 s
Long polling	7, one per event plus the first	130 ms
Server-Sent Events	1, kept open	50 ms
WebSocket	1 connection, kept open	50 ms

Polling: the client asks every few seconds. It’s the simplest to build, and it works through every proxy and cache. But delivery can be up to one interval late, and any request made when nothing has changed comes back empty: two of six in this model, and most of them when news is rare. Shorter intervals trade lateness for load.

Long polling: the client asks, and the server holds the request open until it has news or a timeout passes, then answers. The client immediately asks again. Events usually arrive within one network trip, and it’s still plain HTTP. RFC 6202 lists the costs, including header overhead on every message, timeouts on intermediaries, and latency: “while the average latency of long polling is close to one network transit, the maximal latency is over three network transits”. That worst case is news arriving just after a response left, which has to wait for the response to travel, the next request to arrive, and a new response to travel. In the model, the event at 16.02 s hits that gap. So the client should tell the server the last event it saw, so nothing is skipped across the gap.

Server-Sent Events (SSE): the client opens one request, and the server keeps the response open, writing each event into it as text (Content-Type: text/event-stream). It’s defined in the WHATWG HTML standard, and browsers provide the EventSource API. Its strengths:

• It’s still HTTP, so it works with cookies and with HTTP/2 multiplexing. One catch: the browser’s EventSource API can’t set request headers, so an Authorization: Bearer token can’t be sent that way. Its only option is withCredentials, for cookies.
• It reconnects automatically. The browser sends the last event ID it saw in a Last-Event-ID header, so the server can resume.

Its limits:

• It’s one-way: server to client only, and text only.
• Over HTTP/1.1 each open stream uses one of the browser’s few connections per host. The specification itself warns that clients “might run into trouble when opening multiple pages from a site”. Over HTTP/2 that problem goes away.
• Old proxies may cut idle responses. The specification suggests a comment line “every 15 seconds or so”.

WebSockets (RFC 6455): the client sends an HTTP request asking to upgrade, the server answers 101 Switching Protocols, and from then on the connection carries messages in both directions, framed very lightly. A frame’s header is 2 to 14 bytes. A 20-byte message from server to client takes 22 bytes of WebSocket framing, and 26 from client to server, because clients must mask every frame they send, which adds 4 bytes. TLS, TCP and IP add their own headers on top. WebSockets can also run over one stream of an HTTP/2 connection (RFC 8441) or HTTP/3 (RFC 9220).

What long-lived connections do to a design

SSE and WebSockets are cheap per message, but each open connection is state on a server. That changes the architecture:

• Capacity is about concurrent connections, not just requests per second. A million connected users is a million open connections somewhere, each with memory, buffers and a file descriptor.
• Load balancing gets sticky. A connection stays on the server it reached, so a deploy or a failed server disconnects everyone on it at once. Clients reconnecting together can overload the servers that remain: a reconnect storm. Reconnect with random delays.
• Proxies have port limits too. A reverse proxy or load balancer usually opens one connection to a backend for each WebSocket, and each proxy address to backend address and port pair is limited by the proxy’s local ports. So a million connections need many backend addresses, or many proxy addresses.
• Fan-out needs a backbone. A message for a user must reach whichever server holds that user’s connection, usually through a publish-subscribe system. Part 42, the chat system, designs this.
• Slow clients need backpressure. If a client reads slower than messages arrive, the server’s buffer for it grows. Decide in advance whether to drop, merge or disconnect.

Explain it like I’m ten

You’re waiting for a parcel.

• Polling is walking to the door every five minutes to check. Most trips, there’s nothing there.
• Long polling is waiting at the door until the delivery person comes, then going back to wait again.
• Server-Sent Events is leaving the door open so the delivery person can drop off parcels whenever they arrive, but you can’t hand anything back.
• WebSockets is a hatch in the door that both of you can pass things through, any time.

The precise version

• Polling: repeated independent requests on a timer.
• Long polling: a request the server delays answering until data exists, then repeated.
• SSE: one long-running HTTP response carrying a text/event-stream, with automatic reconnection and resumption by event ID.
• WebSocket: a connection upgraded from HTTP into a two-way message channel with its own framing, ping/pong control frames, and a closing handshake.

Where the analogy breaks: all four still use a network connection that can drop. An open door isn’t a promise that the parcel arrives. SSE’s reconnection with Last-Event-ID and a WebSocket application’s own resume logic are what make delivery reliable across reconnects.

Across languages

HTTP client libraries in C#, Java, Go and Rust differ in which HTTP versions they support out of the box, and those differences shape designs:

Language	HTTP/2	HTTP/3
C# (HttpClient)	Supported; request it with Version and VersionPolicy	Supported through MsQuic when the platform has it. Microsoft recommends allowing fallback to HTTP/1.1 and HTTP/2
Java (java.net.http.HttpClient)	HttpClient.Version.HTTP_2, the default	Not in JDK 25, the current LTS: its enum has only HTTP_1_1 and HTTP_2. JDK 26 added HTTP_3 as an opt-in (JEP 517)
Go (net/http)	Automatic over HTTPS for servers and the default transport. A custom Transport with its own dialer or TLS config needs ForceAttemptHTTP2	Not in the standard library
Rust (reqwest)	The http2 feature, on by default	An http3 feature marked unstable, behind an extra opt-in flag

Whatever the language, the design rules are the same: reuse clients so connections are pooled, set idle timeouts and lifetimes on purpose, and don’t assume HTTP/3 is available on every network.

Trade-offs

One connection versus several. Multiplexing saves handshakes and shares one congestion window, and it puts every stream behind the same losses. Several connections isolate losses and multiply the initial window, and each one pays its handshakes and uses server resources.

TCP versus QUIC. QUIC removes cross-stream head-of-line blocking and a handshake round trip, and connections survive network changes. It costs user-space CPU, a fallback path for networks that block UDP, and less visibility for network tools.

Reliability versus freshness. TCP and QUIC wait for lost data. For live media and game state, waiting makes things worse, and UDP with application-level handling fits better.

Push versus poll. Open connections deliver news quickly and cheaply per message. They cost server state, sticky routing and reconnect handling. Polling is stateless and easy to scale. It is also late, and wasteful when news is rare.

Common mistakes

Assuming the newer protocol is always faster. In our lab, six HTTP/1.1 connections loaded 20 objects faster than one HTTP/2 connection, with or without loss, because they grow six windows and lose them separately. And HTTP/3 removed head-of-line blocking, yet didn’t load the page clearly faster than HTTP/2 under loss; its medians were actually slower. Measure with your own traffic.

Opening a connection per request between services. Handshakes, slow start and, at high rates, running out of local ports in TIME-WAIT. Use a connection pool.

Setting the backend’s idle timeout shorter than the load balancer’s. The load balancer reuses a connection the backend just closed, and users see 502s.

Relying on TCP keepalive to keep connections alive through proxies. It’s off by default in the standard, its standard default interval is two hours, and it only covers one hop. Use application heartbeats.

Choosing WebSockets for one-way updates. If only the server sends, SSE is simpler, uses plain HTTP, and reconnects by itself.

Reconnecting all clients at once after a deploy. Add random delays, and reject new connections gracefully when overloaded. Otherwise the reconnect storm can take down the servers that are left.

Forgetting that UDP may be blocked. Anything built on QUIC needs a TCP fallback.

Interview questions

These questions test the networking behind common design choices. Try answering each one out loud before you open the answer.

1. What is head-of-line blocking, and does HTTP/2 fix it?

Show a strong answer

Head-of-line blocking is when one delayed item holds up the items queued behind it. There are two kinds:

• HTTP-level (HTTP/1.1): on one connection, responses come back in request order, so a slow response blocks the next. Browsers work around it with several connections.
• TCP-level: TCP delivers bytes in order, so one lost packet holds up every byte after it until the resend arrives.

HTTP/2 fixes the HTTP-level kind by multiplexing streams on one connection. It does not fix the TCP kind; RFC 9113 says so explicitly. With many streams on one connection, a single loss stalls all of them.

HTTP/3 over QUIC fixes both across streams, because QUIC keeps order per stream: a loss only blocks the streams whose data was in the lost packet. Order still matters within a stream, and QPACK header compression can make a stream wait if the encoder allows it.

Likely follow-up: “Then why isn’t everyone on HTTP/3?” UDP blocking on some networks, discovery through Alt-Svc, user-space CPU cost, and tooling.

2. When would you choose UDP over TCP?

Show a strong answer

When late data is worse than lost data, or when you need control TCP doesn’t give:

• Real-time media and games: a resent audio frame or position update arrives too late to use.
• Small request-response exchanges: DNS, with TCP as a fallback for large answers.
• Building a custom transport: QUIC runs on UDP to implement its own streams, reliability and encryption in user space.

Then say what you take on: loss handling where it matters, ordering if needed, and congestion control. RFC 8085 says applications sending UDP should control their sending rate themselves. Also mention that some networks block UDP.

Likely follow-up: “How would you make a UDP-based protocol reliable?” Sequence numbers, acknowledgements, retransmission timers and congestion control, which is essentially re-deriving TCP or using QUIC.

3. Why is a large download slow at the start of a new TCP connection?

Show a strong answer

Two reasons:

1. Handshakes: TCP plus TLS take round trips before any data (Part 1).
2. Slow start: TCP doesn’t know the path’s capacity, so it starts with a small congestion window, commonly 10 segments (about 14.6 KB), and roughly doubles it every round trip while no loss is seen. A 1 MB response needs about 7 round trips of data from a cold start, with no loss.

Consequences for design:

• Reuse connections, which skip the handshakes, and keep a grown window while they stay busy. After an idle period, Linux shrinks the window again by default.
• Put critical content early. The first round trip carries only about 14 KB.
• Serve large files from nearby, because each round trip costs more on long paths.

Likely follow-up: “What happens to the window on packet loss?” It’s reduced, and the connection sends more slowly until it grows again.

4. A service calls another service thousands of times a second and starts failing with “cannot assign requested address”. What’s happening?

Show a strong answer

It’s almost certainly running out of local ports. Each new outgoing TCP connection to the same destination address and port needs a free local port. After closing, the side that closed first keeps the connection in TIME-WAIT, 60 seconds on Linux, holding that port. Check who closes first, and count the waiting connections with ss -tan state time-wait.

With Linux’s default range of 28,232 ports, that caps new connections to one destination at about 471 per second if every connection is closed after use. Above that, connect fails.

Fixes, best first:

1. Reuse connections with a pool, so you open a handful, not thousands per second. The usual causes are creating a new HTTP client per call, or not reading response bodies to the end, which stops some clients reusing connections.
2. Use HTTP/2 to the downstream, so many requests share one connection.
3. Spread over more destination addresses if you truly need many connections.
4. Tune the kernel, a wider port range or TIME-WAIT reuse for outgoing connections, only with care. Don’t reach for tcp_tw_recycle: it no longer exists.

Likely follow-up: “How do you size the pool?” From the downstream’s capacity and your concurrency, and watch both queue time in the pool and errors.

5. How would you push live updates to a web page? Compare the options.

Show a strong answer

	Direction	Latency	Cost	Notes
Polling	client asks	up to one interval	many empty requests	simplest; cache and proxy friendly
Long polling	server answers when ready	usually one network trip, worst over three	a request per event	plain HTTP; timeouts and gaps need care
SSE	server to client	about one network trip	one open response	auto-reconnect with Last-Event-ID; text only; browsers can’t add custom headers; best over HTTP/2
WebSocket	both ways	about one network trip	one open connection	light framing; needs its own resume logic

Pick by need: one-way updates such as notifications or dashboards, use SSE. Two-way and interactive, such as chat, collaboration or games, use WebSockets. Rare updates, or strict proxies, use polling.

Then cover what open connections do to the backend: concurrent connection capacity, sticky routing, a publish-subscribe layer to reach the right server, heartbeats for idle timeouts, backpressure, and jittered reconnects after deploys.

Likely follow-up: “How do you deliver a message to a user connected to a different server?” A publish-subscribe backbone keyed by user or channel (Part 42).

6. What are the advantages of QUIC over TCP with TLS?

Show a strong answer

• No cross-stream head-of-line blocking: order is per stream.
• Faster setup: transport and TLS 1.3 handshakes combined, so one round trip in the common case, or zero with 0-RTT for returning clients, with replay limits. A large certificate chain can add a round trip, because of QUIC’s limit on how much a server sends before the client’s address is confirmed.
• Connection migration: connection IDs let a connection survive IP or port changes, such as Wi-Fi to mobile, if the client, server and load balancer support it.
• Clearer loss detection: a new packet number for every transmission avoids ambiguity about retransmissions.
• Encrypted transport headers: less for middleboxes to interfere with, and easier to evolve.

And the costs: some networks block UDP (3% to 5% in the studies RFC 9308 cites), so you need a fallback; CPU cost of a user-space implementation; less visibility for network operators.

Likely follow-up: “Would QUIC always beat HTTP/2?” No. In this post’s lab, HTTP/3 kept small requests from waiting long behind a big download, but a page of 20 equal files didn’t load clearly faster than over HTTP/2 when packets were lost. Implementations, congestion control settings and loss patterns matter, so measure with your own traffic.

7. What’s the difference between HTTP keep-alive and TCP keepalive?

Show a strong answer

• HTTP keep-alive (persistent connections): reusing a TCP connection for many HTTP requests. It saves handshakes and slow start, and it’s the default in HTTP/1.1.
• TCP keepalive: optional probe packets on an idle TCP connection, to detect that the peer is gone. RFC 1122 requires the default interval to be at least two hours.

For long-lived connections through NATs, proxies and load balancers, TCP keepalive isn’t enough: it’s off by default in the standard, slow when on, and it covers only one hop. Use application-level heartbeats (WebSocket pings, SSE comments).

For idle timeouts, use one rule: the side that reuses an idle connection gives up on it first. A client pool closes idle connections before the load balancer does, and the load balancer closes them before the backend does.

Likely follow-up: “Why do we see 502s from the load balancer after idle periods?” The backend closed an idle connection that the load balancer still thought was open. Make the backend’s idle timeout longer than the load balancer’s.

8. You’re designing a chat backend for one million concurrent users on WebSockets. What networking concerns do you raise?

Show a strong answer

• Connections, not requests: a million open connections, each using memory and a file descriptor. Estimate per-connection memory, raise file-descriptor limits, and plan servers by concurrent connections.
• Ports at the proxy: a load balancer that opens one backend connection per WebSocket is limited by local ports per backend address and port, so spread connections over many backend addresses.
• TLS on reconnect: a mass reconnect is also a burst of TLS handshakes, which costs CPU.
• Routing: long-lived connections stick to a server. Use a load balancer that supports WebSockets, and a publish-subscribe backbone so a message reaches the server holding the recipient’s connection.
• Deploys and failures: draining connections gradually, reconnecting clients with random delays, and resuming from the last message ID so nothing is lost.
• Idle timeouts and heartbeats: ping/pong below the load balancer’s and NATs’ idle timeouts.
• Backpressure: bounded per-connection buffers, and a policy for slow clients.
• Fallbacks: networks that block WebSockets, and mobile clients switching networks.

Likely follow-up: “How many servers?” Estimate from connections per server, measured under load, plus headroom for losing servers during deploys (Part 2’s method).

Sources

• IETF, RFC 768: User Datagram Protocol and RFC 8085: UDP Usage Guidelines
• IETF, RFC 9293: TCP, RFC 5681: TCP Congestion Control, RFC 6928: Increasing TCP’s Initial Window, RFC 6298: Computing TCP’s Retransmission Timer and RFC 8985: RACK-TLP
• IETF, RFC 1122: Requirements for Internet Hosts (TCP keepalive), RFC 4787 and RFC 5382 (NAT timeouts)
• IETF, RFC 9112: HTTP/1.1, RFC 9113: HTTP/2, RFC 7541: HPACK and RFC 9114: HTTP/3
• IETF, RFC 9000: QUIC, RFC 9001: Using TLS to Secure QUIC, RFC 9002: QUIC Loss Detection and Congestion Control, RFC 9204: QPACK and RFC 9308: Applicability of QUIC
• IETF, RFC 7766: DNS Transport over TCP, RFC 7838: HTTP Alternative Services and RFC 9460: SVCB and HTTPS DNS records
• IETF, RFC 6455: The WebSocket Protocol, RFC 8441: WebSockets with HTTP/2, RFC 9220: WebSockets with HTTP/3 and RFC 6202: Known Issues with Long Polling and Streaming
• WHATWG, HTML Standard: Server-sent events
• Linux kernel, include/net/tcp.h (TCP_TIMEWAIT_LEN, TCP_INIT_CWND) and Documentation/networking/ip-sysctl.rst (ip_local_port_range, tcp_tw_reuse)
• Chrome for Developers, Remove HTTP/2 Server Push from Chrome
• AWS, Application Load Balancer attributes: connection idle timeout
• Microsoft, Use HTTP/3 with HttpClient; Oracle, java.net.http.HttpClient.Version in JDK 25, and OpenJDK, JEP 517: HTTP/3 for the HTTP Client API; the Go project, net/http: HTTP/2 and net.Dialer.KeepAlive; reqwest, crate documentation (feature flags)
• Linux kernel, commit 4396e46: tcp: remove tcp_tw_recycle
• Checks: system-design/checks/part03_numbers.py (slow start, ports, real-time and head-of-line models, checked against the lab) and system-design/checks/part03_lab/ (the Docker lab)

What to remember

• TCP delivers bytes in order, so one lost packet holds up everything behind it, including other requests on the same connection.
• A new TCP connection grows its window from about 14.6 KB, doubling each round trip. Reused connections skip the handshakes, and keep a grown window while they stay busy.
• HTTP/2 multiplexes requests on one TCP connection, which fixes HTTP’s queueing but not TCP’s head-of-line blocking. HTTP/3 uses QUIC, whose streams don’t block each other.
• UDP gives no guarantees on purpose. Use it when late data is useless, or to build your own transport, and handle congestion yourself.
• Opening a connection per call wastes handshakes and can exhaust local ports: about 471 new connections per second to one destination with Linux defaults.
• For live updates: polling is simple and late, long polling is quick but costs a request per event, SSE is one-way and reconnects itself, and WebSockets are two-way. Open connections are server state, so plan for sticky routing, heartbeats and reconnect storms.

For every protocol choice, ask what happens when a packet is lost, and measure the answer.