TCP, IP, and UDP are the foundational protocols that move data across the internet. TCP handles reliable, ordered delivery with connection setup and flow control, while UDP trades reliability for speed with its connectionless, fire-and-forget model. Understanding when each protocol is appropriate — and how QUIC is changing the equation — is essential for anyone building networked systems. This post covers the layered model, how the three-way handshake works, flow and congestion control, and practical guidance on choosing between TCP and UDP for your application.

Introduction

Networking uses a layered model. Each layer handles specific responsibilities:

graph TB
    A[Application Layer<br/>HTTP, DNS, SMTP] --> B[Transport Layer<br/>TCP, UDP]
    B --> C[Internet Layer<br/>IP]
    C --> D[Link Layer<br/>Ethernet, WiFi]

IP handles addressing and routing. TCP and UDP sit on top of IP, adding their own features. You rarely choose between TCP and UDP directly; you choose an application protocol that uses one or the other.

---

TCP Protocol

TCP: Transmission Control Protocol

TCP is the workhorse protocol of the internet. It gives applications a stream abstraction that looks like a plain byte pipe, but underneath the kernel is doing a lot of work: ordering packets, retransmitting losses, throttling the sender when the network is congested, and verifying that data arrived intact. The application writes bytes, the kernel decides how to ship them.

Before any byte of application data flows, TCP negotiates a connection through a three-way handshake. After the connection is up, every segment carries a sequence number so the receiver can reassemble the original stream even when packets arrive out of order. When the conversation ends, either side initiates a clean shutdown with a FIN exchange.

The cost of all this reliability is latency and overhead. A TCP segment carries a 20-byte header (60 with options), and the connection setup adds at least one round-trip before the first application byte can be sent. The sections below break down each piece of the contract: the handshake, reliable delivery, flow control, and congestion control.

The Three-Way Handshake

TCP uses a three-way handshake to establish connections:

sequenceDiagram
    participant Client
    participant Server
    Client->>Server: SYN (seq=x)
    Server->>Client: SYN-ACK (seq=y, ack=x+1)
    Client->>Server: ACK (ack=y+1)
    Note over Client,Server: Connection established!

1. Client sends SYN with a sequence number
2. Server responds with SYN-ACK, acknowledging the client’s sequence number and sending its own sequence number
3. Client sends ACK, acknowledging the server’s sequence number

This takes a full round trip before any data can be sent. HTTPS adds TLS on top, requiring more round trips.

Reliable Data Transfer

TCP guarantees that data arrives intact and in order. It does this through:

• Acknowledgments (ACKs) - Receiver confirms receipt of data
• Sequence numbers - Data is numbered so the receiver can reorder out-of-order packets
• Retransmission - If data is not acknowledged, it is retransmitted

// TCP guarantees (simplified)
const sender = {
  sequence: 0,
  send(data) {
    const packet = { data, sequence: this.sequence };
    this.sequence += data.length;
    return packet;
  },
};

const receiver = {
  expectedSequence: 0,
  receive(packet) {
    if (packet.sequence === this.expectedSequence) {
      this.expectedSequence += packet.data.length;
      return { ack: packet.sequence + packet.data.length };
    }
    // Out of order - request retransmission
    return { ack: this.expectedSequence };
  },
};

Flow Control

TCP prevents the sender from overwhelming the receiver. The receiver advertises a window size indicating how much buffer space it has. The sender cannot send more than this window without receiving acknowledgments.

// Flow control window
const receiver = {
  bufferSize: 65535,
  usedBuffer: 0,
  windowSize() {
    return this.bufferSize - this.usedBuffer;
  },
};

Congestion Control

TCP also prevents overwhelming the network. It uses algorithms like slow start, congestion avoidance, and fast retransmit to dynamically adjust sending rate.

graph LR
    A[Slow Start] --> B[Congestion<br/>Avoidance]
    A -->|"packet loss"| C[Reduce Rate]
    B -->|"packet loss"| C
    C --> A

Slow start begins with a small window and exponentially increases it until packets are lost. This probing helps TCP find the available bandwidth without causing congestion.

---

UDP Protocol

UDP: User Datagram Protocol

UDP is the minimalist sibling of TCP. There is no connection, no handshake, no acknowledgment, no retransmission, no ordering guarantee. The application hands a datagram to the kernel, the kernel puts it on the wire, and that’s the end of the story. If the packet arrives out of order, duplicated, or not at all, the application finds out only by noticing the gap.

The 8-byte UDP header carries almost nothing: source port, destination port, length, and checksum (optional in IPv4, mandatory in IPv6). The kernel does not even reserve any per-connection state, so a single UDP socket can talk to thousands of peers without a corresponding explosion of socket buffers. That property makes UDP the natural fit for one-to-many multicast and for protocols that manage their own reliability on top.

The trade is that UDP is best understood as a thin wrapper around the IP layer, not a transport in the same sense as TCP. Sections below cover the characteristics, the wire format, and the question of when to choose UDP over TCP or QUIC.

UDP Characteristics

• No connection establishment (zero latency)
• No ordering or sequencing
• No retransmission of lost packets
• Small header overhead (8 bytes vs TCP’s 20+ bytes)

// UDP is simple
const sender = {
  send(data, address) {
    const datagram = { data, destination: address };
    // Send and forget - no acknowledgment
    return datagram;
  },
};

const receiver = {
  receive(datagram) {
    // Handle datagram - might be duplicate, might be missing
    return datagram.data;
  },
};

UDP Header

UDP has a minimal header:

+----------------+----------------+----------------+----------------+
| Source Port    | Dest Port     | Length        | Checksum       |
+----------------+----------------+----------------+----------------+

Four 16-bit fields. Source port is optional (set to 0 if not used). Length includes header and data. Checksum for error detection.

---

TCP vs UDP Comparison

Feature Comparison Table

Feature	TCP	UDP
Connection	Connection-oriented	Connectionless
Reliability	Guaranteed delivery	Best effort
Ordering	In-order delivery	No ordering
Speed	Slower (handshake, ACKs)	Faster (no overhead)
Header size	20+ bytes	8 bytes
Flow control	Yes	No
Congestion control	Yes	No

When to Use TCP

TCP is the right choice when:

• You need all data to arrive intact
• Order matters (files, messages, documents)
• You can tolerate some latency
• You are building HTTP servers, email, file transfer

Most web traffic uses TCP. The reliability guarantees mean you do not have to handle missing or duplicate data yourself.

When to Use UDP

UDP works well when:

• Speed matters more than reliability
• Real-time applications (voice, video, gaming)
• You want minimal overhead
• Application-level error handling is sufficient
• Multicast or broadcast is needed

// Good UDP use cases
const videoStream = {
  protocol: "UDP",
  // Missing frames are less noticeable than delay
  // Accept some packet loss for real-time playback
};

const voiceCall = {
  protocol: "UDP",
  // Prefer hearing the other person with small gaps
  // Over hearing them perfectly but delayed
};

const dnsQuery = {
  protocol: "UDP",
  // Fast lookup matters more than perfect reliability
  // DNS servers retry if no response
};

Port Numbers

Both TCP and UDP use port numbers to multiplex connections. Ports range from 0 to 65535. Well-known ports (0-1023) are reserved for common services:

Port	Service	Protocol
80	HTTP	TCP
443	HTTPS	TCP
53	DNS	UDP (also TCP)
22	SSH	TCP
25	SMTP	TCP

Your application can use any port above 1024. Node.js http.createServer() defaults to port 3000, for example.

---

Topic-Specific Deep Dives

Common Misconceptions

TCP and UDP each get a small set of folk beliefs attached to them. The beliefs are not wrong in their original context, but they get over-applied in systems where they no longer hold. Most of them survive because training material and interview prep love simple rules, so “TCP is reliable, UDP is fast” stays in circulation long after the original caveats got dropped. Conference talks repeat the same framing to save slide time, and vendor marketing of UDP-based products tends to overstate the speed wins. Once a folk belief is in the air, engineers pass it along without ever having to defend it in production.

What makes these myths dangerous is that they often work in the lab and break in production. A test on clean localhost shows UDP blasting data while TCP dawdles through its handshake. A demo streaming local video files works fine on TCP. The gap between demo behavior and real network behavior is where the trouble starts. The sub-sections below look at each myth, trace it back to the conditions that made it true, and describe the conditions where it stops holding.

A useful mental shift when reading claims about protocols: stop asking “which protocol is faster” and start asking “faster under what conditions, measured how, and at what cost to other things I care about.” Reliability, fairness to other flows, NAT traversal, and deployment complexity all shift the answer. The rest of this post covers the building blocks (handshake overhead, congestion control, head-of-line blocking, transport-layer encryption) you will need to actually answer that question for your own system.

”UDP is always faster”

UDP avoids the TCP handshake and per-segment acknowledgment overhead, so in a lab environment it is plainly faster. Real networks change the equation. When packets disappear on the public internet, UDP and TCP diverge sharply. UDP has no congestion control — lost packets vanish, the sender does not slow down, and the receiver gets a stream with gaps. TCP throttles its sending rate on loss, which sounds like a downside but actually limits the damage a lossy link does to the rest of your transfer. On a link running 5% packet loss, a UDP flow can deliver less useful data per second than a TCP flow that has already adapted its window. The speed advantage holds on clean LANs. It stops holding once you are on a variable-latency link full of middle boxes, or once your UDP traffic triggers ISP rate-limiting because something else on the same path was abusing UDP.

The comparison also depends on what you are measuring. UDP latency is lower because there is no handshake and no acknowledgment round trip before the first byte arrives — relevant for DNS queries and real-time gaming where one RTT is a noticeable fraction of the interaction. UDP throughput can be higher in controlled environments because the sender never voluntarily slows down. But if you need reliability and build it on top of UDP, you are now tracking sequence numbers, processing acknowledgments, and handling retransmission. At that point you have reimplemented TCP and lost whatever speed lead you had.

”TCP is for files, UDP is for video”

This rule is a relic of how video delivery evolved, not a technical constraint. Netflix, YouTube, and most streaming platforms deliver video over TCP. The reasoning is not complicated: a buffered stream already absorbs temporary network variability, a dropped frame in the past is gone no matter what the transport does, and TCP infrastructure is everywhere and well-understood. YouTube uses DASH — Dynamic Adaptive Streaming over HTTP — which breaks video into small segments delivered over regular HTTPS, which is TCP-backed. The occasional rebuffering users experience is almost never the transport misbehaving; it is the adaptive bitrate algorithm picking a quality level the connection cannot sustain right now.

Real-time video calls are different. Frames need to arrive within tens of milliseconds to feel responsive — waiting for a retransmit is worse than a lower-resolution frame. That is where UDP enters. But calling Zoom or Google Meet “UDP applications” is reductive. WebRTC, for example, runs media over SRTP on top of UDP and also specifies NACK (negative acknowledgment) for missing packets, forward error correction (FEC) to repair small losses without a retransmit, and jitter buffering to reorder and delimit the stream before it hits the decoder. RFC 8834 spells this out. The reliability layer is not optional add-on software — it is part of the protocol stack. UDP is the transport baseline WebRTC inherits, not the whole story.

”You always choose between TCP and UDP”

Most of the time the choice is made for you by the application protocol you are using. HTTP/1.1 and HTTP/2 are TCP by design — the browser owns the connection, and HTTP semantics assume reliable in-order delivery. gRPC runs over HTTP/2, which means TCP. AMQP, SMTP, SSH, IMAP — all TCP. The list is long enough that most developers never explicitly choose.

DNS is the exception worth knowing. A standard query goes out over UDP because one request-response fits in a single packet and a TCP handshake would double the lookup time. When a response exceeds the traditional 512-byte UDP limit (extended via EDNS), the server sets a truncate flag and the client reconnects over TCP. The DNS protocol itself decides when to switch based on response size — the application developer does not make that call.

WebRTC is the most layered case. Real-time media travels over SRTP on top of UDP. The call setup — session descriptions, ICE candidates, peer coordinates — runs over WebSocket with TLS, which is TCP. One session, two transports. The protocol stack assigns each role its appropriate transport, not a single global choice. This matters when you are debugging a failed WebRTC connection or designing your own protocol: the transport decision should follow from what your data actually needs to guarantee.

---

Connecting It All Together

The layers build on each other:

graph TB
    A[Your Application] --> B[HTTP over TCP]
    B --> C[TCP over IP]
    C --> D[IP over Ethernet]
    D --> E[Physical Network]

Each layer encapsulates the one below it. Your HTTP request becomes a TCP segment, then an IP packet, then an Ethernet frame.

---

QUIC Protocol

QUIC (RFC 9000) runs over UDP and layers TCP’s reliability on top of UDP’s speed. Google built it to get around TCP’s worst inefficiencies, and it’s now the transport behind HTTP/3.

Why QUIC Exists

TCP has three annoying problems QUIC actually solves:

1. Head-of-line blocking: Lose one TCP packet and everything behind it waits—even if other streams could use the bandwidth
2. Handshake latency: TCP needs 1 RTT before you even start TLS, then TLS needs another 1-2 RTTs
3. Congestion control rigidity: TCP’s algorithms live in the OS kernel—you can’t ship a new one without an OS update

QUIC Handshake vs TCP+TLS

sequenceDiagram
    participant Client
    participant Server
    Note over Client,Server: TCP+TLS (TLS 1.3 takes ~1.5 RTT before data)
    Client->>Server: TCP SYN
    Server->>Client: SYN-ACK
    Client->>Server: TCP ACK
    Note over Client,Server: TLS Handshake starts...
    Client->>Server: ClientHello
    Server->>Client: ServerHello
    Client->>Server: Finished
    Note over Client,Server: Data ready at ~1.5 RTT

    Note over Client,Server: QUIC (1 RTT, crypto integrated into transport)
    Client->>Server: QUIC Initial (crypto handshake)
    Server->>Client: QUIC Initial (crypto + data)
    Client->>Server: QUIC Handshake Finished
    Note over Client,Server: Data ready at ~1 RTT

QUIC folds the crypto handshake into the transport layer. That first packet from the client already carries encrypted application data—no more waiting for separate TLS round trips.

QUIC Multi-Stream Advantage

HTTP/1.1 and HTTP/2 both multiplex multiple streams over a single TCP connection. That works until one packet drops—then TCP holds everything until that packet gets retransmitted. QUIC gives each stream its own stream ID:

graph TB
    subgraph "TCP (HTTP/2)"
        A[Stream 1]:::blocked
        B[Stream 2]:::blocked
        C[Stream 3]:::blocked
    end
    subgraph "QUIC (HTTP/3)"
        D[Stream 1]
        E[Stream 2]
        F[Stream 3]
    end
    D --> E --> F

When a QUIC packet drops, only the stream that owns that packet stalls. Every other stream keeps running.

0-RTT Resumption

If you’ve connected to a server before, QUIC can skip the handshake entirely:

1. Client attaches Early Data with a session ticket from the previous visit
2. Server decrypts and accepts the data immediately—no waiting
3. You start sending application data in the very first packet

---

QUIC vs TCP Comparison

Comparison Table

Feature	TCP+TLS 1.3	QUIC (RFC 9000)
Handshake RTT	1-2 RTTs	1 RTT (0-RTT on resumption)
Head-of-line blocking	Yes (TCP)	No (stream-level)
Connection migration	Breaks—IP change kills it	Survives address change
Protocol evolution	Kernel-level (slow)	Userspace (ship anytime)
Encryption	TLS (application)	Built into transport
Flow control	Per-connection	Per-stream

When to Choose QUIC

• Mobile clients switching between WiFi and cellular (connection sticks around)
• Applications that care about latency (HTTP/3, WebRTC data channels)
• High packet loss environments (stream isolation prevents a single loss from cascading)
• If you’re deploying HTTP/3 via CDN, QUIC comes with it automatically

---

TCP Header Deep Dive

The TCP header has a minimum 20 bytes but can expand with options:

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |       Destination Port        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Sequence Number                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Acknowledgment Number                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Offset | Flags  | Window Size                                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|         Checksum              |         Urgent Pointer          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Options (variable length)                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Key Fields

Sequence Number: Byte position of the first data octet in this segment. Enables in-order reconstruction and retransmission identification.

Acknowledgment Number: Next expected sequence number. All data up to this number has been received.

Flags: Nine control bits including SYN (connection establishment), ACK (acknowledgment valid), FIN (graceful termination), RST (abort), PSH (push data to application).

Window Size: How many bytes the receiver accepts. The 16-bit field gets multiplied by the window scaling factor (up to 2^14x) for high-BDP links—critical for 10 Gbps networks with 100ms RTT.

TCP Options (Partial List)

Kind	Length	Name	Purpose
0	1	EOL	End of option list
1	1	NOP	Padding
2	4	MSS	Max segment size
3	3	Window Scaling	Shift count (up to 2^14)
8	10	Timestamps	RTTM and PAWS
4	2	SACK Permitted	Selective acknowledgment allowed
5	Variable	SACK Block	Selective ACK ranges (if negotiated)

Window Scaling (RFC 7323)

On high-bandwidth, high-delay networks, a 65KB window is insufficient:

Link: 10 Gbps, 100ms RTT
Required window: 10 Gbps × 0.1s = 1 GB
Max window with scaling: 65535 × 2^14 = 1 GB

The window scaling option shifts the 16-bit window left by the scale factor (0-14).

---

NAT Traversal Guide

Network Address Translation (NAT) enables multiple devices to share one public IP. But NAT breaks end-to-end connectivity, which matters for UDP-based protocols like QUIC and WebRTC.

How NAT Works

Private Network (192.168.1.x)          NAT Device           Internet
                                      203.0.113.5 (public)
+----------+                          +-----------+          +--------+
| Client   | 192.168.1.100:5000  -->   | PAT entry |  -->     | Server |
|          | <--   203.0.113.5:5001    | (port map)|  <--    |        |
+----------+                          +-----------+          +--------+

The NAT device tracks source IP, source port, destination IP, destination port. When the response returns, it reverses the mapping.

NAT Types and Traversal Difficulty

NAT Type	Behavior	Traversal Difficulty
Full Cone	Any external host can connect	Easiest
Restricted	Only contacted external hosts	Moderate
Port Restricted	Only contacted ext host+port	Hard
Symmetric	Different mapping per destination	Hardest

STUN and TURN

STUN (RFC 8489): Lets clients discover their public IP:port mappings. Works with Full Cone and some Restricted NATs.

// STUN request (simplified)
const stunServer = "stun:stun.l.google.com:19302";
// Client sends binding request
// Server responds with mapped address

TURN (RFC 8656): Relay server for symmetric NATs. All traffic routes through the TURN server—higher latency but always works.

sequenceDiagram
    participant Client
    participant TURN
    participant Target
    Client->>TURN: Allocate request
    TURN->>Client: XOR-Mapped-Address
    Client->>TURN: Send to Target
    TURN->>Target: Data from Client's public IP
    Target->>TURN: Response
    TURN->>Client: Data

NAT Keepalive

NAT mappings expire. Servers send keepalive packets to maintain sessions:

# TCP keepalive (Linux)
echo 60 > /proc/sys/net/ipv4/tcp_keepalive_time     # Start after 60s idle
echo 10 > /proc/sys/net/ipv4/tcp_keepalive_intvl   # Send every 10s
echo 3 > /proc/sys/net/ipv4/tcp_keepalive_probes   # Drop after 3 failures

# Application-level UDP keepalive for QUIC
quic_conn.sendHeartbeat()  # Every 30 seconds typically

QUIC Connection Migration

QUIC handles NAT issues elegantly via connection migration—when your mobile switches from WiFi to cellular, QUIC can continue on the new path using the same connection ID:

// QUIC connection migration (conceptual)
if (networkChanged) {
  const newPath = { address: newIP, port: newPort };
  quicConn.migrate(newPath); // Same connection ID, new path
}

---

TCP Performance Tuning

Stock Linux settings often limit performance on modern networks. Tuning can dramatically improve throughput.

Socket Buffer Sizes

Default buffers are too small for high-BDP networks:

# View current TCP buffer settings
# min / default / max (bytes)
cat /proc/sys/net/ipv4/tcp_rmem   # Receive buffer
cat /proc/sys/net/ipv4/tcp_wmem   # Send buffer

# Example output:
# 4096    16384   6291456

# Set autotuning to higher maximums
echo "6291456 25165824 134217728" > /proc/sys/net/ipv4/tcp_rmem
echo "3145728 12582912 67108864" > /proc/sys/net/ipv4/tcp_wmem

TCP Congestion Control

Linux supports multiple congestion control algorithms:

# List available algorithms
sysctl net.ipv4.tcp_available_congestion_control

# Example output: reno cubic bbr

Algorithm	Best For	Characteristics
cubic	General purpose (default)	Fixed targeting, Reno replacement
bbr	High-BDP, variable networks	Maximizes throughput, not fair to Reno
reno	Legacy, simple networks	Avoids cwnd halving on partial losses
vegas	Low-latency preference	Proactive queue detection

BBR (Bottleneck Bandwidth and RTT)

BBR models the network instead of reacting to loss:

// BBR conceptually targets bottleneck bandwidth + RTT
const bbrState = {
  bw: 0, // Bottleneck bandwidth
  rtprop: Infinity, // Round trip propagation time
  pacing_rate: bw * 1.25,
  cwnd: 12 * 1500, // In bytes (12 packets)
};

TIME_WAIT and Port Reuse

Connections in TIME_WAIT linger for 2×MSL (60 seconds on Linux by default). High-traffic servers can run out of source ports:

# Enable TIME_WAIT reuse (reuse, don't recycle)
echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse

# Reduce MSL to speed up cleanup (not RFC-compliant)
echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

# Increase ephemeral port range
echo "32768 61000" > /proc/sys/net/ipv4/ip_local_port_range

Keepalive Tuning

Keepalives detect dead connections but waste bandwidth:

# Application-agnostic keepalive (system-wide)
net.ipv4.tcp_keepalive_time = 7200      # Default: 7200s (2 hours!)
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9

# Better: Application-level with TCP keepalive options
socket.keepAlive(true, 30, 10);  // enable, idle 30s, interval 10s

Quick Recap Checklist

Before diving into implementation, ensure you understand:

• TCP header fields and their purposes (sequence, ACK, window, flags)
• Window scaling for high-BDP networks (2^14 multiplier)
• TCP options: MSS, SACK, timestamps, window scaling
• NAT types and traversal techniques (STUN/TURN/ICE)
• QUIC connection migration for mobile networks
• BBR vs cubic congestion control characteristics
• TIME_WAIT recycling and port exhaustion prevention
• Keepalive tuning for production systems

---

Trade-off Analysis

Factor	TCP	UDP	QUIC	Recommendation
Reliability	Ordered delivery	None	Ordered delivery	Ordered delivery required
Overhead	20+ bytes	8 bytes	20+ bytes	Low overhead preferred
Speed	Moderate	Fastest	Faster than TCP	Latency-sensitive
Congestion Control	Built-in	None	Built-in	Streaming/bulk transfer
Connection	Stateful (3-way)	Stateless	Stateless (0-RTT)	Connectionless preferred
TLS	Separate handshake	N/A	Built-in	Security + speed
NAT Traversal	Full cone issues	STUN/TURN	Better via ID	P2P/multiplayer

When to Use Each Protocol

• TCP: Web browsing, email, file transfers, any bulk/reliable data
• UDP: VoIP, video streaming, gaming, DNS, IoT
• QUIC/HTTP/3: Modern web apps prioritising low latency

---

Production Failure Scenarios

Failure	Impact	Mitigation
TCP connection timeout	Requests hang; poor user experience	Set appropriate connection timeouts; implement retry logic
UDP packet loss	Missing data; application-level failures	Implement application-level acknowledgment and retransmission
Port exhaustion	Cannot establish new connections; service unavailable	Monitor connection counts; implement connection pooling; increase port range
SYN flood attack	Server overwhelmed with half-open connections	Use SYN cookies; implement rate limiting; use DDoS protection
NAT timeout	Long-lived connections break; clients appear disconnected	Send keepalive packets; use connection-oriented protocols when possible
MTU mismatch	Packets dropped; connectivity issues	Use Path MTU Discovery; set conservative MTU values
TCP congestion collapse	Network throughput drops dramatically	Use proper congestion control algorithms; implement traffic shaping
UDP amplification attack	Your servers used to attack others	Validate source addresses; restrict UDP responses; implement rate limiting

---

Observability Checklist

A transport-level issue is often the first thing that gets blamed when an application slows down, but it is also one of the last things teams have real visibility into. Application metrics show request duration and error rates. Network metrics show link saturation and packet loss. The actual TCP retransmissions, RST counts, and socket buffer pressure sit at a layer that most monitoring setups skim over, even though they often explain the gap between “the app is slow” and “the network is fine.”

What makes transport observability unique is that the signals are per-connection and per-flow, not per-host. A server can have healthy CPU and memory and still be bleeding throughput because of a single misbehaving peer, a stretched RTT to a far region, or a NAT entry expiring under a long-lived connection. Per-host counters hide these. Per-connection counters show them, but at the cost of cardinality that does not scale to a million open sockets. The trick is knowing which aggregate metrics to keep (retransmit rate, RST rate, TIME_WAIT count) and which to drill into only when you have a specific connection in hand.

The checklist below groups the signals worth collecting into metrics, logs, and alerts. Use the metrics for trend analysis and capacity planning, the logs to root-cause individual incidents, and the alerts to catch regressions before users do. A framing note: connection-level counters in Linux live under /proc/net/netstat and /proc/net/snmp, while per-socket state sits in /proc/net/tcp and /proc/net/udp. Most modern exporters (node_exporter, telegraf, the OpenTelemetry host metrics receiver) already surface the headline numbers. The hard part is usually thresholds, not collection. Defaults are tuned for a LAN, so production thresholds need to account for your real RTT distribution and traffic mix.

Metrics

• TCP connection rate (new connections per second)
• Active TCP connections (concurrent connections)
• TCP connection failures (connection refused, timeout)
• Segment retransmission rate
• TCP buffer utilization (bytes in send/receive buffers)
• UDP packet rate (packets sent/received per second)
• UDP error rate (checksum failures, buffer overflows)
• Round-trip time (RTT) for TCP connections
• Throughput (bytes sent/received per second)

Logs

• Connection failures with source IP and port
• TCP reset packets (RST) received
• UDP packet checksum failures
• Connection timeouts
• Port exhaustion warnings
• Network interface errors

Alerts

• Connection failure rate exceeds 5%
• Retransmission rate exceeds 10%
• Active connections approach limits
• UDP error rate increases
• TCP RST rate spikes (potential attack)
• Network latency anomalies

---

Security Checklist

• Use TLS over TCP when encryption is needed (not raw TCP)
• Implement connection timeouts to prevent resource exhaustion
• Monitor for SYN flood attacks; enable SYN cookies
• Use firewall rules to restrict exposed ports
• Implement rate limiting on TCP/UDP services
• Validate UDP source addresses to prevent spoofing
• Use IPsec for network-level encryption when needed
• Monitor for unusual traffic patterns indicating attacks
• Implement connection tracking for stateful firewall rules
• Restrict broadcast and multicast traffic where not needed

---

Common Pitfalls / Anti-Patterns

Assuming UDP is Always Faster

UDP avoids TCP overhead but does not guarantee delivery or ordering.

// Problem: UDP with no reliability
const socket = dgram.createSocket("udp4");
socket.send(data, port, host, (err) => {
  // No confirmation data arrived - you simply do not know
});

// Better: Implement acknowledgment
socket.send(data, port, host);
socket.on("message", (msg) => {
  if (msg.toString() === "ACK") {
    // Confirmed delivery
  }
});

Ignoring TCP Connection Limits

Each TCP connection consumes file descriptors and memory.

# Check current connection limits
cat /proc/sys/net/core/somaxconn        # Max pending connections
cat /proc/sys/fs/file-max               # System-wide file descriptors

# Monitor active connections
ss -s

Not Handling Connection Termination Properly

Abruptly closing connections can cause data loss.

// Graceful close - ensure data is sent
socket.end(); // Send FIN after remaining data is sent
socket.on("close", () => {
  // Connection fully closed
});

Building Custom Reliability on UDP When TCP Would Work

If you need reliable, ordered delivery, just use TCP.

// Problem: Building reliability on UDP
socket.on("message", (data) => {
  // Must implement sequence numbers, acknowledgments, retransmission
  // This is essentially reimplementing TCP
});

// Better: Just use TCP
const server = net.createServer((socket) => {
  // Reliability built in
});

---

Interview Questions

---

Further Reading

TCP and UDP serve different needs. TCP provides reliability, ordering, and flow control at the cost of latency. UDP provides speed and simplicity at the cost of reliability guarantees.

For application-layer protocols, see the HTTP/HTTPS post. For DNS specifically, the DNS & Domain Management post covers name resolution in detail.

---

Conclusion

Key Bullets

• TCP provides reliable, ordered, connection-oriented delivery with flow and congestion control
• UDP provides fast, unreliable, connectionless delivery without overhead
• TCP three-way handshake establishes connections; UDP sends immediately
• TCP uses acknowledgments, sequence numbers, and retransmission for reliability
• UDP header is 8 bytes; TCP header is 20+ bytes minimum
• Choose TCP when correctness matters; choose UDP when speed and low latency matter more
• Common TCP ports: 80 (HTTP), 443 (HTTPS), 22 (SSH), 25 (SMTP), 53 (DNS)
• DNS uses UDP port 53 for queries, TCP for zone transfers and large responses

Copy/Paste Checklist

# Check TCP connection states
ss -tunapl | grep -E "(State|Recv-Q|Send-Q)"

# Monitor TCP metrics
cat /proc/net/tcp
cat /proc/net/tcp6

# Check UDP statistics
cat /proc/net/udp
cat /proc/net/udp6

# Test TCP connection with netcat
nc -zv host.example.com 443

# Test UDP connectivity (limited)
nc -zvu host.example.com 53

# Check for open ports
ss -tunapl | grep LISTEN

# View TCP window sizes
cat /proc/sys/net/ipv4/tcp_rmem
cat /proc/sys/net/ipv4/tcp_wmem

# Test MTU
ping -M do -s 1472 example.com

---