Virtual Memory

Virtual memory is the engineering feat that makes a computer with 8 GB of RAM run applications that collectively demand 64 GB without crashing. It is the OS’s most visible memory management technique — a layer of indirection that allows the logical address space to exceed the physical address space, using disk as a backing store for the pages that do not fit in RAM. The elegance is that application code does not know and does not care whether its pages are resident in physical memory or have been evicted to swap.

The magic happens through demand paging: pages are loaded into physical memory only when a process accesses them. A program that calls malloc(1 GB) pays no cost at allocation time — the OS creates page table entries for the range but marks them as not present. Physical frames are allocated only when a page fault triggers. This lazy allocation extends to disk-backed pages, heap, and stack alike.

Introduction

When to Use / When Not to Use

Virtual memory is always active — you cannot disable it on any modern general-purpose OS. However, you make implicit choices about how aggressively your workload uses it.

When understanding virtual memory matters:

• Debugging memory-related crashes (segmentation faults, OOM kills, thrashing)
• Optimizing applications where working set > physical memory (large databases, ML training)
• Configuring swap space appropriately (or deciding to use no swap at all)
• Understanding why certain programs behave differently under memory pressure

When virtual memory is a liability:

• Write-heavy workloads on spinning disks (swap thrashing destroys I/O)
• Latency-sensitive real-time applications (page faults introduce unbounded latency)
• Embedded systems with limited storage (swap consumes precious flash endurance)

On latency-sensitive systems (trading platforms, real-time control systems), administrators often disable swap entirely (swapoff -a) to eliminate the possibility of a page fault causing a millisecond-scale pause that violates latency SLAs.

Architecture or Flow Diagram

Virtual memory combines demand paging with page replacement to seamlessly spill pages to disk and retrieve them on demand. The flow below shows the decision tree the OS follows when a page fault occurs.

flowchart TD
    PF["Page Fault<br/>Triggered by CPU"]
    HANDLER["Page Fault<br/>Handler"]
    VALID["Is the address<br/>valid and permitted?"]
    PRESENT["Is the page in<br/>physical memory?"]
    SWAPPED["Is the page<br/>in swap?"]
    LOAD["Load page from<br/>swap device to frame"]
    ZERO["Allocate zero-filled<br/>frame from free pool"]
    UPDATE["Update page table entry<br/>Mark page as present"]
    SEGV["Send SIGSEGV<br/>Terminate process"]
    RESUME["Resume interrupted<br/>instruction"]

    PF --> HANDLER
    HANDLER --> VALID
    VALID -->|"No"| SEGV
    VALID -->|"Yes"| PRESENT
    PRESENT -->|"No, in swap"| LOAD
    PRESENT -->|"No, never allocated"| ZERO
    PRESENT -->|"Yes, already in RAM"| RESUME
    LOAD --> UPDATE
    ZERO --> UPDATE
    UPDATE --> RESUME

    style SEGV stroke:#ff6b6b
    style LOAD stroke:#ff9f43
    style ZERO stroke:#00fff9

The OS maintains a page replacement algorithm to choose which physical frame to evict when a newly faulted page needs a physical frame and none are free. This is where LRU, Clock, and Working Set algorithms diverge.

Core Concepts

Demand Paging

Demand paging is the mechanism by which pages are loaded into physical memory only upon first access. The page table entry for a not-yet-loaded page has the present (P) bit cleared. When a process accesses that page:

1. CPU triggers page fault
2. OS determines the fault address (CR2 register on x86)
3. OS allocates a physical frame
4. OS reads the page from its backing store (executable, page cache, or swap)
5. OS updates the page table entry with the frame number and sets P=1
6. CPU retries the instruction

The first access to a large mmap region is dramatically slower than subsequent access because of this fault storm. Tools like mincore (on Linux) can reveal which pages of a mapped region are resident.

Swap Space

Swap space is a disk partition or file that backs pages that have been evicted from physical memory. When the OS needs a physical frame and the free frame list is empty, it runs the page replacement algorithm to select a victim page — one that has not been recently used — writes it to swap if dirty, and reclaims the frame.

Linux swap space is managed in 4 KB blocks called swap slots. Each slot can hold one evicted page. The swapon command activates a swap partition; swapoff deactivates it. The kernel maintains a swap map — an array tracking reference counts for each swap slot.

Modern kernels support swap files (not just partitions), which can be resized dynamically. The performance of swap files on SSDs is generally acceptable; on HDDs it is poor due to random seek requirements.

Page Replacement Algorithms

When physical memory fills up, the OS must evict some pages to make room for newly needed ones. The choice of which page to evict is the domain of page replacement algorithms.

LRU (Least Recently Used): LRU evicts the page whose last access is furthest in the past. The theoretical ideal for minimizing cache misses, but LRU requires tracking every page access — an expensive hardware or software overhead. Software LRU requires updating a timestamp or queue on every memory access, interfering with normal program execution. Hardware LRU uses a special CPU counter but still requires TLB and cache coherence overhead.

Clock (Second Chance): Clock (also called the clock algorithm or approximated LRU) is a practical LRU approximation. Pages are arranged in a circular list. A pointer (the “clock hand”) scans pages; if a page’s accessed bit is set, it is cleared and the hand moves on. If not set, the page is evicted. This avoids per-access updates — only the accessed bit is checked on eviction, not updated on every access. The “second chance” name comes from the fact that pages with the accessed bit set get one additional time around the clock before eviction.

Linux uses a variant called Clock-Pro (or LRU of the CLOCK-PRO family), which distinguishes between hot and cold pages based on their accessed bit patterns.

Working Set Model: The working set of a process is the set of pages it actively uses over a time window (e.g., the last 10 seconds). Pages outside the working set can be evicted without significantly impacting the process’s hit rate. The working set model aims to keep the working set resident, evicting pages outside it first. The concept was formalized by Peter Denning in the 1960s and is still referenced in modern memory management literature.

A variant called WSClock (Working Set Clock) combines the clock algorithm with working set timestamps, providing both efficiency and accuracy.

Thrashing

Thrashing is the pathological state where the system spends more time swapping pages in and out than executing useful work. The classic scenario: total working sets of all runnable processes exceed physical memory. Every process needs pages, the OS evicts other processes’ pages to make room, those processes fault back in, and the cycle repeats at disk I/O speeds rather than CPU speeds.

The cure is either reducing the number of competing processes, adding physical RAM, or tuning memory limits (cgroups, container memory caps). The vm.swappiness sysctl influences the kernel’s tendency to swap out process pages versus reclaiming page cache.

Production Failure Scenarios

Thrashing Under Memory Pressure

Failure: A node running multiple Java microservices with large heaps experiences constant page faults. The GC in each JVM runs frequently, but heap pages are swapped out between GC cycles. GC pause times spike from 200 ms to 20 seconds. Response times balloon.

Mitigation: Set explicit memory limits for each container (docker run --memory=512m). Reduce vm.swappiness to 10-20 on hosts running latency-sensitive services. Consider disabling swap on database hosts (swapoff) to force the OOM killer to make hard decisions rather than slowly swapping. Move large heaps to hosts with ample headroom. Use memory requests/limits in Kubernetes to prevent any single pod from consuming the node’s memory.

Swap Storm from fork() After exec()

Failure: A web server forks a child process for each request, and the child immediately exec()s a new program. After fork(), the parent and child share all pages (copy-on-write). The child then calls exec(), which replaces the entire address space with the new program — discarding all COW pages. If many requests arrive simultaneously, the OS allocates and discards huge numbers of pages, generating massive swap activity even though the total memory footprint is small.

Mitigation: Use preFork or thread-pool models (Apache MPM worker, Nginx worker processes) instead of fork-per-request. Use vfork() on Linux (which shares the parent’s address space without COW overhead until exec()). Use container memory limits to isolate services and reduce cross-service swap pressure.

Transparent Huge Page Defragmentation Pauses

Failure: Linux’s transparent huge page (THP) feature defragmentates memory in the background to coalesce 4 KB pages into 2 MB huge pages. This defragmentation scan can cause latency spikes of several milliseconds in latency-sensitive workloads.

Mitigation: Disable THP for latency-sensitive applications: echo never > /sys/kernel/mm/transparent_hugepage/enabled. Use explicit huge page allocation (mmap(MAP_HUGETLB)) for services that need it (PostgreSQL, JVM). On kernel 5.17+, use madvise(MADV_COLLAPSE) to request huge page backing for specific regions.

Trade-off Table

Algorithm	Implementation Cost	Eviction Accuracy	Hardware Support	Linux Implementation
Optimal (OPT)	Impossible (requires future knowledge)	Perfect (0 extra evictions)	None	Not used
LRU (true)	Very high (per-access timestamp update)	High	Specialized hardware	Not used (too expensive)
Clock (Second Chance)	Low (circular buffer, accessed bit check)	Moderate	Accessed bit (R-bit)	Used as fallback
Clock-Pro (Linux)	Low-moderate (multiple hands, working set tracking)	Good	Accessed bit	Default since 2.6
Working Set / WSClock	Moderate (per-page timestamps)	Very good	Timestamps	Historical
Random	Zero	Poor (uninformed)	None	Used when others fail

Implementation Snippets

Simulating LRU Page Replacement (Python)

#!/usr/bin/env python3
"""LRU Page Replacement Simulator.

Given a sequence of page accesses and a number of physical frames,
simulate the LRU algorithm and count page faults.

LRU: On each access, track when each page was last used.
When a replacement is needed, evict the page with the oldest
(last-most-recently-used) timestamp.
"""

from collections import OrderedDict

def lru_page_faults(access_sequence: list[int], num_frames: int) -> int:
    """Return the number of page faults using LRU replacement."""
    # Use an OrderedDict: order = recency of access (tail = most recent)
    resident_pages = OrderedDict()  # page -> last_access_order
    faults = 0
    access_counter = 0

    for page in access_sequence:
        access_counter += 1

        if page in resident_pages:
            # Page hit: move to end (most recently used)
            resident_pages.move_to_end(page)
        else:
            faults += 1
            if len(resident_pages) >= num_frames:
                # Evict LRU page (first item in OrderedDict)
                resident_pages.popitem(last=False)
            # Load new page and mark as most recently used
            resident_pages[page] = access_counter

    return faults

def clock_page_faults(access_sequence: list[int], num_frames: int) -> int:
    """Approximated LRU using the Clock (Second Chance) algorithm."""
    # resident_pages[i] = (page_number, referenced_bit)
    resident_pages = []  # List of (page, r_bit)
    faults = 0
    hand = 0  # Clock hand position

    for page in access_sequence:
        # Check if page is already resident
        found = False
        for i, (p, r) in enumerate(resident_pages):
            if p == page:
                resident_pages[i] = (p, 1)  # Set referenced bit
                found = True
                break

        if found:
            continue

        # Page fault — need to load page
        faults += 1

        # Find a frame to use
        while True:
            if len(resident_pages) < num_frames:
                resident_pages.append((page, 1))
                break

            # Clock algorithm: scan for victim with r=0
            _, r = resident_pages[hand]
            if r == 0:
                # Evict this page
                resident_pages[hand] = (page, 1)
                hand = (hand + 1) % num_frames
                break
            else:
                # Second chance: clear bit, move on
                resident_pages[hand] = (resident_pages[hand][0], 0)
                hand = (hand + 1) % num_frames

    return faults

# Example from textbook: access sequence 7, 0, 1, 2, 0, 3, 0, 4, 1, 2, 3, 4, 7, 0, 1
accesses = [7, 0, 1, 2, 0, 3, 0, 4, 1, 2, 3, 4, 7, 0, 1]

print("Page access sequence:", accesses)
print("\nLRU algorithm:")
for frames in range(1, 7):
    faults = lru_page_faults(accesses, frames)
    print(f"  {frames} frame(s): {faults} faults")

print("\nClock algorithm:")
for frames in range(1, 7):
    faults = clock_page_faults(accesses, frames)
    print(f"  {frames} frame(s): {faults} faults")

Monitoring Virtual Memory (bash)

#!/bin/bash
# Comprehensive virtual memory monitoring

echo "=== VM Statistics ==="
vmstat 1 5

echo ""
echo "=== Swap Usage ==="
swapon --show
echo ""
free -h

echo ""
echo "=== Top processes by major page faults ==="
ps -eo pid,comm,majflt,minflt,rss,vsz --sort=-majflt | head -10

echo ""
echo "=== Current swappiness ==="
cat /proc/sys/vm/swappiness

echo ""
echo "=== Recent OOM Kills ==="
dmesg | grep -i "oom\|out of memory" | tail -5

Observability Checklist

• Overall swap activity: vmstat 1 — si (swap in) and so (swap out) columns in KB/s
• Major page fault rate per process: ps -eo pid,comm,majflt --sort=-majflt | head
• Minor page fault rate: ps -eo pid,comm,minflt — high minflt = copy-on-write activity
• Page cache pressure: cat /proc/sys/vm/vfs_cache_pressure (default 100; higher = reclaim more page cache vs process pages)
• Swappiness: cat /proc/sys/vm/swappiness (0-100; higher = more willing to swap process pages)
• Available swap: swapon --show and free -h
• Per-process swap usage: cat /proc/PID/status | grep -i swap
• OOM killer events: dmesg | grep "Out of memory" | tail or journalctl -k | grep oom
• Transparent huge page status: cat /sys/kernel/mm/transparent_hugepage/enabled
• Active vs inactive memory: cat /proc/meminfo | grep -E "Active:|Inactive:" — inactive memory can be reclaimed without causing faults

Common Pitfalls / Anti-Patterns

Swapping sensitive data to disk: When pages containing sensitive data (cryptographic keys, passwords, private keys) are evicted to swap, they reside in plaintext on the swap device. If an attacker gains physical access or reads the swap device after the system crashes, they can recover this data. Mitigations include:

• Encrypted swap: LUKS-encrypted swap partitions, or cryptswap with a key stored only in RAM
• Memory locking (mlock/mlockall): Prevent specific pages from being swapped by calling mlock() on the containing region
• Key management: Hardware security modules (HSMs) and Intel SGX enclaves keep keys in memory that cannot be swapped or inspected by the OS

Cold boot attacks targeting swap: Similar to the memory forensics scenario, cold boot attacks can image the swap partition and recover sensitive data. Encrypted swap renders this ineffective.

Memory disclosure via /proc filesystem: The /proc/PID/maps, /proc/PID/smaps, and /proc/PID/pagemap interfaces expose memory layout information that can aid exploits. Most production environments restrict access to /proc/PID/ to the owner or disable it via hidepid=2 mount option on /proc.

Common Pitfalls / Anti-patterns

Pitfall: Configuring too much swap on systems with SSDs. If you set swap equal to RAM on an SSD-backed system, you may never hit OOM — the system will happily swap instead of invoking the OOM killer. This creates a death by a thousand cuts: applications slow down due to page fault latency, but do not crash. The system appears responsive but degrades gradually. Better practice: set vm.swappiness=1 (or 0 on kernels 3.5+) for swap-averse workloads, and use memory limits to enforce hard caps.

Pitfall: Disabling swap entirely on systems with overcommitted memory. Disabling swap (swapoff -a) works well when you have more than enough physical RAM for all workloads. On a system with memory overcommit (common in containerized environments), disabling swap means the first process to exhaust memory triggers the OOM killer immediately, with no graceful degradation. You want some swap as a pressure valve.

Pitfall: Confusing virtual memory size with physical memory usage. pmap -X <pid> shows the virtual address space size (VSZ) and resident set size (RSS). The gap between VSZ and RSS can be enormous — virtual memory includes mapped but never-touched pages, memory-mapped files that were faulted in then swapped, and stack guard pages. A process with VSZ=100 GB and RSS=200 MB is using 200 MB of physical memory, not 100 GB.

Anti-pattern: Large allocations without considering swap. On 32-bit systems (or 32-bit processes on 64-bit kernels), a single large malloc can exhaust the virtual address space. mmap of a large region that is never accessed reserves virtual address space but no physical memory — but malloc reserves virtual address space for the entire heap and commits pages on first write. The combination of malloc + heavy swap usage can cause the OOM killer to terminate processes unexpectedly.

Quick Recap Checklist

• Virtual memory allows logical address spaces larger than physical memory by using disk as backing store
• Demand paging loads pages into RAM only on first access (page fault), not at allocation time
• Page replacement algorithms (LRU, Clock, Working Set) decide which physical pages to evict when RAM is full
• Thrashing occurs when the OS spends more time swapping than executing — total working sets exceed physical RAM
• The OOM killer invoked when physical memory + swap is exhausted, not when virtual address space is exhausted
• Linux uses Clock-Pro as its default page replacement algorithm (approximates LRU efficiently)
• vm.swappiness controls kernel preference: higher = more willing to swap process pages, lower = prefer page cache reclaim
• Huge pages (2 MB, 1 GB) reduce TLB pressure for large working sets — important for databases and JVMs
• Encrypted swap prevents cold-boot and physical-access attacks from reading sensitive data from swap space
• Tools: vmstat 1 (swap activity), ps -o majflt,minflt (fault rates), swapon --show (swap size), /proc/meminfo (memory state)

Interview Questions

Further Reading

Swap Space Configuration and Performance

Swap size guidelines:

• Traditional rule: RAM × 1.5 to 2.0
• For swap-averse workloads (databases, in-memory caches): 0 or minimal
• For memory overcommit scenarios: at least RAM × 0.5 as pressure valve
• For hibernation: at least RAM (Linux hibernate saves memory to swap)

Swap performance on different storage:

Storage	Swap Latency	Sequential Throughput	Random IOPS	Recommendation
NVMe SSD	~100 μs	3-7 GB/s	100k-1M	Good for swap
SATA SSD	~100 μs	0.5-1 GB/s	10k-100k	Acceptable
HDD	~10 ms	100-200 MB/s	100-500	Avoid for active swap

Swap area priorities (Linux mkswap and swapon): Multiple swap files/partitions can have priorities assigned. Higher priority swap is used first. Use pri= option in /etc/fstab or swapon --priority.

ZSwap: Compressed Swap Cache

ZSwap (Linux 3.11+) is a lightweight compressed cache for pages that would be swapped out. Instead of writing to disk, ZSwap compresses pages and stores them in a memory pool. If the compressed pool fills up, the least-recently-used pages are written to disk.

Benefits:

• Reduces disk I/O for workloads with good compression ratios
• Improves performance for intermittent swap usage
• Especially effective for workloads where swapped pages are soon accessed again

Configuration: echo 1 > /sys/module/zswap/parameters/enabled (varies by distribution)

Page Replacement Algorithm Variants

Linux Clock-Pro: Tracks the referenced bit pattern across multiple clock hands to distinguish between hot and cold pages. Pages that are frequently referenced maintain a pattern of accessed bits; pages that were accessed once and not again get evicted first.

Two-List Strategy: Linux maintains active and inactive page lists. Pages are initially added to the active list; if not referenced again, they move to the inactive list and become candidates for eviction. This simple LRU approximation prevents a single reference from keeping a page permanently resident.

Key Takeaways

• Virtual memory extends physical RAM by using disk as a backing store for evicted pages
• Demand paging loads pages only on first access — virtual allocation costs nothing until used
• Page replacement algorithms (Clock-Pro in Linux) approximate LRU without per-access overhead
• Thrashing occurs when total working sets exceed physical memory — the system spends time swapping instead of computing
• OOM killer is invoked when physical memory plus swap is exhausted
• Encrypted swap prevents cold-boot and physical-access attacks from recovering swapped data

Conclusion

Virtual memory extends physical memory by using disk as a backing store for pages not currently in RAM. Demand paging loads pages only when accessed, so allocating 1 GB of virtual memory costs nothing until the pages are actually used. This enables systems to run workloads larger than installed RAM without crashing.

Page replacement algorithms decide which physical pages to evict when RAM fills up. True LRU is too expensive to implement, so Linux uses Clock-Pro — an efficient approximation that tracks hot and cold pages using accessed bits. Thrashing occurs when working sets collectively exceed physical memory, collapsing throughput as the system spends more time swapping than executing.

The OOM killer intervenes when physical memory plus swap is exhausted, choosing a process to terminate. For latency-sensitive workloads, administrators often disable swap entirely to force fast failure rather than slow degradation. Encrypted swap protects sensitive data from cold-boot attacks if the system is powered down while pages reside in swap space.

For your next step, explore paging and page tables to understand the data structures that map virtual pages to physical frames, or process scheduling to see how the OS decides which runnable process gets CPU time when memory is not the bottleneck.