Threads & Lightweight Processes

Picture a factory floor. Each process is like a separate building with its own entrance, utilities, and security clearance. Inside each building, workers (threads) share the same electricity, water, and access cards — they can communicate instantly without going outside.

In computing terms: threads are the unit of execution within a process. They share the process’s address space, open files, and global variables, but each thread has its own stack, program counter, and registers. This shared everything makes threads lightweight compared to processes, and it’s why modern servers can handle millions of concurrent connections with a handful of processes.

Introduction

When you run a web server, each incoming connection doesn’t need a separate process. A single process can handle thousands of connections using threads — each thread processes a different client, but all threads share the same memory and file descriptors. This dramatically reduces overhead compared to forking a new process per connection.

Threads enable concurrency (one CPU doing multiple things by interleaving) and parallelism (multiple CPUs doing multiple things simultaneously). Understanding when to use each — and which threading model to use — is crucial for building scalable systems.

User Threads vs Kernel Threads

The fundamental distinction in threading is where threads are managed:

Aspect	User Threads	Kernel Threads
Managed by	User-level library (pthread, greenlet)	Operating system kernel
Scheduling	User-space library decides	Kernel scheduler decides
Context switch	Fast (no kernel involvement)	Slower (kernel mode required)
Blocking I/O	Can block entire process	Can block single thread
Multi-core	Cannot run in parallel	Can run in parallel
Example	GNU Pth, Green threads	POSIX pthread, Windows threads

User Threads

User threads are managed entirely in user space without kernel involvement. The thread library (e.g., GNU Pth, goroutine runtime) handles thread creation, scheduling, and synchronization. The kernel sees only the single process — it knows nothing about the user-level threads within it.

Advantages:

• Fast creation and context switching (no system calls)
• Custom scheduling policies
• Portable across operating systems

Disadvantages:

• If any user thread blocks on I/O, the entire process blocks (kernel sees only one runnable task)
• Cannot take advantage of multi-core parallelism
• If the user thread library has bugs, the whole process crashes

// User-level threading example (GNU Pth)
#include <pth.h>

void *worker(void *arg) {
    pth_sleep(1);  // Cooperative yield to other user threads
    return NULL;
}

int main() {
    pth_init();
    pth_t t1 = pth_spawn(PTH_ATTR_DEFAULT, worker, NULL);
    pth_join(t1, NULL);
    pth_exit(0);
}

Kernel Threads

Kernel threads are managed by the OS scheduler — each thread appears as a schedulable entity. The kernel handles context switching between threads, and threads from the same process can run on different CPU cores.

Advantages:

• True parallelism on multi-core systems
• Individual threads can block on I/O without affecting others
• Kernel handles all scheduling, no custom library needed

Disadvantages:

• Slower context switches (requires kernel mode transition)
• Higher memory overhead (kernel data structures per thread)
• More expensive creation

// Kernel threading example (POSIX pthread)
#include <pthread.h>

void *worker(void *arg) {
    sleep(1);  // Kernel preempts, other threads run
    return NULL;
}

int main() {
    pthread_t t1, t2;
    pthread_create(&t1, NULL, worker, NULL);
    pthread_create(&t2, NULL, worker, NULL);
    pthread_join(t1, NULL);
    pthread_join(t2, NULL);
    return 0;
}

When to Use

• I/O-bound concurrent workloads: Web servers, API handlers, database connection pools
• CPU-bound parallel computation: Scientific computing, image processing, ML inference
• Real-time systems: Hard deadlines require kernel-level thread control
• High-throughput network services: Millions of connections need efficient threading

When Not to Use

• Simple scripts: Using threads adds complexity without benefit for sequential tasks
• CPU-bound single-task: A single-threaded process often outperforms threaded code for compute-heavy single tasks due to avoiding synchronization overhead
• Debugging complexity: Threading bugs (deadlocks, race conditions) are notoriously hard to reproduce — only use when necessary

---

Thread Models

Modern systems implement threading in several models, each with tradeoffs:

graph BT
    subgraph "Threading Models"
        M1[1:1 Model<br/>One kernel thread<br/>per user thread]
        M2[N:1 Model<br/>Many user threads<br/>to one kernel thread]
        M3[N:M Model<br/>Many user threads<br/>to many kernel threads]
    end

    K[Kernel Space]
    U[User Space]

    U --> M1
    U --> M2
    U --> M3
    M1 --> K
    M3 --> K

    style M1 stroke:#1a3a1a
    style M2 stroke:#3a1a1a
    style M3 stroke:#1a1a3a

1:1 Model (One-to-One)

Each user-level thread maps to one kernel thread.

• Used by: Linux (NPTL), Windows (Win32 Threads), Solaris
• Pros: True parallelism, individual thread blocking works
• Cons: Higher overhead (kernel thread creation), limited scalability (thread count limited)

N:1 Model (Many-to-One)

Many user threads map to a single kernel thread.

• Used by: GNU Pth, early green thread implementations
• Pros: Fast, scalable to millions of fibers
• Cons: Cannot use multi-core, one blocked user thread blocks all

N:M Model (Many-to-Many)

Many user threads are distributed across a pool of kernel threads.

• Used by: Solaris (LWPs), some Go runtimes (goroutines on GOMAXPROCS > 1)
• Pros: Best of both worlds — parallelism without unlimited kernel threads
• Cons: Complex implementation, requires coordination between user and kernel scheduling

---

Architecture Diagram

graph TB
    subgraph Process_Address_Space
        subgraph Code_Region[Code (.text)]
            C1[Function A]
            C2[Function B]
        end

        subgraph Shared_Data[Shared Data]
            G1[Global Variables]
            G2[Heap]
        end

        subgraph Thread_Stacks[Per-Thread Stacks]
            S1[Thread 1 Stack<br/>SP, BP, Return Addr]
            S2[Thread 2 Stack<br/>SP, BP, Return Addr]
            S3[Thread 3 Stack<br/>SP, BP, Return Addr]
        end

        subgraph Per_Thread_Registers[Per-Thread Registers]
            R1[Thread 1: PC=0x401234]
            R2[Thread 2: PC=0x402345]
            R3[Thread 3: PC=0x403456]
        end
    end

    S1 --> G1
    S2 --> G1
    S3 --> G1

    S1 --> C1
    S2 --> C2
    S3 --> C1

    style Shared_Data stroke:#1a3a1a
    style Thread_Stacks stroke:#3a1a1a
    style Per_Thread_Registers stroke:#1a1a3a

Each thread has its own stack and registers, but all threads share the code and global data regions. This shared access is what makes threads efficient for communication but also what creates synchronization challenges.

---

Core Concepts

Thread-Specific Data

Each thread needs its own stack for local variables, but often needs “global” data that’s actually per-thread (thread-local storage):

// Thread-local storage example
#include <pthread.h>

__thread int my_thread_id = 0;  // GCC extension

void *worker(void *arg) {
    my_thread_id = *(int *)arg;
    printf("Thread %d sees my_thread_id = %d\n", pthread_self(), my_thread_id);
    return NULL;
}

Thread Local Storage (TLS)

TLS provides thread-safe access to global-like data without explicit parameter passing. The OS allocates a separate slot for each thread’s copy:

# Python threading with local context
import threading

local_data = threading.local()

def worker(n):
    local_data.value = n
    import time
    time.sleep(0.1)
    print(f"Thread {n} sees value = {local_data.value}")

threads = [threading.Thread(target=worker, args=(i,)) for i in range(5)]
for t in threads: t.start()
for t in threads: t.join()

Thread States

Threads have states similar to processes:

• New: Thread being initialized
• Ready: Thread waiting to run
• Running: Thread executing on a CPU
• Blocked: Thread waiting for I/O or synchronization
• Terminated: Thread finished, waiting to be joined

Thread Cancellation

Threads can be cancelled, but careful handling is required:

#include <pthread.h>

void *worker(void *arg) {
    while (1) {
        // Check for cancellation point
        pthread_testcancel();

        // Do work
        printf("Working...\n");
        sleep(1);
    }
    return NULL;
}

int main() {
    pthread_t t;
    pthread_create(&t, NULL, worker, NULL);

    sleep(5);
    pthread_cancel(t);  // Async cancellation
    pthread_join(t, NULL);
    return 0;
}

Thread Pools

Creating a new thread per task is expensive. Thread pools solve this by pre-creating threads and reusing them:

#!/usr/bin/env python3
"""Thread pool implementation in Python."""

from queue import Queue, Empty
from threading import Thread, Condition
import time

class ThreadPool:
    def __init__(self, num_threads):
        self.tasks = Queue()
        self.workers = []
        self.shutdown = False
        self.lock = Condition()

        for _ in range(num_threads):
            t = Thread(target=self._worker_loop)
            t.daemon = True
            t.start()
            self.workers.append(t)

    def _worker_loop(self):
        while True:
            try:
                task = self.tasks.get(timeout=0.1)
                if task is None:  # Poison pill
                    break
                func, args, kwargs = task
                try:
                    func(*args, **kwargs)
                except Exception as e:
                    print(f"Task failed: {e}")
                finally:
                    self.tasks.task_done()
            except Empty:
                if self.shutdown:
                    break

    def submit(self, func, *args, **kwargs):
        if self.shutdown:
            raise RuntimeError("Pool is shutting down")
        self.tasks.put((func, args, kwargs))

    def wait_completion(self):
        self.tasks.join()

    def shutdown_pool(self):
        self.shutdown = True
        for _ in self.workers:
            self.tasks.put(None)  # Poison pills
        for t in self.workers:
            t.join()


# Usage
def cpu_task(n):
    result = sum(i*i for i in range(1000000))
    print(f"Task {n} completed, result starts with {result}")

pool = ThreadPool(4)
for i in range(8):
    pool.submit(cpu_task, i)
pool.wait_completion()
pool.shutdown_pool()

---

Production Failure Scenarios

Deadlocks

Problem: Two threads each hold a lock the other needs, and neither will release until it acquires the other.

Symptoms: Application hangs with 100% CPU utilization (spinning on locks) or no CPU utilization (deadlocked with sleeping threads).

Mitigation:

• Always acquire locks in a consistent global order
• Use lock timeout with pthread_mutex_timedlock()
• Use lock-free data structures when possible
• Detect potential deadlocks with thread sanitizers

// Lock ordering to prevent deadlock
pthread_mutex_lock(&lock_a);
pthread_mutex_lock(&lock_b);
// Do work
pthread_mutex_unlock(&lock_b);
pthread_mutex_unlock(&lock_a);

// NEVER do this in another thread:
// pthread_mutex_lock(&lock_b);
// pthread_mutex_lock(&lock_a);

Race Conditions

Problem: Multiple threads access shared data without synchronization, leading to unpredictable results.

Symptoms: Intermittent crashes, data corruption, bizarre behavior only in production.

Mitigation:

• Use proper synchronization (mutexes, atomic operations)
• Prefer lock-free data structures
• Use thread-safe primitives from day one
• Run with TSAN_OPTIONS=halt_on_error=1 (ThreadSanitizer)

# Compile with ThreadSanitizer
gcc -fsanitize=thread -g program.c -o program

# Run to detect race conditions
./program

Thread Starvation

Problem: Some threads never get CPU time because other threads monopolize resources.

Symptoms: Certain requests timeout while others complete normally.

Mitigation:

• Set thread priorities appropriately
• Use fair scheduling policies (SCHED_FIFO, SCHED_OTHER)
• Implement work-stealing queues

Premature Thread Termination

Problem: Process exits before spawned threads complete.

Mitigation: Always call pthread_join() or use pthread_detach().

---

Trade-off Table

Thread Model	Parallelism	Scalability	Complexity
1:1 (Kernel)	Full	Limited by thread count	Simple
N:1 (User)	None	High	Simple
N:M (Hybrid)	Full	High	Complex

Thread Creation	Time (µs)	Memory
pthread_create	~10-50	~8MB stack
goroutine	~0.2	~2KB stack
fiber (fthread)	~0.1	~4KB stack

Use Case	Best Model	Reason
Web server (blocking I/O)	1:1	True parallelism needed
Event-driven server	N:1 (green threads)	Millions of connections
CPU-bound parallel	N:M	Balance parallelism and overhead

---

Implementation Snippets

Basic Pthread Creation and Join

#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>

#define NUM_THREADS 4

void *compute(void *arg) {
    int n = *(int *)arg;
    long result = 0;
    for (int i = 0; i < 1000000; i++) {
        result += i * i;
    }
    printf("Thread %d computed %ld\n", n, result);
    return NULL;  // Return value passed to pthread_join()
}

int main() {
    pthread_t threads[NUM_THREADS];
    int args[NUM_THREADS];

    // Create threads
    for (int i = 0; i < NUM_THREADS; i++) {
        args[i] = i;
        if (pthread_create(&threads[i], NULL, compute, &args[i]) != 0) {
            perror("pthread_create failed");
            return 1;
        }
    }

    // Wait for all threads
    for (int i = 0; i < NUM_THREADS; i++) {
        pthread_join(threads[i], NULL);  // NULL = don't capture return
    }

    printf("All threads completed\n");
    return 0;
}

Mutex for Shared Data

#include <pthread.h>
#include <stdio.h>

typedef struct {
    pthread_mutex_t mutex;
    int counter;
} shared_data_t;

void *increment(void *arg) {
    shared_data_t *data = (shared_data_t *)arg;

    for (int i = 0; i < 1000; i++) {
        pthread_mutex_lock(&data->mutex);
        data->counter++;
        pthread_mutex_unlock(&data->mutex);
    }
    return NULL;
}

int main() {
    shared_data_t data = {.counter = 0};
    pthread_mutex_init(&data.mutex, NULL);

    pthread_t t1, t2;
    pthread_create(&t1, NULL, increment, &data);
    pthread_create(&t2, NULL, increment, &data);

    pthread_join(t1, NULL);
    pthread_join(t2, NULL);

    printf("Final counter: %d (expected: 2000)\n", data.counter);
    pthread_mutex_destroy(&data.mutex);
    return 0;
}

Condition Variables for Thread Coordination

#include <pthread.h>
#include <stdio.h>

typedef struct {
    pthread_mutex_t mutex;
    pthread_cond_t cond;
    int ready;
} sync_data_t;

void *producer(void *arg) {
    sync_data_t *data = (sync_data_t *)arg;
    printf("Producer: Doing work...\n");
    sleep(1);

    pthread_mutex_lock(&data->mutex);
    data->ready = 1;
    pthread_cond_signal(&data->cond);  // Wake up consumer
    pthread_mutex_unlock(&data->mutex);
    return NULL;
}

void *consumer(void *arg) {
    sync_data_t *data = (sync_data_t *)arg;

    pthread_mutex_lock(&data->mutex);
    while (!data->ready) {
        pthread_cond_wait(&data->cond, &data->mutex);  // Atomically unlock and wait
    }
    pthread_mutex_unlock(&data->mutex);

    printf("Consumer: Data is ready!\n");
    return NULL;
}

int main() {
    sync_data_t data = {.ready = 0};
    pthread_mutex_init(&data.mutex, NULL);
    pthread_cond_init(&data.cond, NULL);

    pthread_t prod, cons;
    pthread_create(&cons, NULL, consumer, &data);
    pthread_create(&prod, NULL, producer, &data);

    pthread_join(prod, NULL);
    pthread_join(cons, NULL);

    pthread_mutex_destroy(&data.mutex);
    pthread_cond_destroy(&data->cond);
    return 0;
}

---

Observability Checklist

Key Metrics

# Show all threads (LWPs) for a process
ps -eLf -p <pid>

# Show thread-specific CPU usage
top -H -p <pid>

# Count threads
cat /proc/<pid>/status | grep Threads

# Check thread IDs
cat /proc/<pid>/task/*/tid

Trace Commands

# Trace thread creation
sudo perf trace -e clone

# Trace mutex operations
perf record -e 'mutex:*' -a -- sleep 10
perf report

# Show lock contention
perf sched record -- sleep 5
perf sched latency

Debugging Commands

# Attach to process and view threads
gdb -p <pid>
(gdb) info threads

# View thread stack traces
pstack <pid>

# Show where threads are blocked
cat /proc/<pid>/stack

---

Common Pitfalls / Anti-Patterns

Thread Safety in Security-Critical Code

Security-sensitive data (keys, tokens, credentials) must be protected with proper synchronization. Use thread-local storage for sensitive per-thread data to avoid accidentally leaking between threads.

Resource Limits

Thread counts are limited by system resources:

# Check thread limits
cat /proc/sys/kernel/threads-max
cat /proc/sys/vm/max_map_count

# Set soft limit
ulimit -u

Race Conditions as Security Vulnerabilities

Race conditions in privilege operations can lead to TOCTOU (Time-of-check to time-of-use) attacks. Use atomic operations for security-critical code paths.

---

Common Pitfalls / Anti-patterns

Creating too many threads

Each thread has a stack (default 8MB on Linux). Creating 1000 threads consumes 8GB just for stacks. Use thread pools instead.

Not handling thread cancellation safely

If you call pthread_cancel(), ensure cancellation points exist. Async cancellation can leave data structures corrupted.

Ignoring false sharing

Threads modifying data in the same cache line (even with different variables) cause cache line invalidation storms. Pad data structures to cache line boundaries (64 bytes).

// False sharing example
struct {
    pthread_mutex_t mutex;
    long counter;  // These two might share a cache line
} data[4];  // Each thread modifies data[i].counter

// Better: pad to prevent cache line sharing
struct {
    char pad[64];  // Cache line padding
    long counter;
};

Mixing locked and lock-free access

Don’t use mutexes for some accesses to shared data and atomics for others — this creates undefined behavior.

---

Quick Recap Checklist

• Threads share process address space but have own stack and registers
• User threads: fast but can’t use multi-core; kernel threads: slower but truly parallel
• 1:1 model (Linux, Windows) is most common; N:M (Solaris, some Go) balances tradeoffs
• Thread pools avoid creation overhead and limit thread count
• Deadlocks occur when locks are acquired in inconsistent orders
• Race conditions cause intermittent failures; use ThreadSanitizer to detect
• Each thread needs ~8MB stack on Linux; too many threads = memory exhaustion
• Always join or detach threads to prevent resource leaks
• Use thread-local storage for per-thread globals without explicit parameter passing

---

Interview Questions

Further Reading

• Process Concept — Understand the PCB, process states, and fork-exec pattern
• Process Scheduling — How schedulers allocate CPU time to processes
• Process Scheduling Algorithms — Deep dive into scheduling algorithm implementations

---

Conclusion

Threads are the unit of execution within a process — they share the process’s address space, open files, and global variables while each maintaining its own stack, program counter, and registers. This shared-everything model makes threads lightweight compared to processes and enables efficient communication between concurrent tasks.

User threads (managed in library space) are fast but cannot achieve true multi-core parallelism because the kernel sees only one schedulable entity. Kernel threads are managed by the OS scheduler and can run on different CPU cores simultaneously. The 1:1 model (used by Linux and Windows) is the most common implementation — each user thread maps to one kernel thread. The N:M model (used by some Go runtimes) multiplexes many user threads over a pool of kernel threads for best-of-both-worlds performance.

Thread pools avoid the overhead of creating and destroying threads for short-lived tasks, while bounding memory usage by limiting the number of concurrent threads. Synchronization primitives (mutexes, condition variables, semaphores) protect shared data, but incorrect usage leads to deadlocks, race conditions, and performance pathologies like false sharing.

For your next step, explore process scheduling algorithms to understand how the OS decides which thread runs on each CPU core, or synchronization primitives to master the tools that make multi-threaded programming safe and correct.