Signals

Every process in a Unix-like system lives under the constant awareness of signals — asynchronous notifications from the kernel and other processes that something important has happened. When you press Ctrl+C in a terminal, a signal is sent. When a process crashes and the kernel detects an illegal memory access, a signal is delivered. When a child process exits, its parent receives a signal. Signals are the oldest and most fundamental form of inter-process communication, predating pipes, message queues, and sockets.

Despite their simplicity, signals are often misunderstood and mishandled. The async nature of signals makes them notoriously tricky to reason about — the signal can arrive at any point in your program’s execution, interrupting system calls, modifying global state, and causing subtle bugs if not handled carefully. Getting signal handling right is a hallmark of systems programming competence.

Introduction

A signal is a notification sent to a process (or thread) to indicate that an event has occurred. Signals are asynchronous — they can arrive at any time, interrupting the normal flow of execution. When a signal arrives, the process either:

1. Runs a signal handler — a function you define to handle the specific signal
2. Takes the default action — which varies by signal (terminate, ignore, stop, dump core)
3. Ignores it — if you explicitly request to ignore the signal

There are over 30 standard signals defined in POSIX, each with a name (like SIGTERM, SIGINT, SIGSEGV) and a numeric value. Signals are delivered for various reasons:

• User-initiated: Ctrl+C sends SIGINT, Ctrl+Z sends SIGTSTP
• Process communication: One process sending a signal to another via kill()
• Kernel-generated: Hardware exception (SIGSEGV, SIGFPE), timer (SIGALRM), child exit (SIGCHLD)
• Resource limits: SIGXCPU (CPU time exceeded), SIGXFSZ (file size exceeded)

Signals are delivered to a specific thread. If the process has multiple threads, the signal is delivered to one arbitrarily selected thread that is not blocking the signal.

When to Use / When Not to Use

Use signals when:

• You need to send a simple notification to a process (terminate, pause, resume)
• You need to handle asynchronous events like timer expirations or child process exits
• You need to implement a clean shutdown mechanism (catch SIGTERM and cleanup gracefully)
• You are building a daemon that responds to external control commands
• You need to interrupt a blocking system call

Do not use signals when:

• You need to transfer data (signals carry no payload — use pipes or sockets)
• You need synchronous, ordered communication (use message queues or pipes)
• You need many different message types (use sockets with custom protocols)
• You are building complex IPC that requires confirmation/reply patterns

Architecture or Flow Diagram

sequenceDiagram
    participant P1 as Process A (Sender)
    participant Kernel
    participant P2 as Process B (Receiver)

    Note over P1: kill(PID_B, SIGTERM)
    P1->>Kernel: raise(SIGTERM, target=PID_B)
    Kernel->>Kernel: Validate target process exists<br/>Check sender permissions

    alt Process B has handler
        Kernel->>P2: Deliver signal (may interrupt syscall)
        P2->>P2: Run signal handler sigaction(SIGTERM)
        P2-->>P2: Resume execution after handler
    else Process B ignores
        Kernel->>P2: Signal delivered but ignored
        Note over P2: No action taken
    else Default action
        Kernel->>P2: Terminate process
        Note over P2: Process exits, status = 143 (SIGTERM)
    end

Core Concepts

Signal Dispositions and sigaction

Every signal has a “disposition” — what happens when the signal is delivered. The default disposition varies by signal. You can change the disposition using signal() (simple, legacy) or sigaction() (more control, portable):

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>

void handle_sigterm(int sig) {
    // Signal-safe operations only here
    // No printf, malloc, or non-async-signal-safe functions!
    const char msg[] = "Received SIGTERM, shutting down...\n";
    write(STDOUT_FILENO, msg, sizeof(msg) - 1);
    // Perform cleanup...
    _exit(0);
}

int main() {
    struct sigaction sa;
    sa.sa_handler = handle_sigterm;  // Handler function
    sigemptyset(&sa.sa_mask);        // No signals blocked during handler
    sa.sa_flags = 0;                  // No special flags
    // SA_RESTART: restart syscalls interrupted by this signal
    // SA_NODEFER: don't block this signal during its own handler

    if (sigaction(SIGTERM, &sa, NULL) == -1) {
        perror("sigaction");
        exit(1);
    }

    // Now SIGTERM will call handle_sigterm instead of terminating
    while (1) {
        pause();  // Wait for signals
    }
}

Key sigaction fields:

• sa_handler: Function pointer (SIG_DFL for default, SIG_IGN to ignore)
• sa_mask: Set of signals to block during this handler’s execution
• sa_flags: Modifiers like SA_RESTART (restart syscalls), SA_SIGINFO (use sa_sigaction with extra info)

Signal Masks

Each thread (and by extension, process) has a signal mask — a set of signals that are currently blocked. Blocked signals are not delivered, but they may be pending. You can manipulate the signal mask with pthread_sigmask() (for threads) or sigprocmask() (for single-threaded processes):

#include <signal.h>

void block_sigchld() {
    sigset_t mask;
    sigemptyset(&mask);
    sigaddset(&mask, SIGCHLD);

    sigset_t oldmask;
    pthread_sigmask(SIG_BLOCK, &mask, &oldmask);
    // In this block, SIGCHLD won't interrupt us

    // ... do work ...

    // Restore previous mask
    pthread_sigmask(SIG_SETMASK, &oldmask, NULL);
}

Sending Signals

You send signals with the kill() system call:

#include <signal.h>
#include <unistd.h>

pid_t target_pid = 1234;

// Send SIGTERM to a process
if (kill(target_pid, SIGTERM) == -1) {
    perror("kill");
}

// Send SIGUSR1 with no default action (user-defined)
kill(target_pid, SIGUSR1);

The raise() function sends a signal to the current process:

raise(SIGALRM);  // Equivalent to kill(getpid(), SIGALRM)

Signal Safety and Async Signal Safety

Not all functions can be safely called from a signal handler. A function is async-signal-safe if it can be safely called from a signal handler (it does not use non-reentrant locks, does not allocate memory, etc.). Functions like write(), _exit(), sync(), wait() are safe. Functions like printf(), malloc(), free(), pthread_mutex_lock() are NOT safe.

The list of async-signal-safe functions is given in man 7 signal-safety. When writing signal handlers, keep them minimal — set a flag, write to a pipe, call _exit(). Do complex cleanup in the main program after checking the flag.

Waiting for Signals: pause(), sigsuspend(), sigwaitinfo()

// Simple pause (wait for any signal)
pause();

// Block in sigsuspend atomically replacing signal mask
sigset_t mask;
sigemptyset(&mask);
sigaddset(&mask, SIGINT);
sigprocmask(SIG_BLOCK, &mask, NULL);  // Block SIGINT
// Critical section - SIGINT blocked
sigprocmask(SIG_SETMASK, &mask, NULL);  // Unblock, delivering SIGINT

// Synchronous wait for specific signals (real-time signals)
sigset_t waitmask;
sigemptyset(&waitmask);
sigaddset(&waitmask, SIGRTMIN);
sigwaitinfo(-1, &waitmask, NULL);  // Wait for SIGRTMIN

Production Failure Scenarios

Interrupted System Calls (EINTR)

When a signal handler runs during a blocking system call (like read(), write(), connect(), sleep()), the kernel may interrupt the call and return -1 with errno = EINTR. If you do not handle this, your program may interpret the interrupted call as an error.

Mitigation: Always check for EINTR and retry interrupted calls. Use SA_RESTART flag with sigaction() to automatically restart some syscalls. Review all blocking calls for EINTR handling:

ssize_t n;
while ((n = read(fd, buf, sizeof(buf))) == -1 && errno == EINTR) {
    // Retry
}
if (n == -1) {
    perror("read");
}

Signal Handler Race Conditions

If your signal handler modifies a global variable that is also modified by the main program without synchronization, you get a race condition. For example, a handler setting a shutdown_requested flag while the main loop checks it:

volatile sig_atomic_t shutdown_requested = 0;  // Volatile needed!

void handler(int sig) {
    shutdown_requested = 1;
}

int main() {
    // ...
    while (!shutdown_requested) {
        // Do work - but this is still racy on some architectures!
    }
}

Mitigation: Use volatile sig_atomic_t for simple flags (atomic enough for single-flag access). For more complex synchronization, use a self-pipe — the signal handler writes to a pipe, and the main loop monitors the pipe with select()/epoll().

Missing SIGCHLD Handling — Zombie Processes

If a child process exits and the parent does not wait() for it, the child becomes a zombie. If the parent never calls wait() (or uses waitpid() with WNOHANG), zombie processes accumulate. In modern init systems (systemd), this is less of a problem, but in long-running daemons it is a real issue.

Mitigation: Call signal(SIGCHLD, SIG_IGN) to ignore SIGCHLD and let the system automatically reap children (on Linux). Or use sigaction() with SA_NOCLDWAIT flag. Or implement a SIGCHLD handler that calls waitpid() in a loop with WNOHANG.

SIGPIPE Misconfiguration

When writing to a pipe or socket whose reading end has closed, the writer receives SIGPIPE. The default action terminates the process. If you have not handled SIGPIPE, any write() to a broken pipe terminates your program.

Mitigation: Always set signal(SIGPIPE, SIG_IGN) early in program initialization, or use MSG_NOSIGNAL flag on send():

signal(SIGPIPE, SIG_IGN);

Signal Masking Deadlocks

If a signal is blocked and arrives while your program is in a critical section (holding a lock), and that signal’s handler tries to acquire the same lock, you deadlock. This is especially tricky with recursive mutexes.

Mitigation: Keep signal handlers very simple and lock-free. Design your locking to ensure that signal handlers never need locks that the interrupted code might hold.

Trade-off Table

Implementation Snippet(s)

C: Clean Shutdown with SIGTERM

#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdbool.h>

volatile sig_atomic_t g_shutdown = false;

void sigterm_handler(int sig) {
    g_shutdown = true;
}

int main() {
    struct sigaction sa;
    sa.sa_handler = sigterm_handler;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = SA_RESTART;  // Restart interrupted syscalls

    sigaction(SIGTERM, &sa, NULL);
    sigaction(SIGINT, &sa, NULL);

    printf("Service running (PID %d). Send SIGTERM to stop.\n", getpid());

    while (!g_shutdown) {
        // Do work — simulate with sleep
        sleep(1);
    }

    printf("Shutting down gracefully...\n");
    // Cleanup: close files, release resources, flush buffers
    printf("Done.\n");
    return 0;
}

Python: Signal Handling

import signal
import sys

def sigterm_handler(signum, frame):
    print("Received SIGTERM, shutting down gracefully...")
    # Note: in Python, most functions are safe to call from signal handlers
    # because the GIL provides some protection
    sys.exit(0)

def sigint_handler(signum, frame):
    print("Received Ctrl+C, shutting down...")
    sys.exit(0)

signal.signal(signal.SIGTERM, sigterm_handler)
signal.signal(signal.SIGINT, sigint_handler)

print(f"PID: {sys.executable}")
while True:
    signal.pause()

Bash: Signal Trapping in Scripts

#!/bin/bash
# Trap signals for graceful shutdown
cleanup() {
    echo "Received signal, cleaning up..."
    rm -f /tmp/myapp.lock
    exit 0
}

trap cleanup SIGTERM SIGINT SIGQUIT

# Or ignore signals during critical section
trap '' SIGINT
echo "In critical section - Ctrl+C ignored"
sleep 5
trap SIGINT  # Restore default handling

# Background job with signal control
sleep 100 &
sleep_pid=$!
# Wait for job or signal
wait $sleep_pid 2>/dev/null || echo "Interrupted"

Observability Checklist

• Pending signals: Check pending signals for a process with cat /proc/<pid>/status | grep -i SigPnd
• Blocked signals: Check blocked signals with cat /proc/<pid>/status | grep -i SigBlk
• Signal handler registration: Use strace -e trace=signal to see signal-related system calls
• Process state on signal: ps aux shows processes in interruptible sleep (state S) waiting for signals
• Core dumps: Check ulimit -c and /proc/sys/kernel/core_pattern to ensure core dumps are captured for crashes
• Signal delivery timing: Use perf sched or ftrace to measure signal delivery latency
• SIGCHLD zombies: ps aux | grep -i zombie to find zombie processes indicating improper wait handling

Common Pitfalls / Anti-Patterns

Signal permissions: Sending a signal requires appropriate permissions — either the sender must have the same effective user ID as the receiver (kill(pid, sig) succeeds), or the sender must have CAP_KILL capability (root). This prevents arbitrary processes from sending signals to each other.

Signal DoS: A process could flood another process with signals (e.g., repeatedly sending SIGURG which has no default action but still triggers handler execution), consuming CPU time in the handler. Use rate limiting if you are implementing signal-sending functionality.

Signal-based attacks: Some privilege escalation exploits use signals to manipulate process state. Ensure that signal handlers do not perform unsafe operations and that security-sensitive state is not accessible via signal handler interfaces.

Audit: Signal delivery may not generate standard audit events. For compliance requirements requiring complete process behavior tracking, implement application-level logging around signal handling.

Common Pitfalls / Anti-patterns

1. Calling non-async-signal-safe functions in handlers — printf(), malloc(), free(), pthread_mutex_lock() are all unsafe in signal handlers. Keep handlers minimal: set a flag, write to a pipe, call _exit().

2. Not handling EINTR on blocking calls — always check for and retry interrupted system calls, or use SA_RESTART to let the kernel handle it.

3. Using non-volatile global flags — the compiler may optimize away reads of a global variable that the signal handler sets, causing infinite loops. Use volatile sig_atomic_t.

4. Ignoring SIGCHLD — not handling child exits causes zombie processes that accumulate and consume process table entries.

5. Not setting SA_RESTART on time-consuming operations — if a signal arrives during a long read() from a slow device, and you did not set SA_RESTART, the read() returns -1 with EINTR. Handle this or use select()/poll().

6. Signal handler modifying errno — signal handlers clobber errno. Save and restore errno in handlers if needed.

7. Mixing signal() and sigaction() — the behavior of signal() varies across platforms (some restart syscalls, some do not). Always use sigaction() for portability.

8. Sending SIGKILL to processes that need cleanup — SIGKILL cannot be caught, blocked, or ignored. It terminates the process immediately without running cleanup handlers. Use SIGTERM for graceful shutdown.

Quick Recap Checklist

• Signals are async notifications delivered to processes; they can interrupt any execution point
• Use sigaction() for portable, full-featured signal handling (preferred over signal())
• Signal handlers must only call async-signal-safe functions — keep handlers minimal
• volatile sig_atomic_t for simple flags; use self-pipe pattern for complex synchronization
• SIGPIPE defaults to terminating the process — set SIG_IGN or use MSG_NOSIGNAL
• SA_RESTART flag automatically restarts certain interrupted system calls
• SIGCHLD should be handled or ignored to prevent zombie processes
• kill(pid, sig) sends a signal to a specific process; raise(sig) sends to current process
• Real-time signals (SIGRTMIN to SIGRTMAX) support queuing and carry extra payload
• Always handle EINTR on blocking calls or use SA_RESTART

Interview Questions

struct sigaction sa;
sa.sa_handler = handle_sigchld;
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART | SA_NOCLDWAIT;
sigaction(SIGCHLD, &sa, NULL);

int sigpipefd[2];
pipe(sigpipefd);
signal(SIGTERM, handler);  // handler does write(sigpipefd[1], "!", 1)
while (epoll_wait(efd, …) > 0) {
for each event:
if (event.fd == sigpipefd[0]) {
read(sigpipefd[0], &byte, 1);
// Do cleanup now - we are in safe context
}
}

Further Reading

• signal(7) — Linux man page — Overview of signals on Linux
• sigaction(2) — Linux man page — Signal action API reference
• signal-safety(7) — Linux man page — Async-signal-safe functions list
• kill(2) — Linux man page — Sending signals between processes

Conclusion

Signals are the oldest form of process communication in Unix systems — async notifications that can interrupt execution at any point. While the concept is simple, signal handling exposes the tension between the kernel’s event delivery model and safe user-space execution. The rules around async-signal-safety, EINTR handling, and signal mask management exist because signals cross protection boundaries in ways that normal function calls do not.

As systems evolved, signals gave way to more sophisticated event delivery mechanisms: event loops with select()/poll()/epoll(), kernel event queues in modern kernels, and asynchronous I/O APIs. Yet signals persist because they remain the only kernel-to-process notification mechanism for certain events (child exit, resource limits, crashes) that have no alternative.

For continued learning, explore how eventfd() provides a signal-like notification mechanism compatible with epoll(), and study how modern Linux uses real-time signals (SIGRTMIN to SIGRTMAX) with siginfo delivery for richer signal payloads beyond simple notifications.