Introduction

Building your own operating system is the ultimate learning exercise in computer science. It forces you to understand what the “magic” of an OS actually does: how does a computer start? How does code run after power-on? How does the CPU switch between tasks? Every abstraction that seems “obvious” in modern operating systems becomes a concrete implementation choice you must make.

This is not an academic exercise—understanding OS fundamentals makes you a better systems programmer, debugger, and architect. Even if you never ship your OS, the mental model you’ll develop applies to kernel development, embedded systems, firmware, and understanding why things break in production.

When to Use / When Not to Use

Building an OS from scratch is appropriate when:

• Deep learning is the goal — You want to understand every layer of the stack
• Embedded constraints require minimal footprint — IoT devices with 64KB RAM
• Security-critical systems — You need a TCB you can fully verify
• Academic or personal projects — Self-directed learning with tangible outcomes

This approach is NOT appropriate when:

• You need production-ready functionality — Use Linux, FreeRTOS, or Zephyr
• Time constraints are tight — OS development takes months for even minimal features
• Your team lacks low-level expertise — Missteps can brick hardware or cause data loss
• Requirements are unclear — Start with requirements before architecture

Architecture or Flow Diagram

flowchart TB
    subgraph "Boot Phase"
        HW[Hardware Reset]
        BIOS[BIOS / UEFI Firmware]
        MBR[MBR Boot Sector<br/>512 bytes @ 0x7C00]
        BOOT[Bootloader Stage 2<br/>GRUB or custom]
    end

    subgraph "Protected Mode"
        PM[Protected Mode Entry<br/>32-bit GDT setup]
        KERNEL[Kernel Loader<br/>Parse multiboot header]
        KERNEL_MAIN[Kernel Main<br/>Initialize subsystems]
    end

    subgraph "Kernel Subsystems"
        GDT[GDT & IDT Setup]
        MEM[Memory Manager<br/>Paging / Physical alloc]
        IRQ[Interrupt Handler<br/>PIC / APIC setup]
        SCHED[Scheduler<br/>Context switching]
        FS[Simple File System<br/>FAT12 / custom]
        DRIVERS[Basic Drivers<br/>VGA, keyboard, ATA]
    end

    HW --> BIOS
    BIOS --> MBR
    MBR --> BOOT
    BOOT --> PM
    PM --> KERNEL
    KERNEL --> KERNEL_MAIN
    KERNEL_MAIN --> GDT
    GDT --> IDT
    IDT --> MEM
    MEM --> IRQ
    IRQ --> SCHED
    SCHED --> FS
    FS --> DRIVERS

    style KERNEL_MAIN stroke:#ff6b6b,stroke-width:3px
    style SCHED stroke:#ffa94d,stroke-width:3px

Core Concepts

Boot Loader (x86 BIOS/MBR)

The first sector (512 bytes) must end with 0xAA55 and fit everything needed to load the second stage:

; boot.asm - Simple 16-bit boot sector
; Assemble: nasm -f bin boot.asm -o boot.bin

BITS 16

ORG 0x7C00        ; BIOS loads us here

start:
    ; Set up segments
    xor ax, ax
    mov ds, ax
    mov es, ax
    mov ss, ax
    mov sp, 0x7C00

    ; Set video mode
    mov ah, 0x00
    mov al, 0x03    ; 80x25 color text
    int 0x10

    ; Print message
    mov si, msg
    call print_string

    ; Load kernel from disk
    mov ah, 0x02    ; BIOS disk read
    mov al, 10      ; Sectors to read
    mov ch, 0       ; Cylinder 0
    mov cl, 2       ; Start from sector 2
    mov dh, 0       ; Head 0
    mov dl, 0x80    ; First hard disk
    mov bx, 0x1000  ; Load to 0x1000:0000
    mov es, bx
    xor bx, bx
    int 0x13
    jc disk_error

    ; Jump to kernel
    jmp 0x0000:0x1000

print_string:
    lodsb
    or al, al
    jz .done
    mov ah, 0x0E
    int 0x10
    jmp print_string
.done:
    ret

disk_error:
    mov si, err_msg
    call print_string
    jmp $

msg db 'Loading OS...', 0
err_msg db 'Disk read error!', 0

; Fill remainder with zeros and boot signature
times 510-($-$$) db 0
dw 0xAA55

Protected Mode Transition

Switching from 16-bit real mode to 32-bit protected mode:

; switch_to_pm.asm - Enter protected mode

BITS 16

switch_to_protected_mode:
    ; Disable interrupts
    cli

    ; Load GDT
    lgdt [gdt_descriptor]

    ; Enable A20 line (access extra memory)
    in al, 0x92
    or al, 2
    out 0x92, al

    ; Set CR0 bit to enter protected mode
    mov eax, cr0
    or eax, 1
    mov cr0, eax

    ; Far jump to flush pipeline and load CS
    jmp 0x08:flush_pm

BITS 32

flush_pm:
    ; Set up data segments
    mov ax, 0x10    ; Data segment selector
    mov ds, ax
    mov ss, ax
    mov es, ax
    mov fs, ax
    mov gs, ax

    ; Set stack
    mov esp, 0x90000

    ; Jump to kernel C code
    extern kernel_main
    call kernel_main
    jmp $

; GDT
gdt_start:
    ; Null descriptor (required)
    dq 0

    ; Code segment descriptor
    dw 0xFFFF       ; Limit
    dw 0x0000       ; Base (low)
    db 0x00         ; Base (middle)
    db 10011010b    ; Access byte
    db 11001111b    ; Flags + Limit (high)
    db 0x00         ; Base (high)

    ; Data segment descriptor
    dw 0xFFFF
    dw 0x0000
    db 0x00
    db 10010010b
    db 11001111b
    db 0x00

gdt_end:

gdt_descriptor:
    dw gdt_end - gdt_start - 1
    dd gdt_start

Simple Kernel in C

A minimal kernel with basic output and memory management:

/* kernel.c - Minimal 32-bit kernel */

#define VIDEO_MEMORY 0xB8000
#define WHITE_ON_BLACK 0x0F

/* VGA text mode functions */
void clear_screen(void)
{
    volatile unsigned short *video = (unsigned short *)VIDEO_MEMORY;
    for (int i = 0; i < 80 * 25; i++) {
        video[i] = (WHITE_ON_BLACK << 8) | ' ';
    }
}

void print_string(const char *str)
{
    volatile unsigned short *video = (unsigned short *)VIDEO_MEMORY;
    int i = 0;
    while (str[i] != '\0') {
        *video++ = (WHITE_ON_BLACK << 8) | str[i++];
    }
}

/* Basic memory management */
#define KERNEL_HEAP_START 0x100000
#define KERNEL_HEAP_SIZE 0x100000

static unsigned int next_free = KERNEL_HEAP_START;

void *kmalloc(size_t size)
{
    void *ptr = (void *)next_free;
    next_free += size;
    if (next_free >= KERNEL_HEAP_START + KERNEL_HEAP_SIZE) {
        return 0;  /* Out of memory */
    }
    return ptr;
}

/* Kernel main entry */
void kernel_main(void)
{
    clear_screen();
    print_string("Welcome to MyOS!\n");

    /* Test memory allocation */
    char *test = kmalloc(100);
    test[0] = 'H';
    test[1] = 'i';
    test[2] = '\0';
    print_string(test);
}

/* Define empty interrupt handlers */
void isr0(void) { while(1); }
void isr1(void) { while(1); }
/* ... more ISRs ... */

Simple Round-Robin Scheduler

/* scheduler.c - Round-robin task scheduler */

#include <stdint.h>

#define MAX_TASKS 64
#define TASK_STACK_SIZE 4096

struct task {
    uint32_t id;
    uint32_t esp;          /* Stack pointer */
    uint32_t eip;          /* Instruction pointer */
    uint32_t priority;
    enum { TASK_RUNNING, TASK_READY, TASK_BLOCKED, TASK_DEAD } state;
    uint8_t stack[TASK_STACK_SIZE];
};

static struct task tasks[MAX_TASKS];
static int num_tasks = 0;
static int current_task = 0;

/* TSS for hardware context switching */
struct tss_entry {
    uint16_t link, :16;
    uint32_t esp0;
    uint16_t ss0, :16;
    uint32_t esp1, ss1, esp2, ss2;
    uint32_t cr3;
    uint32_t eip, eflags, eax, ecx, edx, ebx;
    uint32_t esp, ebp, esi, edi;
    uint16_t es, :16, cs, :16, ss, :16, ds, :16;
    uint16_t fs, :16, gs, :16, ldt, :16;
} __attribute__((packed));

static struct tss_entry tss;

int create_task(void (*entry)(void), uint32_t priority)
{
    if (num_tasks >= MAX_TASKS) return -1;

    struct task *t = &tasks[num_tasks++];
    t->id = num_tasks;
    t->state = TASK_READY;
    t->priority = priority;

    /* Set up stack to look like we were interrupted */
    t->esp = (uint32_t)&t->stack[TASK_STACK_SIZE - 1];

    /* Push fake interrupt frame */
    t->esp -= sizeof(uint32_t);  /* EIP placeholder */
    *(uint32_t *)t->esp = (uint32_t)entry;

    return t->id;
}

void schedule(void)
{
    /* Find next ready task */
    int prev = current_task;
    do {
        current_task = (current_task + 1) % num_tasks;
    } while (tasks[current_task].state != TASK_READY
             && current_task != prev);

    if (current_task != prev) {
        tasks[prev].state = (tasks[prev].state == TASK_RUNNING)
                           ? TASK_READY : tasks[prev].state;
        tasks[current_task].state = TASK_RUNNING;
        switch_task(&tasks[prev].esp, tasks[current_task].esp);
    }
}

/* Called from timer interrupt (IRQ0) */
void timer_interrupt_handler(void)
{
    /* Acknowledge PIC */
    outb(0x20, 0x20);

    schedule();
}

Simple FAT12 File System

/* fs_fat12.c - Minimal FAT12 file system */

#include <stdint.h>

#define SECTOR_SIZE 512
#define FAT_START 1
#define ROOT_START 19
#define DATA_START 33

struct fat12_boot {
    uint8_t jmp[3];
    uint8_t OEM[8];
    uint16_t bytes_per_sector;
    uint8_t sectors_per_cluster;
    uint16_t reserved_sectors;
    uint8_t num_fats;
    uint16_t root_entries;
    uint16_t total_sectors;
    uint8_t media_type;
    uint16_t sectors_per_fat;
    uint16_t sectors_per_track;
    uint16_t num_heads;
    uint32_t hidden_sectors;
    uint32_t large_sector_count;
    uint8_t drive_number;
    uint8_t reserved;
    uint8_t boot_signature;
    uint32_t volume_id;
    uint8_t volume_label[11];
    uint8_t fs_type[8];
} __attribute__((packed));

struct fat12_entry {
    uint8_t filename[8];
    uint8_t ext[3];
    uint8_t attributes;
    uint8_t reserved[10];
    uint16_t time;
    uint16_t date;
    uint16_t start_cluster;
    uint32_t file_size;
} __attribute__((packed));

/* Read file from FAT12 */
int read_file(uint16_t cluster, uint8_t *buffer, int max_size)
{
    int bytes_read = 0;
    uint8_t fat[FAT12_MAX_SIZE];

    /* Read FAT into memory */
    read_sectors(FAT_START, FAT12_SECTORS, fat);

    while (cluster < 0xFF8) {  /* Not end-of-file marker */
        if (bytes_read >= max_size) break;

        /* Read data cluster */
        uint32_t data_sector = DATA_START + (cluster - 2);
        read_sectors(data_sector, 1, buffer + bytes_read);
        bytes_read += SECTOR_SIZE;

        /* Get next cluster from FAT */
        uint16_t fat_index = cluster + (cluster / 2);
        cluster = *(uint16_t *)&fat[fat_index];
        if (cluster & 1) {
            cluster >>= 1;
        } else {
            cluster = cluster >> 1;
        }
    }

    return bytes_read;
}

Production Failure Scenarios

Scenario 1: Stack Overflow Corrupts Memory

Problem: Kernel stack is too small or grows into other regions, causing mysterious crashes.

Mitigation:

• Define proper stack sizes with guard pages
• Use __attribute__((interrupt)) for ISR functions (handles stack frame)
• Check stack pointer against bounds before function calls
• For recursive code, ensure termination conditions are always met

Scenario 2: Triple Fault During Protected Mode Transition

Problem: CPU triple-faults (reset) during mode transition, making debugging difficult.

Mitigation:

• Ensure GDT is correctly aligned and loaded
• Verify CS descriptor has proper access byte (0x9A for code)
• Use a boot ROM or serial debug output before the crash
• Build with debugging symbols and use a hardware debugger (QEMU + GDB)

Scenario 3: Paging Enabled Incorrectly

Problem: Enabling paging with incorrect page tables causes immediate crash.

Mitigation:

• Identity-map the kernel first (virtual == physical) before enabling
• Set CR3 to point to valid page directory
• Enable paging with mov cr0, eax followed immediately by a jmp to flush pipeline
• Use QEMU with -d int to trace interrupt/exceptions during boot

Trade-off Table

Component	Bare Metal	Emulator (QEMU)	Real Hardware
Development Speed	Slowest (flash/reset cycle)	Fast (snapshot/restore)	Slow
Debugging	Limited (serial output)	Full GDB support	Limited
Risk	Can brick device	None	Potential for damage
Realism	True hardware	May differ from real HW	Perfect
CI/CD	Difficult	Easy	Difficult

Implementation Snippet: Linking the Kernel

/* kernel.ld - Linker script for bare-metal kernel */

OUTPUT_FORMAT("elf32-i386")
ENTRY(kernel_main)

SECTIONS
{
    . = 0x100000;  /* Kernel load address */

    .text : {
        *(.text)
        *(.rodata)
    }

    .data : {
        *(.data)
    }

    .bss : {
        *(.bss)
    }

    /DISCARD/ : {
        *(.comment)
        *(.eh_frame)
    }
}

Building and running:

#!/bin/bash
# build.sh - Build and run MyOS

set -e

# Assemble boot sector
nasm -f bin boot.asm -o boot.bin

# Assemble protected mode switch
nasm -f elf32 switch_to_pm.asm -o switch_to_pm.o

# Compile kernel and scheduler
gcc -m32 -ffreestanding -O2 -c kernel.c -o kernel.o
gcc -m32 -ffreestanding -O2 -c scheduler.c -o scheduler.o

# Link everything
ld -T kernel.ld -m elf_i386 boot.bin switch_to_pm.o kernel.o scheduler.o -o kernel.bin

# Create floppy image with FAT12
mkdir -p tmp
dd if=/dev/zero of=myos.img bs=512 count=2880
./mkfat.sh myos.img  # Create FAT12 filesystem
dd if=boot.bin of=myos.img conv=notrunc

# Run in QEMU
qemu-system-i386 -fda myos.img -display curses
# Or with GDB debugging:
# qemu-system-i386 -fda myos.img -s -S &
# gdb -ex 'target remote localhost:1234' -ex 'break kernel_main'

Observability Checklist

For OS development, instrument these early:

• Serial output — The most reliable debug channel before VGA/text is set up
• QEMU logging — -d int for interrupt/debug output, -D log.txt for all
• GDB stub — QEMU’s -gdb tcp::1234 enables full source-level debugging
• QMP (QEMU Machine Protocol) — Programmatic control and inspection
• Ranchu logs — make V=1 to see all command invocations during build

Common Pitfalls / Anti-Patterns

• UEFI secure boot — Custom OS may not boot on new hardware without signing
• DMA attacks — Without IOMMU, any device can access all memory
• No ASLR — Simple kernels typically don’t implement address space layout randomization
• No userspace isolation — Bugs in kernel code affect entire system

Common Pitfalls / Anti-patterns

1. Starting without a bootable environment — Set up QEMU + GDB first, verify it works
2. Skipping protected mode — 16-bit real mode limitations will cripple your kernel
3. Not handling interrupts early — Unhandled interrupts cause immediate triple-fault
4. Assuming hardware behaves correctly — Test on real hardware early; emulators differ
5. Not using a linker script — You’ll spend days debugging random crashes without proper memory layout

Quick Recap Checklist

• Boot process: BIOS/UEFI -> MBR -> Bootloader -> Kernel
• 16-bit real mode transitions to 32-bit protected mode via GDT
• VFS abstraction separates file system implementation from system calls
• A simple scheduler needs: task structures, context switching, timer interrupts
• Use QEMU for development; test on real hardware for validation
• Debug output early and often—serial console is your friend
• Start minimal: get to “Hello World” in protected mode before adding features

---

Real-World Case Study: xv6 Educational OS

MIT’s xv6 is a teaching OS derived from Unix v6, rewritten for modern x86. It demonstrates:

1. Minimal but complete - ~8,000 lines covering process management, scheduler, syscalls, file system
2. Pedagogical clarity - Each component is small enough to understand in isolation
3. Modernized implementation - While inspired by v6, xv6 uses modern structures like spinlocks and a proper file system layer

Studying xv6 before building your own OS provides a reference implementation that shows how components interact without the complexity of production kernels.

Advanced Topic: UEFI Boot Process

Modern systems boot via UEFI (Unified Extensible Firmware Interface) instead of legacy BIOS:

UEFI advantages:

• Support for GPT partitioned disks (no MBR limitations)
• Driver support (can access disks before loading OS)
• Secure Boot (verifies bootloader signatures)
• Network boot via PXE
• Modular shell and applications

UEFI boot sequence:

1. Power-on → UEFI initialization → Load bootloader from ESP (EFI System Partition)
2. Bootloader (GRUB2, systemd-boot) loads kernel with UEFI runtime services
3. Kernel calls ExitBootServices() to release firmware control
4. OS has full control of hardware

Your OS can be built for UEFI by: (1) declaring EFI boot_stub section, (2) linking with Gnu EFI toolchain, (3) placing .efi file in ESP/EFI/BOOT/BOOTX64.EFI

Interview Questions

Further Reading

• osdev.org Wiki - Comprehensive OS development documentation
• Linux from Scratch - Build your own Linux system from source
• Writing a Simple Operating System from Scratch - Step-by-step tutorial
• Intel Software Developer Manual - Official x86 documentation
• Broken Thorn OS Development Series - Detailed OS development tutorial

---

Conclusion

Building your own OS provides the deepest understanding of how computers actually work—from the moment power is applied through boot loader execution, protected mode transition, and into kernel initialization. The boot process (BIOS/UEFI -> MBR -> bootloader -> kernel) requires understanding real-mode addressing, GDT setup, and protected mode entry. A minimal kernel needs memory management (physical allocation or paging), interrupt handling (IDT setup, PIC/APIC), and a scheduler with context switching capability.

QEMU with GDB debugging transforms OS development from blind guessing into systematic investigation. Serial console output provides reliable debugging before VGA/text initialization. Start minimal: get to “Hello World” in protected mode before adding features, and debug output early and often.

For continued learning, explore advanced boot concepts (UEFI, multiboot2), virtual memory with page tables and TLB management, user space process creation with fork/exec, and system call interface design. Consider implementing a simple shell, filesystem (FAT12 or a custom minimal FS), and basic device drivers as next steps beyond the minimal kernel.

---