🔥 FlamesLearn

Introduction to Operating Systems

Why This Matters

Every program you've ever run — from a browser to a database to a game — relies on the operating system to do the heavy lifting. The OS manages the CPU, memory, and disk so that your code doesn't have to. Understanding how it works gives you a mental model for why programs behave the way they do, and it's the foundation for everything in systems software: performance tuning, security, embedded development, and kernel engineering.

What Is an Operating System?

An operating system (OS) is system software that manages computer hardware and provides a platform for running application programs. Familiar examples: Linux, Windows, macOS, FreeBSD.

Two useful mental models:

Metaphor Meaning
OS ≈ SDK for hardware Just as an SDK hides API details, the OS hides hardware details
OS = kernel + utilities The kernel is the core; utilities like sh, ls, cat sit on top

The kernel is the privileged component that actually talks to hardware. Everything else — your editor, web browser, build tools — lives in user space and asks the kernel for services.

What Does an OS Do?

The OS serves four fundamental roles:

  1. Abstraction — exposes clean interfaces (files, processes, sockets) instead of raw hardware registers and interrupts.
  2. Multiplexing — time-shares the CPU and divides memory among multiple running programs.
  3. Isolation — prevents a buggy or malicious program from corrupting another program's memory or crashing the system.
  4. Sharing — lets programs communicate safely (shared files, pipes, network sockets).

Core Services

The OS kernel provides abstractions that applications rely on:

System Calls: The Interface

Applications talk to the kernel through system calls — a controlled jump from user space into kernel code. Examples:

fd = open("out", 1);      // open a file, get a file descriptor
write(fd, "hello\n", 6);  // write bytes through the descriptor
pid = fork();             // create a child process

System calls are the only sanctioned entry points into the kernel. This boundary is what makes isolation possible.

User Space vs. Kernel Space

A CPU runs in one of two privilege modes at any moment:

Mode Also called What it can do
User space ring 3 (x86) Execute normal instructions; cannot touch hardware directly
Kernel space ring 0 (x86) Full hardware access; runs privileged instructions

The hardware enforces this boundary. If a user-space program tries to execute a privileged instruction (e.g., reprogram the interrupt controller), the CPU raises a fault and the kernel handles it — typically by killing the offending process.

The only legitimate way to cross from user space into kernel space is via a system call (or an interrupt/exception).

Kernel Design: Monolithic vs. Micro-Kernel

Once you decide what the kernel is, the next question is how to structure it. Two major schools of thought:

Monolithic Kernel

All OS services (file system, scheduler, device drivers, networking, virtual memory) run inside the kernel at ring 0, sharing the same address space.

Advantages:

Disadvantages:

Linux is the canonical monolithic kernel.

Micro-Kernel

The kernel is stripped to the minimum: IPC, basic virtual memory, and thread scheduling. Everything else — file servers, device drivers, network stacks — runs as ordinary user processes.

Advantages:

Disadvantages:

SeL4 and QNX are micro-kernel examples. Mach (the base of macOS/iOS) started as a micro-kernel but evolved into a hybrid.

The Tanenbaum–Torvalds Debate

In 1992, professor Andrew Tanenbaum argued that monolithic kernels (like early Linux) were architecturally obsolete and that micro-kernels were the future. Linus Torvalds pushed back, arguing that micro-kernels had unacceptable performance overhead for real-world use.

The practical outcome: almost all widely deployed kernels today are hybrids. Linux is mostly monolithic but runs certain subsystems (like the X Window System) in user space. macOS uses a Mach-derived hybrid. Windows has both kernel and user-mode components.

The xv6 Teaching Kernel

This course uses xv6, an open-source teaching OS from MIT, written in C and targeting x86-64. It is small enough to read completely (≈10 000 lines) but implements real OS concepts: processes, virtual memory, a file system, device drivers, and system calls.

You will:

Setting Up the Environment

Linux

sudo apt update -y
sudo apt install -y build-essential git qemu-system-x86

macOS

brew install git qemu x86_64-elf-gcc make

Windows

Install WSL2 (or VirtualBox + Ubuntu 24.04), then follow the Linux steps.

Get and Run xv6

git clone https://github.com/sysec-uic/xv6-public.git
cd xv6-public
make qemu-nox
# To quit: Ctrl-A, then X

Study the Makefile to understand build targets like fs.img, xv6.img, and UPROGS.

Key Takeaways

Practice

  1. Which of the following best describes the primary role of an operating system?
  2. On an x86 processor, which protection ring does the OS kernel run in?
  3. What is the correct way for a user-space application to request a service from the OS kernel?
  4. A monolithic kernel differs from a micro-kernel primarily in that a monolithic kernel:
  5. Which of the following is a key disadvantage of the micro-kernel design compared to a monolithic kernel?
  6. Name the four fundamental roles an operating system plays for application programs and give a one-sentence description of each.
  7. Explain the Tanenbaum–Torvalds debate: what position did each side take, and what is the practical outcome visible in modern OS kernels?
  8. Consider this code snippet: c fd = open("out", 1); write(fd, "hello\n", 6); pid = fork(); What abstraction does fd represent, and which component of the OS provides it?
  9. You are writing a device driver for a new storage controller. Would you more likely place it inside a monolithic kernel or implement it as a user-space server in a micro-kernel design? Describe one concrete benefit and one concrete drawback of each placement.
  10. What is xv6, and why is it used in this course instead of studying the Linux kernel directly?