Memory Management & Swap
Swap is not extra RAM. It is the last line before the OOM killer.
Overview
Swap activity in production is a warning sign, not a feature. Understanding the full memory management stack — from page cache to OOM killer — is what separates reactive firefighting from proactive capacity management. Answer the five questions below at the diagnostic depth a senior sysadmin would use during a live incident.
Read before you answer
Linux memory management is one of the most misunderstood areas of systems administration, and misunderstanding it is expensive. The kernel manages physical RAM through a virtual memory system: every process gets a virtual address space, and the kernel maps pages of that address space to physical RAM frames on demand. When physical RAM is exhausted, the kernel has two options: reclaim memory by evicting clean page cache entries (data that can be re-read from disk) and writing dirty pages (modified data) to swap, or trigger the OOM (Out of Memory) killer to terminate one or more processes to free memory. The OOM killer chooses its victim using an `oom_score` — a kernel-calculated value based on the process's memory usage and other heuristics. The process with the highest score is killed. This is not always the right process, and OOM kills in production are almost always a symptom of insufficient memory planning rather than a normal operational event.
The `vm.swappiness` kernel parameter (range 0–100, default typically 60) controls how aggressively the kernel prefers swapping anonymous memory (process heap and stack) versus reclaiming page cache. A value of 60 means the kernel will begin considering swap relatively early; a value of 10 means it will strongly prefer to keep process memory in RAM and reclaim page cache instead. For database servers where the page cache is cold and process memory is hot, a low swappiness (10) is often appropriate. For general-purpose servers with mixed workloads, the default may be acceptable. Setting swappiness to 0 in recent kernels does not disable swap entirely — it means the kernel will only use swap when memory is critically exhausted, not to avoid OOM. Disabling swap entirely (by removing the swap partition) removes the safety net: a single memory spike can cause OOM kills that a small swap partition would have absorbed.
Identifying memory problems requires distinguishing between the legitimate use of memory for page cache (which appears as "used" in naive readings of `free` but is immediately available to processes) and actual memory pressure. The output of `free -h` shows `buff/cache` separately from process memory; the `available` column is the correct measure of how much memory is available for new processes without swapping. Tools for deeper analysis include: `vmstat 1` for real-time memory and swap I/O rates; `smem` for per-process proportional set size (PSS) analysis; `/proc/<pid>/smaps` for detailed process memory maps; and `sar -r` for historical memory usage trends. A memory leak presents as steady growth in a process's RSS over time with no corresponding increase in workload — diagnosing it requires tracking per-process memory over time, not just looking at current totals.