When physical RAM and swap space are completely exhausted, the Linux kernel triggers its Out-Of-Memory (OOM) Killer routine to terminate offending processes and prevent kernel panic. But how does the kernel decide which process to kill first, and how can developers manually trigger the OOM killer for testing?

In this article, we demonstrate how to invoke the OOM Killer on demand using the Magic SysRq trigger (/proc/sysrq-trigger), inspect raw oom_score badness metrics under /proc/[pid]/, and adjust process termination priority using oom_score_adj.

Quick Command Sequence

Use these commands to enable SysRq, trigger the OOM Killer, and check kernel diagnostics:

Terminal Commandsbash
# 1. Enable SysRq triggers in kernel sysctl
sudo sysctl -w kernel.sysrq=1
 
# 2. Trigger the Out-Of-Memory (OOM) Killer manually
echo f | sudo tee /proc/sysrq-trigger
 
# 3. Read kernel ring buffer logs for OOM execution report
sudo dmesg -T | grep -i -E "oom-killer|killed process"

How the Linux Kernel Calculates oom_score

The Linux kernel evaluates every running user-space task using a heuristic scoring function (badness() in mm/oom_kill.c). The process with the highest calculated oom_score is terminated first.

  • Base Memory Consumption: oom_score is directly proportional to the percentage of total RAM and swap space consumed by the process (RSS + Swap).

  • Root Privilege Exemption: Processes running under root (uid 0) receive a slight score reduction (~3%) because administrative daemons are critical for system recovery.

  • Adjustment Offset (`oom_score_adj`): Developers can manually bias process selection using /proc/[pid]/oom_score_adj (values range from -1000 to 1000).

Inspecting and Adjusting OOM Scores

You can check the current OOM score for any process using its PID:

Terminal Commandsbash
# Inspect current OOM badness score for PID 1234
cat /proc/1234/oom_score
 
# Check user adjustment bias (default: 0)
cat /proc/1234/oom_score_adj
 
# Make a critical daemon completely immune to OOM Killer (-1000 score)
echo -1000 | sudo tee /proc/1234/oom_score_adj
 
# Make a sacrificial background worker the first target (+1000 score)
echo 1000 | sudo tee /proc/1234/oom_score_adj

Simulating Real Memory Pressure with C

To test how your system reacts under genuine Out-Of-Memory exhaustion, write a simple C stress allocator that fills physical RAM pages:

oom_stressor.cc
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
 
#define CHUNK_SIZE (100 * 1024 * 1024) // 100 MB per iteration
 
int main() {
    printf("OOM Stressor PID: %d. Consuming RAM...
", getpid());
    size_t total = 0;
    while (1) {
        char *ptr = malloc(CHUNK_SIZE);
        if (!ptr) {
            perror("malloc failed");
            break;
        }
        // Dirty allocated memory to force physical page allocation (bypass lazy allocation)
        memset(ptr, 0xEE, CHUNK_SIZE);
        total += CHUNK_SIZE;
        printf("Allocated %zu MB RAM
", total / (1024 * 1024));
        usleep(100000); // Sleep 100ms
    }
    return 0;
}

Analyzing the Kernel OOM Log Output

When the OOM Killer triggers, dmesg outputs a detailed memory dump listing all candidate tasks and their respective page counts:

dmesg Kernel Output Logtext
[ 3452.129481] Out of memory: Killed process 4821 (oom_stressor) total-vm:2097152kB, anon-rss:1982300kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:4096kB oom_score_adj:0