Debugging Linux OOM Killer and Memory Leaks: A Production Root Cause Analysis Playbook

It is the middle of the night. Your monitoring system fires a critical incident alert: a core microservice or background worker container in production abruptly vanished without logging an uncaught exception, a stack trace, or a panic. The container status simply reads OOMKilled: true with Exit Code 137.

Memory issues in production are among the most elusive bugs in distributed systems. Memory metrics displayed in standard dashboards (like Prometheus or Datadog) often appear normal right up until the process terminates. Traditional profiling tools may slow down production traffic, and restart scripts merely mask the problem until the next traffic surge.

To systematically troubleshoot and eliminate these crashes, engineers must understand the mechanics of the Linux Out-Of-Memory (OOM) Killer, how the Linux kernel evaluates memory pressure, how cgroup v2 memory accounting works inside Docker and Kubernetes, and how to use tools like /proc/smaps_rollup and eBPF memleak to identify the exact line of code leaking bytes.

In this production debugging playbook, we deconstruct the anatomy of Exit Code 137, analyze kernel dmesg murder confessions, demystify cgroup v2 page cache vs. anonymous memory traps, diagnose language-specific leak patterns (Node.js, Go, Python), and build an automated diagnostic script.

The Anatomy of Exit Code 137

In Unix and Linux systems, when a process terminates due to an unhandled signal, the exit status code follows a standard mathematical convention:

Exit Code = 128 + Signal Number
Exit Code 137 = 128 + 9 (SIGKILL)

Because SIGKILL (Signal 9) cannot be caught, blocked, or handled by user-space code, the application runtime never gets the opportunity to execute cleanup hooks, flush error logs, or print a stack trace. The process is instantly vaporized by the operating system kernel.

flowchart TD
    Crash[Process Terminated with Exit Code 137] --> Check{What triggered SIGKILL?}
    Check -->|Triggered by Linux Kernel| SysOOM["1. System-Wide OOM Killer
• Host physical RAM + Swap exhausted
• Kernel selects victim via oom_score heuristics"]
    Check -->|Triggered by Container Runtime| CgroupOOM["2. cgroup v2 Memory Limit Exceeded
• Host has plenty of free RAM
• Container breached its memory.max quota"]
    Check -->|Triggered by Operator / Orchestrator| ManualKill["3. External SIGKILL
• Docker stop timeout (SIGTERM -> SIGKILL)
• Kubernetes failed liveness probe"]

    style Crash fill:#e11d48,stroke:#fda4af,color:#fff
    style SysOOM fill:#f97316,stroke:#fdba74,color:#fff
    style CgroupOOM fill:#e11d48,stroke:#fda4af,color:#fff
    style ManualKill fill:#64748b,stroke:#94a3b8,color:#fff

Reading the Kernel Murder Confession: Decoding dmesg

Whenever the Linux kernel invokes the OOM killer, it writes a detailed autopsy report to the kernel ring buffer. To view it, run:

$ sudo dmesg -T | grep -i -E "(out of memory|oom_reaper|killed process)"

A typical production OOM log looks like this:

[Sun Oct 11 04:12:38 2026] node invoked oom-killer: gfp_mask=0x1100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=998
[Sun Oct 11 04:12:38 2026] CPU: 3 PID: 84920 Comm: node Not tainted 6.8.0-45-generic
[Sun Oct 11 04:12:38 2026] memory: usage 2097152kB, limit 2097152kB, failcnt 18421
[Sun Oct 11 04:12:38 2026] Memory cgroup stats for /kubepods.slice/kubepods-burstable.slice/pod123...:
[Sun Oct 11 04:12:38 2026]   anon 1985412kB
[Sun Oct 11 04:12:38 2026]   file 98420kB
[Sun Oct 11 04:12:38 2026]   kernel_stack 4096kB
[Sun Oct 11 04:12:38 2026]   pagetables 9216kB
[Sun Oct 11 04:12:38 2026] Tasks state (memory values in pages):
[Sun Oct 11 04:12:38 2026] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[Sun Oct 11 04:12:38 2026] [ 84920 ]  1000 84920   612450   524288        9437184        0           998 node
[Sun Oct 11 04:12:38 2026] oom-killer: killed process 84920 (node) total-vm:2449800kB, anon-rss:1985412kB, file-rss:98420kB

Key Metrics to Analyze in the Log:

• limit 2097152kB: The exact hard memory limit (here: 2GB) that was breached.
• anon 1985412kB: Anonymous Memory (Heap + Stack). This is memory allocated dynamically via malloc() or runtime heaps. Unlike file cache, anonymous memory cannot be dropped by the kernel unless written to swap! Here, anonymous memory consumed 95% of the limit—confirming an active application heap leak.
• file 98420kB: Page Cache (Disk I/O cache). Reclaimable memory holding cached files from disk.
• oom_score_adj: The badness adjustment score. In Kubernetes, pods with BestEffort QoS receive 1000, Burstable pods receive up to 998, and Guaranteed pods receive -997.

The cgroup v2 Memory Trap: Page Cache vs. RSS

One of the most frequent sources of confusion for developers is why monitoring dashboards show a container at 98% memory usage for days without crashing, while another container crashes at 70% usage.

Inside Linux cgroup v2 (located at /sys/fs/cgroup/memory.*), memory accounting is split into two primary components:

flowchart LR
    TotalMem["cgroup memory.current (Total Usage)"] --> Reclaimable["1. Inactive File Page Cache
(Disk I/O, logs, temp files)
✅ Reclaimable under pressure without killing"]
    TotalMem --> NonReclaimable["2. Active Anonymous RSS
(App Heap, Stack, Native Buffers)
❌ NON-RECLAIMABLE without swap!"]

    NonReclaimable -->|Grows beyond memory.max| OOM["💥 KERNEL INVOKES OOM KILLER"]

    style TotalMem fill:#0284c7,stroke:#38bdf8,color:#fff
    style Reclaimable fill:#059669,stroke:#6ee7b7,color:#fff
    style NonReclaimable fill:#e11d48,stroke:#fda4af,color:#fff
    style OOM fill:#e11d48,stroke:#fda4af,color:#fff

Why free -m Inside Containers is Misleading

Legacy tools like free or top read from /proc/meminfo, which reports the entire host machine's memory (e.g. 128GB RAM), rather than the container's 2GB cgroup ceiling! Always inspect the actual cgroup virtual files:

# Current memory usage
$ cat /sys/fs/cgroup/memory.current

# Memory hard limit (Breach triggers OOM)
$ cat /sys/fs/cgroup/memory.max

# OOM events counter
$ cat /sys/fs/cgroup/memory.events

If memory.events shows oom_kill > 0, the container was killed by the cgroup controller.

Deep Inspection with /proc/[pid]/smaps_rollup

To inspect a live running process without attaching invasive debuggers, read its Proportional Set Size (PSS) and Unique Set Size (USS) from /proc/[pid]/smaps_rollup:

$ cat /proc/$(pgrep -f "my-app")/smaps_rollup

Rss:              1482904 kB    # Resident Set Size in physical RAM
Pss:              1478120 kB    # Proportional Set Size (accounting for shared libs)
Pss_Anon:         1420100 kB    # PSS Anonymous (The true application heap size!)
Pss_File:           58020 kB    # PSS mapped files
Private_Clean:      12400 kB    # Unmodified shared memory
Private_Dirty:    1420100 kB    # Memory modified exclusively by this process

If Pss_Anon or Private_Dirty grows monotonically over time regardless of incoming request traffic, your application has a confirmed memory leak.

Language-Specific Leak Signatures & Root Causes

Advanced Production Profiling: Zero-Overhead eBPF memleak

When an application leaks memory through native C extensions, database drivers, or glibc allocators, standard language heap profilers cannot see the leak. In production, attaching valgrind introduces a 20x to 50x slowdown, making it impossible to run on live traffic.

The modern zero-overhead solution is eBPF memleak (part of BCC / bpftrace tools). It attaches kernel probes to user-space malloc() and free() calls to find allocations that are never freed:

# Trace top outstanding allocations in PID 84920 older than 60 seconds
$ sudo memleak-bpfcc -p 84920 --older 60000 -T 5

Sample eBPF output showing the exact call stack responsible for the leak:

[04:35:12] Top 1 stacks with outstanding allocations:
  184,320 bytes in 45 allocs of size 4096:
    __GI___libc_malloc+0x18
    allocate_image_buffer+0x42 [/usr/lib/libgraphics.so]
    process_upload_request+0x18a [/app/bin/server]
    handle_client_connection+0x94 [/app/bin/server]
    start_thread+0x384 [/usr/lib/libc.so.6]

Within seconds, eBPF identifies that allocate_image_buffer inside libgraphics.so is allocating 4KB buffers without a corresponding free() call!

Hands-On Diagnostic Script: Production Process Memory Analyzer

Here is an automated, robust Bash diagnostic script you can drop onto any production Linux server or Kubernetes pod to instantly audit memory distribution, cgroup limits, and OOM risk:

#!/usr/bin/env bash
# ==============================================================================
# Linux Production Memory & OOM Diagnostics Script
# ==============================================================================
set -euo pipefail

TARGET_PID="${1:-}"

if [[ -z "$TARGET_PID" ]]; then
    echo "Usage: $0 "
    exit 1
fi

if [[ ! -d "/proc/$TARGET_PID" ]]; then
    echo "Error: Process $TARGET_PID does not exist."
    exit 1
fi

echo "========================================================"
echo " Process Memory Audit for PID: $TARGET_PID ($(cat /proc/$TARGET_PID/comm))"
echo "========================================================"

# 1. Inspect OOM Score and Badness
OOM_SCORE=$(cat /proc/$TARGET_PID/oom_score)
OOM_ADJ=$(cat /proc/$TARGET_PID/oom_score_adj)
echo "[OOM Scoring]"
echo "  Current oom_score:     $OOM_SCORE (Higher = First in line to be killed)"
echo "  Current oom_score_adj: $OOM_ADJ (-1000 = immune, 1000 = kill target)"

# 2. Inspect smaps_rollup metrics
if [[ -f "/proc/$TARGET_PID/smaps_rollup" ]]; then
    echo -e "\n[Physical Memory Breakdown (smaps_rollup)]"
    grep -E "(Rss:|Pss:|Pss_Anon:|Private_Dirty:)" /proc/$TARGET_PID/smaps_rollup | awk '{printf "  %-20s %s %s\n", $1, $2, $3}'
fi

# 3. Inspect cgroup v2 boundaries
CGROUP_PATH=$(cat /proc/$TARGET_PID/cgroup | cut -d: -f3)
FULL_CGROUP_DIR="/sys/fs/cgroup$CGROUP_PATH"

echo -e "\n[cgroup v2 Resource Limits]"
if [[ -d "$FULL_CGROUP_DIR" ]]; then
    if [[ -f "$FULL_CGROUP_DIR/memory.max" ]]; then
        MAX=$(cat "$FULL_CGROUP_DIR/memory.max")
        CUR=$(cat "$FULL_CGROUP_DIR/memory.current")
        echo "  cgroup Path:           $CGROUP_PATH"
        echo "  memory.current:        $(( CUR / 1024 / 1024 )) MB"
        if [[ "$MAX" == "max" ]]; then
            echo "  memory.max:            No limit configured"
        else
            echo "  memory.max:            $(( MAX / 1024 / 1024 )) MB"
            USAGE_PCT=$(( CUR * 100 / MAX ))
            echo "  cgroup Saturation:     $USAGE_PCT%"
        fi
    fi
    if [[ -f "$FULL_CGROUP_DIR/memory.events" ]]; then
        echo "  cgroup Events:"
        awk '{printf "    %-18s %s\n", $1, $2}' "$FULL_CGROUP_DIR/memory.events"
    fi
else
    echo "  Standard cgroup directory not found or cgroup v1 in use."
fi

echo "========================================================"

Production Remediation Checklist

1. Configure JVM / Node.js Runtime Ceilings Below cgroup Limits: If a container is restricted to 2GB in Kubernetes (resources.limits.memory: 2Gi), configure the runtime heap ceiling to ~75% of that limit (e.g., --max-old-space-size=1536 for Node.js or -XX:MaxRAMPercentage=75.0 for JVM) to leave sufficient headroom for native off-heap allocations, thread stacks, and glibc overhead.
2. Adjust oom_score_adj on Critical Daemons: Protect critical databases (PostgreSQL, HAProxy) from accidental kernel termination by setting echo -900 > /proc/$(pgrep postgres)/oom_score_adj.
3. Avoid Unbounded In-Memory Caches: Never use plain JavaScript/Go maps as global caches without strict Least Recently Used (LRU) eviction and maximum entry size limits.
4. Implement Early Memory Warning Alerts: Set Prometheus alerts on container_memory_working_set_bytes / container_spec_memory_limit_bytes > 0.85 to detect leaks hours before the OOM killer triggers.