环境

  • Red Hat Enterprise Linux (RHEL) 4
  • Red Hat Enterprise Linux (RHEL) 5
  • Red Hat Enterprise Linux (RHEL) 6
  • Red Hat Enterprise Linux (RHEL) 7

问题

  • How to disable the Out of memory or oom-killer?
  • How do I determine and configure the likelihood that a process will be killed in a out-of-memory situation?

决议

 

Disabling OOM-KILLER

 

  • Red Hat Enterprise Linux 4 :
  • Red Hat Enteprise Linux 4.2 and newer releases have the /proc/sys/vm/oom-kill tunable. Set this to 0 to disable the oom-killer.
  • /proc/sys/vm/oom-kill
    Raw
    # cat /proc/sys/vm/oom-kill 1 # echo 0 > /proc/sys/vm/oom-kill # cat /proc/sys/vm/oom-kill 0
  • /etc/sysctl.conf
    Raw
    vm.oom-kill = 0
  • Red Hat Enterprise Linux 5/6/7 :
  • Red Hat Enteprise Linux 5, 6 and 7 do not have the ability to completely disable OOM-KILLER. Please see the following section for tuning OOM-KILLER operation within RHEL 5, RHEL 6 and RHEL 7.

 

Tuning OOM-KILLER

 

  • Red Hat Enterprise Linux 4/5/6/7 :
  • If a kernel panic occurs in an out of memory condtion, check if panic_on_oom is set.
  • /proc/sys/vm/panic_on_oom
    Raw
    # cat /proc/sys/vm/panic_on_oom 1 # echo 0 > /proc/sys/vm/panic_on_oom # cat /proc/sys/vm/panic_on_oom 0
  • /etc/sysctl.conf
    Raw
    vm.panic_on_oom = 1
  • Red Hat Enterprise Linux 5/6/7 :
  • It is also possible to prioritize which processes get killed by adjusting the oom_killer score. In /proc/PID/ there are two tools labelled oom_adj and oom_score. Valid scores for oom_adj are in the range -16 to +15. This value is used to calculate the 'badness' of the process using an algorithm that also takes into account how long the process has been running, amongst other factors. To see the current oom_killer score, view the oom_score for the process. oom_killer will kill processes with the highest scores first.
  • This example adjusts the oom_score of a process with a PID of 12465 to make it less likely that oom_killer will kill it.
    Raw
    # cat /proc/12465/oom_score 79872 # echo -5 > /proc/12465/oom_adj # cat /proc/12465/oom_score 78
  • There is also a special value of -17, which disables oom_killer for that process. In the example below, oom_score returns a value of O,indicating that this process would not be killed.
    Raw
    # cat /proc/12465/oom_score 78 # echo -17 > /proc/12465/oom_adj # cat /proc/12465/oom_score 0
  • Setting "overcommit_memory" to 2, allows you to be precise about memory overcommitting. This instructs the kernel never to commit address space larger than swap space plus a fraction "overcommit_ratio" of the physical memory.
    Raw
    RHEL5: Commit = swap + (overcommit_ratio/100) * physical memory RHEL6, 7: Commit = swap + (overcommit_ratio/100) * (physical memory - hugepage tlb)
  • When memory usage hits this number no more allocations should be possible. Committed_AS in /proc/meminfo shows the currently committed amount of memory on the system.

诊断步骤

  • Out of Memory (OOM) refers to a computing state where all available memory, including swap space, has been allocated. Normally this will cause the system to panic and stop.
  • The vm.oom-kill for RHEL4 is provided by linux-2.6.9-proc-disable-oom.patch. That patch is not available for Red Hat Enterprise Linux 5 or above.