Poor man’s processes monitor (bash & awk script)

Recently a member of the Romanian Ubuntu Community asked for a script to monitor the running processes on his server . He didn’t requested for anything fancy, just a small utility that will be able to detect a hanging application (a process that is “eating” more than 80% of the CPU for a long period of time) and then log the results .

I am no sysadmin, but I am sure there are a lot of dedicated open-source solutions for monitoring a server .Still the functionality he asked for can be easily achieved by combining bash and awk .

One of the things I like Linux for is the power of the shell and the full-control over the your system . You can write a script for every repeating task that arise as bash is easy to learn but of course, hard to master .  More as an exercise for myself I’ve proposed the following solution :

#!/bin/bash

#DATE: Nov 5, 2010
#AUTHOR: nomemory

#Maximum memory for a process (%)
declare -i MEM_LIMIT=1

#Maximum CPU for a process (%)
declare -i CPU_LIMIT=1

#Loop sleep interval
declare -i SEC_INT=30

while true; do
ps aux | awk -v MEM_LIMIT=${MEM_LIMIT} 
        -v CPU_LIMIT=${CPU_LIMIT} 
        -v CDATE="`date`" '{
          if ($3 > CPU_LIMIT) {
            printf "%s [ %10s %d %40s ] CPU LIMIT EXCEED: %2.2f (MAX: %2.2f) n", 
              CDATE, $1, $2, $11, $3, CPU_LIMIT
          }
          if ($4 > MEM_LIMIT) {
            printf "%s [ %10s %d %40s ] MEM LIMIT EXCEED: %2.2f (MAX: %2.2f) n", 
              CDATE, $1, $2, $11, $4, MEM_LIMIT
          }
        }'
sleep ${SEC_INT}
done

If you run this script the output will probably look similar to this one :

Mon Nov  8 00:01:08 EET 2010 [     andrei 1718         /opt/google/chrome/google-chrome ] MEM LIMIT EXCEED: 2.20 (MAX: 1.00)
Mon Nov  8 00:01:08 EET 2010 [     andrei 1726                                   pidgin ] MEM LIMIT EXCEED: 1.40 (MAX: 1.00)
Mon Nov  8 00:01:08 EET 2010 [     andrei 1853                /opt/google/chrome/chrome ] CPU LIMIT EXCEED: 5.70 (MAX: 1.00)
Mon Nov  8 00:01:08 EET 2010 [     andrei 1853                /opt/google/chrome/chrome ] MEM LIMIT EXCEED: 2.70 (MAX: 1.00)
Mon Nov  8 00:01:08 EET 2010 [     andrei 2054                           gnome-terminal ] CPU LIMIT EXCEED: 1.50 (MAX: 1.00)
Mon Nov  8 00:01:08 EET 2010 [     andrei 2058                                     bash ] CPU LIMIT EXCEED: 1.70 (MAX: 1.00)

The output can then be redirected to a file (>>) and interpreted as:

How to get the active process list on a Linux machine using python

To retrieve the active process list on a Linux machine we will use the subprocess module . This module will allow us to create new subprocesses, connect to their input / output / error pipes, and eventually retrieve their exit codes .

It offers us a higher level and cleaner approach by defining only one class: subprocess.Popen(), intended to replace the functionality offered by methods such as: os.system(), os.popen() or os.popen2() .

To obtain the active processes and their associated information (USER, PID, CPU, MEM, TTY, STAT, TIME, COMMAND, etc.) we will spawn a ps aux subprocess, connect to its output pipe and parse the results .

But first let’s analyze a little the output of ps aux :

andrei@andrei:~$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   2872  1692 ?        Ss   14:15   0:00 /sbin/init
root         2  0.0  0.0      0     0 ?        S    14:15   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    14:15   0:03 [ksoftirqd/0]
root         4  0.0  0.0      0     0 ?        S    14:15   0:00 [migration/0]
root         5  0.0  0.0      0     0 ?        S    14:15   0:00 [watchdog/0]
root         6  0.0  0.0      0     0 ?        S    14:15   0:00 [migration/1]
root         7  0.0  0.0      0     0 ?        S    14:15   0:00 [ksoftirqd/1]
root         8  0.0  0.0      0     0 ?        S    14:15   0:00 [watchdog/1]
root         9  0.0  0.0      0     0 ?        S    14:15   0:00 [events/0]
root        10  0.0  0.0      0     0 ?        S    14:15   0:00 [events/1]
root        11  0.0  0.0      0     0 ?        S    14:15   0:00 [cpuset]
.....................................................................................................

Our first step will be to create a data structure capable of encapsulating the above information . We will write the Proc class that will wrap all the process meta-information present in the ‘ps aux‘ output :

#!/usr/bin/env python

from subprocess import Popen, PIPE
from re import split
from sys import stdout

class Proc(object):
    ''' Data structure for a processes . The class properties are
    process attributes '''
    def __init__(self, proc_info):
        self.user = proc_info[0]
        self.pid = proc_info[1]
        self.cpu = proc_info[2]
        self.mem = proc_info[3]
        self.vsz = proc_info[4]
        self.rss = proc_info[5]
        self.tty = proc_info[6]
        self.stat = proc_info[7]
        self.start = proc_info[8]
        self.time = proc_info[9]
        self.cmd = proc_info[10]

    def to_str(self):
        ''' Returns a string containing minimalistic info
        about the process : user, pid, and command '''
        return '%s %s %s' % (self.user, self.pid, self.cmd)

A list of Proc objects will probably do the job .

ROT13 (Caesar Cipher)

In this ProgrammingPraxis challenge we have to build a simple Caesar Cipher with a special property, called rot13. ROT13 ("rotate by 13 places", sometimes hyphenated ROT-13) is a simple substitution cipher used in online forums as a means of hiding spoilers, punchlines, puzzle solutions, and offensive materials from the casual glance. ROT13 has been described as the "Usenet equivalent of a magazine printing the answer to a quiz upside down".ROT13 is an example of the Caesar cipher, developed in ancient Rome . In this article there is a small "functional" implementation for ROT13 written in python .

Converting infix to RPN (shunting-yard algorithm)

If you've tried to write your own calculator (something in the style of gcalctool) you've probably had to build a simple converter for your mathematical expressions from infix notation to RPN (Reverse Polish Notation). Inspired by this classical SPOJ challenge I wanted to write my own simplified version of an expression converter. In order to parse and convert a given infix mathematical expression to RPN we will use the shunting-yard algorithm . Just like the evaluation of RPN, the algorithm is stack-based . To find out more please read the full article .

Generic data structures in C

I've decided to explore the strategies that will allow us to write generic code in C. By generic features I mean similitudes with C++ templates, or better, with Java/C# Generics . In order to prove that generic programming can be achieved in plain C, we will write an implementation of a generic stack data structure by following the two major approaches: using C macros to generate specific code (and some of the advanced features, x macros, macro concatenation, macro stringification, macro arguments) or using the void pointer (void*) as the good old container that will hide the specificity of a certain type, but will allow casting. The second approach is the one used in GLib.