System Control Over Python With Pipes

Even though major system scripting is usually done via bash (or other shell) scripting, it’s almost as common to see Python as a system administration scripting platform. Today, most major installers, daemons and package management software are written in Python.

While using Python as a system scripting tool, it’s essential to keep using the standard gnu tools if possible, since these tools are reliable, fast and have been tested for years by thousands of other system administrators as well.

The best way to keep using your good old fashioned gnu tools via Python is opening pipes to such tools. Even though it is possible to use the os module to do such tasks, the subprocess module is more reliable and you have more control over the tasks.

Below you’ll see two different methods to approach the “echo” tool via Python.

>>> import os, subprocess
>>> os.system("echo bla")
bla
0
>>> subprocess.Popen(["echo","bla"],stdout=subprocess.PIPE).communicate()[0]
'bla\n'

As you can see, both give us more or less the same output, but the subprocess module gives us a lot more control. First of all it has pipe support, universal newline support, better handling of exceptions etc.

Below is a method I use to determine the number of processor on the system. Using this, you can find out if you have multiple cores on the system.

cpu_num=subprocess.Popen(["grep","-c","processor","/proc/cpuinfo"],stdout=subprocess.PIPE).communicate()[0]

As you can see, what is does is very very simple, it just counts the number of word “processor” in the /proc/cpuinfo file using grep.

Don’t forget that it is very easy to pipe commands through each other with popen, here’s an example.

>>> import subprocess
>>> p1=subprocess.Popen(["cat","/proc/cpuinfo"],stdout=subprocess.PIPE)
>>> p2=subprocess.Popen(["grep","-B","2","-A","2","processor"],stdin=p1.stdout,stdout=subprocess.PIPE)
>>> output = p2.communicate()[0]
>>> print output
processor : 0
vendor_id : GenuineIntel
cpu family : 15

Using the popen, it is very easy to get information on processes via the pgrep and ps tools. Below is a script when run with the valid parameters, checks if any of it’s processes uses more (or equal) to the percentage of the CPU specified, if true, kills the process and tries to send a message to the affiliated terminal window.

To accomplish this, we get the pid’s of the processes via pgrep, send it to ps so to check the cpu percentage and which tty (or pts) it’s running on. After the check, if necessary, the PID is killed and a message is sent to the terminal it was running on.

#!/usr/bin/python
import subprocess, sys, string

if len(sys.argv) != 3 :
    print "Usage : cpukiller.py <process-to-kill> <max-cpu-percentage>"
    raise SystemExit

proc_to_kill = sys.argv[1]
cpu_to_kill = sys.argv[2]

for i in cpu_to_kill :
    if i not in string.digits :
        print "max-cpu-percentage must consist of only digits."
        raise SystemExit

cpu_to_kill = int(cpu_to_kill)

try :
	a=subprocess.Popen(["pgrep",proc_to_kill],stdout=subprocess.PIPE).communicate()[0]
	if a == '' :
		raise SystemExit
	else :
		procc = subprocess.Popen(["ps --no-headers -o pid,pcpu,tname -p $(pgrep %s)"%proc_to_kill],shell=True,stdout=subprocess.PIPE).communicate()[0]
		procc=procc.strip()
except : raise SystemExit 
for lines in procc.split('\n') :
    if lines != '' :
        l=lines.split()
        if int(l[1].split('.')[0]) >= cpu_to_kill :
            if l[2] != '?' :
                try :
                    terminal = open('/dev/'+l[2],'a')
                    subprocess.Popen(["echo","You have exceeded your CPU limit, process with PID %s will be terminated"%l[0]],stdout=terminal).communicate()[0]
                except :
                    print "Couldn't tell the user."
                    pass
            try :
                killer = subprocess.Popen(["kill","-9",l[0]],stdout=subprocess.PIPE).communicate()[0]
            except :
                print "Couldn't kill the process."
                pass
        else : pass

Try to use it, understand it, and expand it!