Wednesday, May 16, 2012

Python subprocess daemon manager: Pylot

One of the things I usually need is to execute subprocess daemons from python scripts to run task or even to offer some service that I need to be controlled by python code.
Executing subprocesses from python is quite easy with the Popen function from the subprocess module. You can exec subprocesses using an "exec" call, using a shell and you have a lot of parameters. This is quite cool, but sometimes you need to manage many daemons and you need them to be killed if the parent process dies and relaunched if they die. And there is other problem, if you send a SIGKILL signal to the parent process, the child processes will keep running and uncontrolled by any other program.
That is the reason I've implemented a module called Pylot. This module has two clases, Pylot and PylotManager. The first one is a thread that will execute the subprocess you want and other subprocess that acts as a watcher: if the main process dies, all your subprocesses will be killed by their watchers. Why to use watchers instead of capturing the SIGKILL signal??? Well, that is because... you can't capture the SIGKILL signal!
The PylotManager class will be used as an interface to handle the Pylots. You will be able to create, delete, check status and access the Pylots from PylotManager. PylotManager will check for the status of the Pylots and will relaunch them if they die, so it keeps all the daemons up and running.
Let's see a little example of how it works:

# Import PylotManager
from Pylot import PylotManager

# Create a new PylotManager and start it
pm = PylotManager()

# Once started, we can add a pylot for our daemon: htop
# NOTE: we set flush_output to False so we can later access
#   stderr and stdout. By default, flush_output is True so
#   you do not have to read the process output and you save
#   memory.
pm.addPylot('myfirstPylot','htop', flush_output=False)

# Now we access to our pylot
p = pm.getPylots()['myfirstPylot']


# Lets stop our pylot

# Now lets stop PylotManager. All remaining pylots will also
# be stopped
As easy as it looks like. All the code is commented so you can see all the functions and parameters with the python help() command.
You can get the code from and you are free to use it and change it. If you find some bug or have a feature request, just comment.

Bye for now!