So I like nothing more than to spend the early hours of every morning looking at anaconda source[0] in the hopes of learning more about some of the more fun things one could to do with Python, if one felt so inclined. Actually, that’s partly true – I am trawling through the anaconda source at the moment of my own volition – everything from figuring out the pipe communication startup hacks in mini-wm/Anaconda to the isys library. Why? Because I work at a company that has heavy reliance on Python and feel I should better understand the language, and use it more in my own work as a result. Besides, I’m not a perl weenie, I should write fewer shell scripts, and I’m not particularly desperate to go near a Java compiler, having been subjected to that more than quite enough in college.
I actually quite like Python. I’ve written a bunch of scripts using it, but nothing huge and I tend to write very C-like imperative code in “OO” languages – even when I’ve played with pygtk in the past. Still, that’s just fine. One of the things I have been doing is reading up on the more interesting features of the language, best practices, and the like. I’m using the two standard O’Reilly books as a reference and the online p.o documentation. The point of the exercise being that I’m reasonably keen to know how I “should” approach writing Python code, as opposed to my hackish approach in the past. I want to understand the intricacies of classes in Python, pickling, the internal differences between pyc and pyo “optimised” bytecode and some of the more funky APIs – like the HAL/gtk bindings I’m going to be playing with some more.
Anyway. That’s all fairly boring. Not having looked at Python’s more funky features in a while, I wanted to dive right in and write some simple C/API plugin to play with Python Objects. This turns out to be a little harder these days than the (normally excellent, but in this case not so much) python documentation would have you believe. Here’s a simple test “library” I wrote to figure it out:
/*
* pytest - playing with Python's C API.
*/
#include <python .h>
static PyObject *list_set(PyObject *list, PyObject *item);
void initpytest(void);
static PyMethodDef pytestModuleMethods[] = {
{ "list_set", (PyCFunction) list_set, METH_VARARGS, NULL },
{ NULL, NULL, 0, NULL }
};
void initpytest(void)
{
PyObject *m;
m = Py_InitModule("pytest", pytestModuleMethods);
}
static PyObject *list_set(PyObject *self, PyObject *args)
{
PyObject *list, *value;
int item;
if (!PyArg_ParseTuple(args, "OiO", &list, &item, &value))
return NULL;
if (!PyList_Check(list))
goto error;
if (PyList_SetItem(list, item, value))
goto error;
Py_DECREF(value);
return Py_None;
error:
Py_XDECREF(value);
return NULL;
}
You can compile this into a dynamically loadable (read: Python can use it) library on most Linux systems by driving the gcc front end, thus:
gcc -o pytest.so -g -shared -fPIC -Lpython -I/usr/include/python2.4 pytest.c
(this enables debugging, “-g”, and assumes a default 2.4 python installation, such as that on my Fedora Core 6 desktop box that I’m using today).
Once built, such libraries can be used very simply:
[jcm@perihelion pytest]$ python
Python 2.4.3 (#1, Oct 1 2006, 18:00:19)
[GCC 4.1.1 20060928 (Red Hat 4.1.1-28)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pytest
>>> foo=[1,2,3]
>>> print foo
[1, 2, 3]
>>> pytest.list_set(foo,0,2)
>>> print foo
[2, 2, 3]
It’s a simple example, pytest provides a single function that changes values in Python lists. Everything in Python is an object, and there is a need to be aware of reference-counting that’s happening underneath (well, sometimes you need to worry about it). Every C/API extension needs to define an init function, which educates the Python runtime about those functions it provides – in this case, the list_set function. These are defined in a NULL-terminated PyMethodDef array, and usually defined to require METH_VARARGS argument passing (I’m sure this was once different).
Actual functions, such as list_set work in PyObjects (everything’s an object) too, using PyArg_ParseTuple to pull out C-style arguments, according to a format string. They also need to use occasional calls to Py_DECREF (and it’s NULL-friendly wrapper counterpart, Py_XDECREF). Though this varies – for example, in the above, PyList_SetItem “steals” a reference to “list” and so I don’t need to worry about freeing that particular reference. The documentation tries to explain what needs tracking.
Anyway. This has been a random post. For those of you who are already die-hard Python programmers, I’m sure this is old-hat – and that’s appropriate to us now-officially-old uncle-types (my sister just gave birth last night). For me, it’s stuff I looked at too long ago but didn’t have a need for. Now that I’m looking at writing more funky stuff using Python, I’m working on becoming a Python nut, just like you 
Jon.
[0] The installer that was originally written by Red Hat (and used by a metric fucktonne of other people and projects needing a graphical Linux* installer not based YaST, Debian, or their own custom code). The thing that I (blindly, thanks to my previous Debianiteness/Ubuntu craziness that I’m recovering from over time) used to criticize because it wasn’t the d-i du jour. Actually, anaconda is a very powerful Linux installer that gets a lot of things right. And when you stop blindly agreeing with Debian rhetoric, this becomes abundantly more apparent (this isn’t an anti-Debian rant).
Anaconda’s source can probably best be described as something that seems to have evolved over time. There are top level classes (like Anaconda) and some OO concepts in there, but it’s still largely a tangled mess of functions/function pointers (method object references, whatever) that works out in the end. Not that I’m being critical – I’m sure I couldn’t do it any better – but it’s certainly useful to pick at
* I think some people have used anaconda for non-RPM distros, but I’m not aware of anyone trying to install weird stuff like Open Slowaris with it. Random aside: Sun seem to have stopped holding up a new edition of Slowaris Internals. There’s a new edition in Borders, covering Open Solaris and Solaris 10. It’s a shame it’s a bazillion years too late (the last edition was on Solaris 7, and like, totally ruled, dude) but then, I expected nothing less than to have to wait almost a decade for an updated edition. At least they now give out source code to the OS – unlike the many times I asked for it as a student and was told the programme was temporarily suspended…a million years ago.
