Archive for April, 2009

quote for today

In ancient times skillful warriors first made themselves invincible,
And then watched for vulnerability in their opponents.
Making yourself invincible means knowing your self,
Seeing vulnerability in opponents means understanding others.
Sun Tzu ~ The Art of War

Leave a comment

using awk inside makefiles

Got bitten by this early in the morning. When using awk inside a makefile, many of awk’s built-in variables ($0..n, $NF, $FS, etc) disappear because make substitutes the dollar sign and first letter out into a (presumably) undefined value, leaving you with a mess that doesn’t work. 😛

After a little time with google, it transpires that the solution is to use two dollar signs, which make doesn’t interpolate, leaving you with a working script and leaving me with only this horror to content with (some formatting changes made so wordpress renders correctly) 🙂

find src -type f
  | awk -F "/"
  '{ print substr($$0, 0, index($$0, $$NF) - 1) }'
  | sort -u
  | awk -v cwd=$(CWD) '{ 
  print "{""\""$$0"*\",[\ndebug_info,\n{outdir, "
    {i,"cwd"/include}\n]""}." }' >> Emakefile

, ,

1 Comment

testing django with badrabbit

One of the design decisions I made when writing badrabbit was to make it fully compatible with unittest (one of the two unit testing frameworks that come with the python stdlib). My reasoning was twofold, firstly to minimize the amount of disruption caused by having to learn a new API and secondly, to give folks a chance to integrate badrabbit‘s mocking library and test decorator syntax in to existing tests with a minimum of fuss.

This decision paid off recently when I tried integrating badrabbit unit tests into a django application. Django has built in support for running unit tests as part of the development lifecycle. The script that gets generated when you run django’s equivalent of Rails’ scaffolding, takes one of a several commands and one of these is ./ test <appname>.

This has the effect of going off a running all your tests, where the tests for an application are either in your model definition ( a file named, or contained in (multiple) modules in a package of the same name. By default as described here, django will go off and look for tests defined with the unittest and doctest frameworks, both of which are part of python‘s stdlib. You can override this behavior by providing your own test runner and doing so is, in fact, ridiculously easy. Nevertheless, this isn’t neccessary because badrabbit is 100% compatible with unittest. And so all you need do to use it in your tests is put and import statement in your script, subclass badrabbit.testmagic.autotest instead of unittest.TestCase and you’re off. Here’s an abridged example:

#!/usr/bin/env python

import badrabbit
from badrabbit import *
from users.models import ServiceUser

class ServiceUserTests(autotest):
    def it_should_set_absolute_uri_correctly(self):
        user = ServiceUser(username='Johannes')

Of course, django has its own subclass of unittest.TestCase already, making subclassing badrabbit.testmagic.autotest less attractive (not least since they share the same superclass!) so there is another way. If you were to go take a peek at the implementation of badrabbit‘s autotest class you would soon notice that it does nothing apart from setting the metaclass to AutoTest:

class AutoTestCase(unittest.TestCase):
    __metaclass__ = AutoTest
autotest = AutoTestCase

So instead of subclassing autotest you can just as well set the metaclass (in exactly this way) on your own test class which can then happily subclass django’s TestCase instead.

I can’t imagine the integration being much simpler really. Because badrabbit depends on hamcrest-python, which doesn’t appear to come with an installer, we stuck it a couple of directories above our code and run the utility using a wrapper shell script which configures the PYTHON_PATH environment variable to include this first.

Leave a comment

urlconf for erlang

O2 and I are at it again – this time with a mixture of python, xslt and erlang soup’n up to provide what will be the best and best value e-marketting tool around. The python part is that our shop window is written in django, with lots of jQuery magic providing a slick UI. I’m particularly fond of jQuery and my favoured style of web development is to implement the back-end as a RESTful web service and then consume and decorate this web exposure with a dynamic UI. The reason behind django (as opposed to one of the millions of other web frameworks) is that we both know python inside out. And I get plenty of ruby hacking done both at work (where we are near OCD about capistrano) and in developing much of axiom.

So where does the erlang part come in? Well that’s not all up for public consumption, but suffice to say there is a fair chunk of what we’re planning to do which is what I would call “infrastructure programming” and that is something that erlang is basically perfect for. Not least of what we need is a distributed, fault tolerant grid without spending much money either building or hosting the thing, and again, we’re on to the right tool for the job.

Some of what we’ll be doing involves handling http requests, and for that, I’m currently in favour of using either mochiweb or yaws. For our needs, the former is a simpler choice but one thing that mochiweb doesn’t provide out of the box is a way to map requests to the appropriate chunk of request handling code. Our latest open source forray aims to add this capability, albeit in a way that isn’t specifically tied to mochiweb.

Enter urlconf-erlang. There’s no link because the source code still lives in a private repo and we’re not ready to release it yet. 🙂

Inspired by someone’s laudible effort to implement the django templating language in erlang, we decided to implement something akin to django’s url mapping system for our own evil purposes. Our current implementation only really deals with a single layer of mappings and does so by simply matching the url against a regex and returning the MFA (Module, Function, Args) configured against it. A quick peek at one of the test cases should make this a bit clearer:

named_arguments_supported_by_regex_impl(_) ->
    Path = "/users/fwnext-retail/billing-address",
    Rx = "/users/(?.*)/(?.*)$",
    Config = {
        [{Rx, my.controllers.users, {
            [uname, subcomponent]
            {mfa, {
                {uname, "fwnext-retail"},
                {subcomponent, "billing-address"}

As you can see there is some cruft because none of the Erlang regex libraries support named capture groups particularly well. In addition to this, we’re using the .re top level module (which is still not officially supported in R12B) as it is the only implementation we’ve found that is both fast and reliable. Because the choice of regex ibrary is likely to be a contentious point, we’ve taken the step of making the urlconf server a parameterized module, allowing you to pass in your own regex implementation instead of ours.