Andy McKay's blog

Headed down for the Northwest Python Day on Saturday. Seattle really seems to be becoming a home away from home these days, I was surprisingly able to easily navigate around and knew a lot of people. I went down Friday night and stayed over and had a few beers and chat with Jon Stahl and his wife.

Despite that I still somehow ended up getting there at 10:30, most of which was figuring out that if you go into the University of Washington, one way and one way only since that way has the parking ticket booth. If you go in any other way you are left puzzling over how to get a ticket.

There were a few talks, the two highlights for me were:

Google App Engine talk was great, as I've deployed stuff to the App Engine, but held back on anything big. There's too many wierd thing's limitations and so on. It's brilliant wonderful and I love the deployment. But running unit tests is pain, dropping into a pdb is a pain and the limits worry me. I do have the impression it feels like Zope in the old days (I'm sure Guido doesn't read this, but if you do - sorry for comparison), for different reasons, but it makes using the tools I know and use hard.

The talk about Sage was brilliant had all the wow factors you could ever need. I'm not a math head, things like matlab are completely beyond me, but the stuff he was demoing in Sage looked brilliant. Clearly a bunch of very bright and intelligent people have worked on that project. Scarily so.

I met a bunch of new nice folks including John DeRosa and Ted Leung. Also nice to hear that Brian Gershon is happily building up a cool business building Django and Plone sites.

In what seems to be a mad flurry of meetings, it's off to Vancouver Python meetup Tuesday.

The Seattle Python user group, SEAPIG are doing a "Northwest Python Day" in Seattle. A free day of talks and socializing around Python.

When: Saturday, January, 31st 9am - 5pm
Where: University of Washington campus, Seattle, Washington
Price: Free!

More information: http://www.seapig.org/NorthwestPythonDay

I'm probably heading down to it, if anyone would like to car share, please drop me a line.

Today I had a small interesting project I didn't quite complete, but should do in the morning its adding in a python authentication wrapper pulling from our LDAP server so that we can control our SVN and Trac access. We've already sort of got one, but it requires LDAP munging to move people around and instead we are moving that up to Python and so may as well give it a web interface in Django.

Some of the things I learned:

• mod_python has a very useful handler called PythonAuthenHandler that allows you to just plugin a directive to get a peice of Python to handle authentication - documentation is online.
• There's a sample in the Django docs of using this handing directive.
• When it doesn't work you can get to a pdb in mod_python, called PythonEnablePdb, that is documented here.
• You will then need to start Apache using -DONE_PROCESS and then you get a pdb and bob is your uncle.

Until you realise the LDAP data is dodgy. Tthere I was about to jump ship to mod_wsgi, but got the job done for now (almost)

DjangoCon 2008 Keynote: Cal Henderson, quite a bit swearing in there but i was laughing quite a lot to cover it up in the office.

Alright who's not working on a blogging engine in Django? Why not?

Only about 12 mins in...

I've posted I was looking for a user agent parser in Python. Yay I found one. I ended up at the browsercaps project which looks like its probably the biggest and most conclusive list of user agents out there. From that I found a Django snippet and there we go an excellent user agent parser in a few mins. Only slight problem seems to be the time taken to run the parser, but I might move that to an extra process anyway.

Dear lazy web, I've been looking for a while for a good user agent parser in Python. I've looked in something called sisynala and a few others to no available. Most web analysis software is not written in Python (usually Perl or PHP) and I might have to port one over. Anyone know of one?

Always nice to remember a feature in Python that I haven't used for ages. This happened to me the other day with this... given a list:

x = ["hair", "brown", "eyes", "blue"]

How do I get the hair and eye colours easily? One way, the oft forgotten 3rd argument to list slicing:

Python 2.5.1 (r251:54869, Apr 18 2007, 22:08:04)
>>> x = ["hair", "brown", "eyes", "blue"]
>>> for z in zip(x[::2], x[1::2]):
...   print z
... 
('hair', 'brown')
('eyes', 'blue')

For people like me with multiple pythons installations and can never remember his current modules are installed, I added the simple script, where-python:

python -c "import $1; import os; print os.path.dirname($1.__file__)"

And hence:

$ where-python django
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/django

This post on eWeek (via Slashdot) had me annoyed.

1. It's a Ruby conference. "Microsoft is pushing a different scenario, known as Asynchronous Ruby and XML, or ARAX". Pushing isn't the word I would use, showing off that silverlight can host scripting languages is more likely.
2. Yes writing Ruby as opposed to the RJS stuff can be more productive? Well duh, I've blogged about this before and my opinion of RJS is low. There's nothing wrong with JavaScript and learning it is not difficult.
3. Silverlight supports lots of languages, in fact it supports CLR, .NET and hence a lot of other stuff. A nice overview is here and information about the DLR is here.
4. Want to use Python with Silverlight in your browser, well go for it using IronPython. Oh wait is Microsoft pushing APAX now?

Ruby on the browser, c'mon pick a language that's popular. Or perhaps it will just end up in a decent browser anyway.

For a while I've been working on a site for a great client of ours. It's a Zope 2 site that uses ZSQLMethods, (Script) Python and ZSQL Methods to pull together a site, it's been around for a long time and working well. Recently we've been adding some functionality to it and I wanted to write some new functionality.

Sadly that does mean manually writing some: SQL, Python and Page Templates. I was just about to start it all off and in the back of my head was toying just importing Django and using its ORM, model and form code from within Zope (would that work, perhaps)?

But for the moment I've just prototyped it all out in Django. It took about one hour to create the models and relations and add in the tests. Testing is quick and easy, the unit tests run nice and quickly. The free UI gives me a quick sanity check that everything is fine.

All I have to do now is spend 4 days translating that prototype into ZSQLMethods and associated stuff. Fortunately I can now steal all the sql from django queries.

Update: this helps.

One nice thing about Google Web Apps, is that you can automatically log in with a Google account. That is if you live in the US. Otherwise it doesn't work. Well that saves me from working on it tonight. Look forward to seeing that bug fixed.

Update: appears to be fixed, that was fast. Hopefully they won't take down my app when I write it though.

With some fanfare Google Web Apps have been launched. This is great because its using Python and Django, how much more perfect can it get. This morning I've been having a play and I'm just waiting for my account so I can upload something.

Would be nice to take my (as yet unreleased) Plone to Django conversion script, plonk the result on Google Web Apps and then link up a nice little Adobe Air app. Anyway two things that irked me (of course I'm going to complain): 1) the example Python all has two spaces as the indent and 2) it uses Django templating.

Fortunately it says:

You can bundle a framework of your choosing with your application code by copying its code into your application directory

And looking at later comments, it's not going to block a module like simpletemplates, which I use for templating in Django. So. lets alter helloworld.py a bit (original):

from simpletemplate.google import DjangoSimpleTAL

And change the MainPage class to read (only changed last two lines):

class MainPage(webapp.RequestHandler):
    def get(self):
        greetings = Greeting.all().order('-date')

        if users.get_current_user():
            url = users.create_logout_url(self.request.uri)
            url_linktext = 'Logout'
        else:
            url = users.create_login_url(self.request.uri)
            url_linktext = 'Login'

        template_values = {
          'greetings': greetings,
          'url': url,
          'url_linktext': url_linktext,
          }

        path = os.path.join(os.path.dirname(__file__), 'index.html')  
        template = DjangoSimpleTAL(path)
        self.response.out.write(template.render(template_values)

And finally lets change index.html to read:

<html>
    <body>
        <tal:loop repeat="greeting greetings">
            <tal:block condition="greeting/author|none">
                <b tal:condition="python:greeting.author.nickname">Author</b> wrote:
            </tal:block>
            <tal:block condition="not:greeting/author|none">
                An anonymous person wrote:
            </tal:block>
            <blockquote tal:content="greeting/content">Comment</blockquote>
        </tal:loop>

        <form action="/sign" method="post">
            <div><textarea name="content" rows="3" cols="60"></textarea></div>
            <div><input type="submit" value="Sign Guestbook"></div>
        </form>

        <a tal:attributes="href url" tal:content="url_linktext">Login</a>
  </body>
</html>

And things seem to work just dandy. For those who do use simpletemplate, the google module isn't in SVN yet, and needs a bit of work, but just a quick refactoring of the simpletemplate code to be friendly with the new imports that the Google app server uses.

Update: by the time my blog post was done, my account was activated. Unfortunately I have to work today though.

Just so I don't get forget, the correct syntax for setting a quota in imaplib against cryus is:

size = 100000
imap.setquota(account, "(storage %s)" % size) 

Won't say this often, but thank you Perl doc's for showing me the correct syntax.

At PyCon there was a talk "How to use Grok to Walk like a duck" that makes a great read. It's about how the Zope 3 component framework. A great line in the presentation is:

Ruby people do this

... a reason for why you shouldn't do monkey patching. Presentation at: http://rhodesmill.org/brandon/adapters/.

Alright Microsoft has a long way to go to winning the hearts and minds of developers. But on this article talking about IronPython and the ability to access .NET. That in itself is cool, but we've know about that for a long time. But there is also an advert for Visual Studio. Using Halo characters. Now that is pretty cool.

Prompted by this article on Chandler, I did a little bit of digging on Chandler. I remember Chandler being launched and a bunch of people sending me emails about this Outlook killer. They were all very excited by it, but me being the grumpy git said "yeah right". I was mildly interested for one reason: it was written in Python.

Like most people I tried some builds and then some more and still there wasn't anything there. It was a shame and it quickly dived off the radar. Looks like the application is reaching a turning point and may or may not be around much longer. A quick review sounds like its the planning and not the code that's the problem.

Joel pointed out the problems you get trying to do an open source project, something Alan and Alex found a long time ago with Plone. You have to make something before people will contribute:

But when you have an app that doesn’t do anything yet, nobody finds it itchy. They’re not using it. So you don’t get volunteers.

This post doesn't blame Python, rather the management philosophy:

There was no objective basis for decision-making.

It's that simple, really. Without an objective basis, there was no way to argue from anything except opinion, with nobody's opinion being more important than anyone else's.

But that post was by Philip J Eby, a die hard Python fan. However I do believe if Python was the problem, he'd say so. So then I started worrying about Python being the problem. This site seems to think it be:

Unfortunately, I'm am still left in the dark to speculate. Can large scale projects be successful implemented around a dynamic programming language? The burden of proof was out there several years ago, now more than ever, the development community needs some incontrovertible evidence that dynamic programming languages can scale.

I don't buy that. Python is a strong language. There are a ton of projects out there that fail that are using closed source tools. The list of projects using Python is long (just realised YouTube is written in Python), off the top of my head I think of the stuff Enfold and Blue Fountain have done, some of which completely kick ass.

No off to find some proof to counter this argument, but first I have to get some work done.

I've got a bug in my Rails application, well actually a few of them. It allows you to browse and administer users in an LDAP tree. Yes, the first time I was confronted with this, I asked "hasn't this been done already?" The answer was: "not well". So it was written as a Rails app. A while back I had to trip through this application and it was hell.

We put this application onto another site at Blue Fountain and sure enough it failed. It's not doing the job. So on my commute I rationalised why I'm going to throw that Rails away and start again. Here's my list:

• Was the developer so damn clever doing X, Y and Z that he forgot some mortal might have to maintain it? Does the person now maintaining not have super ninja Widget Fromboid foo? Was there any code documentation or comments? Was this just a developer getting high off being so damn clever?
• Is it reusable? Are you going to get rich off maintaining this thing? Is it really just going to start dragging you down time and time again? Is this write once and maintain until you can run screaming away from the compamy?
• Does it have automated tests of any kind? Unit tests, integration tests, browser tests anything, this is the 21st century right?
• How many bugs has it had, has it got some now, are those bugs hard to solve than the Middle East peace process? You can tell the death march of a project when banging noises are made on developer's desks and the walk round with a dull glaze and red mark on their forehead.
• Really how hard would this to be replace? Remember everything is a) easier the second time and b) way harder because of all the undocumented wierd edge cases. Every developer will tell you a) and every project manager will roll his eyes and say b). The answer is really c). I'm not sure where that is but you'll find out.

In this case:

• I've tried and when you start messing in the Active Record internals you know something is wrong.
• Nope, we've seen this already. Further the UI is tied into the back end nice and tight. This is actually a useful library.
• Yes it does, but they don't work.
• Lots and still hasn't got created confidence, having problems with scaling and leaving file descriptors open all over the place.
• Not hard, honest no edge cases. All the edge cases work around LDAP or Active Record.

So tommorrow its moving to Django. I reserve the right to amend this blog post to change all the above to justify this change.

This one confused me for a minute when a colleague asked me:

"some string"""

My first thought was that should be a syntax error, but it's not. It is infact two strings, the first contains the text "some string", the second is empty "". If you output that you'll get:

'some string'

It's the same output as:

"some ""string"

It's actually a pattern we've all used for a long time, if a string is long and splits over a line you can append multiple strings together. Useful when you want to break the text up, but not embed spaces and new lines, like a triple quoted string would. For example:

    "This is a long message "\
    " that goes over two lines"

So I've got this neat Django app that pulls together photo's linked in with locations on a google map. I want to scroll through those photos using a nice little Ajax or Flash app that does all those nice things like cross fading and browsing. There's loads out there, but they've all got server requirements like php. I haven't found anything that takes a xhtml page, parses the DOM and produces the photo viewer from that.

Had to laugh when saw these at our local supermarket. For those of you without kids, these are diaper liners. They go on the diaper, so when the kid has a poo you can flush this down the toilet. Leaving the diaper ready to be cleaned (or back to the diaper service). I've seen some code that needs flushing down the toilet, and there's probably some PHP that needs flushing too I'm sure...

Almost got into an interesting argument the other day on irc, but managed to get back to work and avoid the distraction. Somebody made the comment that they weren't too keen on Python because they'd learnt most of it in a day and there wasn't much more to learn. This is, of course, not true. But it is true that if you are familiar with other languages, you can become productive in Python in about a day. You just have to hop over to the Ruby irc channel to see the opposite.

dict["key"] |= ["list item a",]

I saw CSpace on digg touted as the "a platform for secure, decentralized, user-to-user communication over the internet". And guess what, it's written in Python.