Andy McKay's blog

A while back I posted on Heathrow terminal 5's project management philosophy. Unfortunately it turns out that the launch of Heathrow terminal 5 has gone tit's up. The stories are numerous, but for those who don't know the baggage system went wrong. Something to do with computers. Looks like the cunning plan broke down somewhere:

The system was designed by an integrated team from BAA, BA and Vanderlande Industries of the Netherlands, and will handle both intra-terminal and inter-terminal luggage and will actually process 70,000 bags a day.

I guess they do that by piling them all up in corner and not actually putting them on planes:

...28,000 bags had been placed in temporary storage by the end of the weekend

One interesting up shot is that via the BBC I found that David Miliband has a blog. He's the UK's Foreign Secretary and he has a blog. That's cool. The reason is that:

One foreign minister ... arrived merely to transit, but his bags are nowhere to be seen and it was whispered that it might take weeks. He asked me to pass on a message to BA/BAA: for goodness sake get your act together

...there's even foreign office photos on Flickr.

This release adds in:

• Increased performance, but not recompiling on every request and utf-8 templates (thanks Mr Topf)
• More detailed errors through SimpleTAL patch

Download: simpletemplate.0.5.zip

Adobe Air is nice, but I have to say, moving onto the next steps has stalled me because I don't have a set of libraries to play with. Consider these two toy quick apps:

• SVN proxy that monitors all your SVN traffic, allowing you to log exactly what you logged when (useful with multiple SVN servers) and show statistics, including automatic timesheet hint generation.
• IMAP statistics tool that tells when and where your email traffic is coming from and going to.

Both of these would be simple to do in Python and matter of calling out to say HTTPServer, urllib, imaplib for the various protocols to setup proxies or query IMAP servers. However how do I do that. I've got access to javascript and Flex. I'm stuck with: clunky system calls to another language, passing things off to the cloud (not sure thats possible or desire-able) or looking for SWF libraries.

There is a C compatibility layer which gives the prospect of being able to access Python, but still no news.

There were a few red herrings, this article by Bruce Eckel covers calling Python for XML-RPC. Another red herring was this project in PyPI, thankfully I found a download URL that got me around the site that it's hosted on that requires registration and makes no sense. In the end it turns it out its Windows only and comes as an Air APP and a py2exe app. A quick rummage through the source of the .exe reveals it contains links to XMLRPCServer, so I'm guessing its just an implementation of Bruce's ideas. I must admit I was incensed that sort of thing gets into PyPI, but I know there's no policing of that.

Those examples both ship an Air app and a "frozen" python instance. My wife wouldn't be able to run that, it's not user friendly and fraught with problems (one is running the other isn't etc). So currently I'm stuck, I can't see how I can easily extend Adobe Air to do anything beyond interact with what I get. Surely there must be a way forward and I'm missing the big picture.

Vancouver is finally getting an Apple store. About time. Although I would note I always got good intelligent, friendly service at MacStation. In fact the "make an appointment" attitude at a Mac store is damn annoying, but still going there is the only chance you have to have some problems addressed. Pic and article thanks to TechVibes. Now all Canada needs is the iPhone in time for me to arrive back there.

Another interesting point was on the question: will Canada ever get a Silicon Valley? Every country wants one, be it the UK or Canada. The answer is no and here was an interesting take on Startup North. Most people love there city and don't want to move. With notable exceptions in the US (say Austin, New York, Boston, Portland, SF) most American cities are extremely homogenous *. The population is extremely mobile, large numbers of people move to find jobs and a large number moved to that area.

In Canada people tend to move a lot less, they like where they are. For example, if you spend a long time on say the west coast you will either a) love it and stay or b) not be there very long. That causes small numbers of concentrations in Vancouver, Montreal, Ottawa, Waterloo area. Lots of Canadians like where we live, that inertia prevents another Silicon Valley. And let's face it if you've been there, you might think that's a good thing.

* I know I'm asking for a flood of email proving the opposite. Hey it's an opinion based on my travels and visiting lots of cities with the exact same malls and shops every 3 blocks.

Today i hit a road block on a project. Not the usual ones (clients, technologies, money), but a mental one. I spent 6 hours of my day staring at this project and felt like I came away understanding less. This has happened to me before on projects and I've realised it's hit me now.

How can you tell when this hits? It takes a bit of practice, this is isn't an average tough bug. I hit those and a simple change of scenery normally solve this problem. For me in the past this meant one of the following: a) a good bike ride, b) walk the dog, c) a glass of scotch (or two)... and bingo I'd run back to the computer and carry on.

Neither is this a bad day. I get those, oh boy do I get those at the moment. You can tell this isn't one of those because it's hitting back to back. Interestingly today, I thought I'd broken the back of this particular bug, only to find 8 more underneath and come away from the situation understanding less of one particular area than when I started.

It's not a matter of blaming the application, it could get its fair share of blame. It's too complicated, has spawned an entry on DailyWTF, has no documentation, very small unit test coverage, all the developers left, the project manager left and the client doesn't really know what they want. All par for the course there then. But in the end this is the sort of stuff we have to deal with.

Fortunately a while back all the belief that I was decent developer got beaten out of me, I at least now know what I do well. Tackling this sort of thing truly isn't my forte. That's fine, but I've still got to get over this hurdle and move on. So that means just keep plugging away or calling in help. I'll go for the latter and see if that get's me anywhere, it might or might not. But I think realising I'm now completely spinning wheels and not moving forward is the first challenge.

For the record, anyone who might be reading this blog - it's not your project, relax. And for the record, former colleagues who read this and go what about that project X years ago? Yep I hit it then (my count is this is the 3rd time in 8 years). Alright time for a glass of scotch, perhaps that will loosen up ability to cope with this application.

Just so I don't get forget, the correct syntax for setting a quota in imaplib against cryus is:

size = 100000
imap.setquota(account, "(storage %s)" % size) 

Won't say this often, but thank you Perl doc's for showing me the correct syntax.

At PyCon there was a talk "How to use Grok to Walk like a duck" that makes a great read. It's about how the Zope 3 component framework. A great line in the presentation is:

Ruby people do this

... a reason for why you shouldn't do monkey patching. Presentation at: http://rhodesmill.org/brandon/adapters/.

Yesterday I travelled down to the OSS Watch event in Oxford. The conference had one of the nicest locations I've seen in a while. Even I could find it, cross from the train station and is the first door on the left.

There seemed to be about 65 people there, with about 20 of those being companies providing OSS solutions or speakers. The conference started off with a few talks and then in the afternoon a panel. Thanks to a few hitches on the train, I got there half way through the first talk.

One talk by Imperial College was interesting on their move to Linux. At the end a question focused on the fact that his department of 4 (5 including himself) maintained a Linux distribution for around 7,500 computers, which i thought was neat. An offside was that at the request of the organisers, the speaker tried hard not to mention the name of vendors. Spoke about the "large Linux vendor from Germany" and the "large Linux vendor in the US" (although his screenshots mentioned RHEL).

In the afternoon I sat on a panel (twice) about "Working with the OSC". This was a little odd to me because within Blue Fountain I have no idea what relationship we have the OSC. So I just came out and said so and waffled a bit more about Plone.

The questions were mostly about procurement, it was interesting to hear the same points raised again, things (which might be cyncial in my summation) like:

• Size and scale of open source companies is a perceived barrier. In reality if you wanted to a large support contract with the international consultancy Flibberty, Pooch and Posh they would go out and set up a support infrastructure anyway. Just like smaller companies would.
• Risks are reasonably arbitrary and often weighted on cultural differences, the more different you are from what they are used to, the more those risks are weighted.
• The end outcome of a procurement process isn't as important as being seen to have done the process (and hence your job) properly.

The end result is that the procurement process does not result in the solution is necessarily the best for the procuring organisation. Rather it's about finding an organisation who's structure meets the clients needs. Not how well the job can be delivered the needs. One panelist said that on most large procurements the focus is purely on the company structure, not on the solution.

It made me think of what Paul Everitt said at the Plone Planning Summit: "Enterprise customers buy whatever Deloitte tell them to".

One the way out I chatted to Michael from Alfresco because I was curious about their licensing, something that was brought up at the Plone Planning Summit. Plone provides itself on being free as beer and free as speech. It also has an incredibly open development process. Michael said that there have been issues with the Alfresco license in the past, but those are mostly past issues. The current licensing is a simple dual licensing. The community edition is released before the enterprise version. The community version is GPL, the enterprise one isn't. There isn't support on the community edition. He compared it Fedora and Red Hat's official licenses.

Then it was time to hop on the train back home shame that took 5 hours. Bloody rail in Britain.

The Plone File Uploader is a little experimental product I wrote using Adobe Air. It's designed to all you to just easily drag and drop multiple files into your Plone site. After installing, tell it where you want files to go and then just drag and drop files in, it queues them up and sends them to your server as HTTP PUT's (with all the problems that involves in Plone).

It's in it's beta stages and done more for fun than anything else. It doesn't recurse folders - if it you drag in a folder it just grabs the files from that folder and uploads them. It ignores sub folders, shortcuts etc. If you want a super duper upload tool, go look at Enfold Desktop.

Note: A few people tried it on Windows and Plone 3 including Kaell in irc (thanks) and we got differing responses, including this bug. So it looks like some Plone or Zope combo's might need patching to allow PUT's.

Requires

Adobe Air installed on your computer and access to a Plone site.

Tested on

Plone 2.5, OS X.

Download

uploader.0.2.2.air

plu·zzy

• The warm fuzzy feeling you get on navigating to a web site, only to find unexpectedly that it is in fact built in Plone. The opposite of ploop.

pl·oop

• Normally felt after encountering pluzzy. The slight let down felt, when you realize that the site in question is not in fact Plone. Usually felt when you realise the web site is in fact using Drupal, Typo 3 or something else PHP.

Alright Microsoft has a long way to go to winning the hearts and minds of developers. But on this article talking about IronPython and the ability to access .NET. That in itself is cool, but we've know about that for a long time. But there is also an advert for Visual Studio. Using Halo characters. Now that is pretty cool.

This one had me confused for quite a while, simply I wanted to do a PUT in Adobe Air to my Plone site. Something that you think would be quite easy but somehow all I got were errors. The problem turned out to be I was doing an upload, when it should have been uploadUnencoded which has a slightly different signature.

function uploadFile(file) {
    // an internal function to get our upload url
    var url = getPref("url");
    // an internal function to get the file name for the PUT
    var urlRequest = new air.URLRequest(url + "/" + fileFromPath(file.url));

    var headers = new Array()
    // an internal function to get the Plone security cookie
    headers.push(new air.URLRequestHeader("Cookie", "__ac=" + getPref("cookie")));
    // an internal function to get the file extension
    headers.push(new air.URLRequestHeader("Content-Type", extensionFromFile(file)));    
    
    urlRequest.requestHeaders = headers; 
    urlRequest.method = air.URLRequestMethod.PUT;
    
    file.addEventListener(air.ProgressEvent.PROGRESS, uploadProgress);
    file.addEventListener(air.Event.COMPLETE, uploadComplete);
    file.addEventListener(air.SecurityErrorEvent.SECURITY_ERROR, uploadError);
    file.addEventListener(air.HTTPStatusEvent.HTTP_STATUS, uploadError);	
    file.addEventListener(air.IOErrorEvent.IO_ERROR, uploadError);
    
    file.uploadUnencoded(urlRequest);
} 

If you are dragging multiple files into your Air and you need to grab them, make sure you do the following:

function dropHandler(event) {
    var files = new Array;
    files = event.dataTransfer.getData("application/x-vnd.adobe.air.file-list")
    ...
}

That mime-type is what you need to get a file list and not just the last file.

Supposing I have a model sample, and column foo for the model samples in the DB. In the following example, what does Sample.find(:first).wtf print out?

require 'sampletwo'

class Sample < ActiveRecord::Base
    def wtf
        foo
    end
    
    def foo
        "here 1"
    end
end

def foo    
    "here 2"
end

Too easy, the answer is "here 1". Ok how about something a bit harder. What does Sample.find(:first).wtf print out now?

class Sample < ActiveRecord::Base
    def wtf
        foo
    end
end

Is it the column on the database? No. It's the method foo in the library sampletwo.rb, which you won't know. But is now imported in environment.rb. The order for lookup (if I've got this right is):

• Does the method exist on the object (and presumably in the inheritance tree, but not sure)
• Does the method exist somewhere else in the namespace, eg in the module
• Does anything else import that value into the namespace, eg any require you might have in your environment or your current model
• Lookup the database column

I've never like the fact that Ruby's require imports everything into that namespace the module wants to import, you don't have any choice. It's rather like this in Python:

from wx import *

The end result is that when you collaborate on things or end with big honking 2,000 line models you get a huge amount of conflict and sitting there trying to figure out where something comes from. You normally keep your database manager open in another window so you can query it and see what's there.

Meanwhile in Django land, there's an explicit model, explicit import statements (* imports are possible but discouraged in Python) and explicit use of self.

This blog post started out as talking about Adobe Air frustrations, but as I ran through the post I actually started fix my issues so within 3 days expect this post to be completely inaccurate.

1. Some simple HTML, CSS and DOM things just seemed to fail. They are the sort of things that you sit and look at and think continually you've done something wrong. As it turns out adding event listeners to DOM nodes is slightly different, but I still couldn't get background images assigned in CSS.
2. Air comes with a nice URLRequest object to talk to external sites and give progress over the result via an asynchronous event. However it can only do GET and POST to external sites. So no PUTs to a Plone site (for example). Ouch. This might kill off my little project to provide a simple file upload tool for Plone.
3. I've just now found a way to handle multiple file drag and drops, in theory.
4. Almost everything is an asynchronous event, which is great for flexibility. Writing everything in JS without any templating and attaching events to everything is starting to get tortuous. Perhaps it's just growing pains and learning a new style, but at this point a big complicated project looks like it might get insanely complex.

On the plus side there were a few nice things in there that rocked. It comes with it's own SQL database either in memory on the filesystem. User preferences for the application can be stored away in its own encrypted space with no configuration, just:

function setPref(key, value) {
  var bytes = new air.ByteArray();
  bytes.writeUTFBytes(value);
  air.EncryptedLocalStore.setItem(key, bytes);
}
...
setPref("plone-cookie", cookie);

...and simple preferences data is being stored. Hopefully first Air app should be done next few days, depending how much real life interferes again.