I have just uploaded a new version of the
haxr package (for writing XML-RPC clients and servers), having become the new maintainer. Here is how it happened.
In a previous post I announced the release of BlogLiterately 0.4, a tool for authoring and uploading blog posts. I mentioned that the image upload feature did not yet work, due to inexplicably closed HTTP connections. After a lot of digging, installing wireshark, and some excellent advice and answers from StackOverflow, I present to you the following basically-technically-accurate transcript of what was actually going on:
BlogLiterately: Hi there!!
Server: Hello. How may I help you?
BL: I would like to upload some data plz kthx!
S: OK, go ahead.
BL (thinking to self): Oh sh**, I haven’t actually computed the data I want to send! Better get started. Hrmrmrmrmmmmm….
…four seconds later…
BL: ..i thunk i can i thunk i can i thunk i can…
S: You are wasting my time. I have better things to do. Goodbye.
BL: …i thunk i can i thunk i can… done!
BL: OK, I’m ready now! data data data data data data data data data …
S: AUUGGH SHUT UP GO AWAY
So the surface problem seemed to be too much laziness, and indeed, forcing evaluation of the data prior to opening the HTTP connection did make it work. However, the deeper problem is that base64-encoding a 28k image should not take four seconds! Indeed, it turned out that
haxr was using
dataenc to do base64-encoding, which suffered from quadratic runtime due to left-nested list appends. After switching to
base64-bytestring, the encoding now takes basically no time at all.
The other change I made to
haxr was in the type of the
ValueBase64 constructor of the
Value type. It used to take a
String, but this is silly, since the argument is supposed to represent binary data (which will then be base64-encoded), not text. In the original author’s defense,
haxr was written before
ByteString existed! But now that we have
ByteString, I made it the argument to
ValueBase64. This means fewer conversions (in order to provide a
String containing binary data, one byte per character, you basically have to read the file as a
ByteString in the first place anyway and then
unpack it), and also has the nice side benefit of being able to add a new
XmlRpcType instance for
ByteString, so that
ByteStrings can be passed directory to
remote, resulting in a base64 parameter. (Previously, to distinguish a base64 parameter the only way was to explicitly wrap it in a
After a bit of discussion about these issues with Gracjan Polak,
haxr‘s previous maintainer, he suggested that I take it over since he wasn’t really using it anymore. I’m grateful to Bjorn Bringert (
haxr‘s original author) and Gracjan for their work on
haxr over the years.
And yes, this does mean that
BlogLiterately now supports image uploads—and also a few other new features I’ve added in the meantime. A new release will follow shortly!