BlogLiterately 0.5 release

I have now released version 0.5 of BlogLiterately. (You can read about the 0.4 release here.) This version does uploading of images! Here is proof:


(My previous post explains the problem and solution with image uploads.)

It also allows you to specify expected outputs in a ghci session (a feature suggested by Dan Burton). This block


now produces

ghci> 7+6

ghci> 9+4

Outputs that match the expected output are shown normally; outputs that don’t match the expected output are shown with the actual output in red and expected in blue. The idea is that this helps you catch errors in your code before uploading the post. (Of course, you don’t have to specify expected outputs if you don’t want to.)

Another new feature is that BlogLiterately will prompt you for your password if you don’t specify it on the command line (another feature requested by Dan).

Finally, one of the coolest new features (in my opinion) is that the internals are now exposed as a library, and in particular you can easily add your own custom transformation passes (of type Pandoc -> IO Pandoc) to the existing ones. So, for example, you could do something particular with your own specially tagged blocks (like [ghci] blocks), or wrap images in some fancy HTML to produce frames and captions, or automatically turn certain things into links, or whatever you can dream up. If you come up with any transformations you think might be more generally useful, please send them to me so I can include them in future releases for others to use.

Happy blogging!

Posted in haskell, writing | Tagged , , , , , | 3 Comments

New haxr release

I have just uploaded a new version of the haxr package (for writing XML-RPC clients and servers), having become the new maintainer. Here is how it happened.

In a previous post I announced the release of BlogLiterately 0.4, a tool for authoring and uploading blog posts. I mentioned that the image upload feature did not yet work, due to inexplicably closed HTTP connections. After a lot of digging, installing wireshark, and some excellent advice and answers from StackOverflow, I present to you the following basically-technically-accurate transcript of what was actually going on:

BlogLiterately: Hi there!!
Server: Hello. How may I help you?
BL: I would like to upload some data plz kthx!
S: OK, go ahead.
BL (thinking to self): Oh sh**, I haven’t actually computed the data I want to send! Better get started. Hrmrmrmrmmmmm….

…four seconds later…

BL: ..i thunk i can i thunk i can i thunk i can…
S: You are wasting my time. I have better things to do. Goodbye.
BL: …i thunk i can i thunk i can… done!
BL: OK, I’m ready now! data data data data data data data data data …

So the surface problem seemed to be too much laziness, and indeed, forcing evaluation of the data prior to opening the HTTP connection did make it work. However, the deeper problem is that base64-encoding a 28k image should not take four seconds! Indeed, it turned out that haxr was using dataenc to do base64-encoding, which suffered from quadratic runtime due to left-nested list appends. After switching to base64-bytestring, the encoding now takes basically no time at all.

The other change I made to haxr was in the type of the ValueBase64 constructor of the Value type. It used to take a String, but this is silly, since the argument is supposed to represent binary data (which will then be base64-encoded), not text. In the original author’s defense, haxr was written before ByteString existed! But now that we have ByteString, I made it the argument to ValueBase64. This means fewer conversions (in order to provide a String containing binary data, one byte per character, you basically have to read the file as a ByteString in the first place anyway and then unpack it), and also has the nice side benefit of being able to add a new XmlRpcType instance for ByteString, so that ByteStrings can be passed directory to remote, resulting in a base64 parameter. (Previously, to distinguish a base64 parameter the only way was to explicitly wrap it in a ValueBase64 constructor.)

After a bit of discussion about these issues with Gracjan Polak, haxr‘s previous maintainer, he suggested that I take it over since he wasn’t really using it anymore. I’m grateful to Bjorn Bringert (haxr‘s original author) and Gracjan for their work on haxr over the years.

And yes, this does mean that BlogLiterately now supports image uploads—and also a few other new features I’ve added in the meantime. A new release will follow shortly!

Posted in haskell | Tagged , , , , | 2 Comments

BlogLiterately 0.4 release

I have just released version 0.4 of BlogLiterately, a tool for authoring and uploading blog posts (especially Haskelly ones). Rob Greayer authored the first few versions of this useful tool, for which I’m very thankful. However, he doesn’t have the time or interest to maintain it anymore, and I had a few ideas for extending and improving it, so I offered to take over as the maintainer.

The full (dog-fooded) documentation can be found here. This blog post is just to announce the release and show off a few capabilities.

Posts are written in Markdown format (as recognized by pandoc), and BlogLiterately handles converting them to HTML and uploading them to any blog that supports the MetaWeblog API (such as WordPress). Haskell code can be syntax highlighted using hscolour (and any code can be syntax highlighted using highlighting-kate):

> -- An awesome Haskell function
> fib :: Integer -> Integer
> fib 0 = 0
> fib 1 = 1
> fib n = fib (n-1) + fib (n-2)

Special support for WordPress LaTeX is built in: \pi^2 / 6. ghci sessions can be automatically generated from a list of inputs:

ghci> fib 20  

ghci> [1..10]  

The one major planned feature that is still missing is uploading of embedded images. Sadly, this feature ran into a major roadblock in the form of inexplicably closed HTTP connections (can you help answer my StackOverflow question?). Ultimately my goal is to have completely automated support for writing blog posts with inline diagrams code.


Posted in haskell, writing | Tagged , , , | 9 Comments

Unsubscribing from Wolfram emails (rant)

I have been getting a bunch of emails from Wolfram and decided I wanted to unsubscribe.  No problem, right?  I’ll just scroll to the bottom of the email and click the…

To view our privacy policy, click here:

To update your email, click here:

…the, uh… hmm. Well, maybe unsubscribing is included in “updating one’s email”? I click on that.

Nope. OK, let’s try “privacy policy”.

Aha, now we’re getting somewhere! I click on “unsubscribe”.

Posted in humor | Tagged , , , | 3 Comments

Announcing diagrams 0.5

I am pleased to announce the release of version 0.5 of diagrams, a full-featured framework and embedded domain-specific language for declarative drawing. Check out the gallery for examples of what it can do!

Naive fibonacci call tree

Highlights of this release include:

  • A new diagrams-contrib package of user-contributed modules, which so far contains code for tree drawing, Apollonian gaskets, planar tilings, "wrapped" layout, and turtle graphics.
  • Experimental support for animation, built on top of the new active library.
  • Numerous small additions and improvements, including more general rounded rectangle shapes and better text support.
  • Much better performance in some common situations, such as laying out a very long list of diagrams using ‘cat’ and related combinators.
  • Added support for GHC 7.4.

See the release notes for complete details, and the diagrams wiki for help migrating code from 0.4 to 0.5 (changes should be minimal).

Try it out

For the truly impatient:

cabal install gtk2hs-buildtools
cabal install diagrams

Diagrams is supported under GHC 6.12, 7.0, 7.2, and 7.4. However, getting cairo to build can be tricky on some platforms; see the diagrams wiki for more information and workarounds regarding specific platforms. (A new native SVG backend is in the works, targeted for the 0.6 release.)

To get started with diagrams, read the quick tutorial, which will introduce you to the fundamentals of the framework.

For those who are even less impatient but want to really dig in and use the power features, read the user manual.

Get involved

Subscribe to the project mailing list, and/or come hang out in the #diagrams IRC channel on for help and discussion. Make some diagrams. Fix some bugs. Submit your cool examples for inclusion in the gallery or your cool code for inclusion in the diagrams-contrib package!

Happy diagramming!

Brought to you by the diagrams team:

  • Peter Hall
  • Ian Ross
  • Michael Sloan
  • Ryan Yates
  • Brent Yorgey

with contributions from:

  • Sam Griffin
  • Claude Heiland-Allen
  • John Lato
  • Vilhelm Sjöberg
  • Luite Stegeman
  • Kanchalai Suveepattananont
  • Scott Walck
Posted in diagrams, haskell, projects | Tagged , , , , | Leave a comment

Parsing context-sensitive languages with Applicative

Many parser combinator libraries in Haskell (such as parsec) have both a Monad interface as well as an Applicative interface. (Actually, to be really useful, you also need MonadPlus along with Monad, or Alternative along with Applicative, in order to encode choice; from now on when I talk about Monad and Applicative note that I really have MonadPlus or Alternative in mind as well.) The Applicative interface is often nicer, but it is less powerful than the Monad interface: in particular, using Applicative you can only parse context-free languages, whereas Monad lets you parse arbitrary context-sensitive languages. Intuitively, this is because the structure of Applicative computations cannot depend on intermediate results, whereas Monad computations allow you to choose which computation to run next based on intermediate results.

This is a good bit of intuition, with only one minor caveat: it isn’t true! I believe it was two years ago, during the second Hac phi, when I first learned from Edward Kmett how Applicative (by which I mean, of course, Alternative) can be used to parse arbitrary context-sensitive languages. The question just came up again in the #haskell IRC channel, and I figured it would be useful to have this all written down somewhere. In particular, Reid Barton gave a nice sketch which I decided to turn into some working code.

Here’s the key insight: normally, grammars are defined as finite objects: a finite set of terminals, a finite set of nonterminals, and a finite set of productions. However, Haskell’s general recursion means that we can write down a "grammar" with an infinite set of production rules. This is what lets us get away with parsing context-sensitive languages with Applicative: we just make a different production rule for every possible input!

First, some imports. Notice that I do not import Control.Monad.

> import Text.Parsec
> import Text.Parsec.String
> import Control.Arrow ((&&&))
> import Control.Applicative hiding ((<|>))
> import Data.List (group)

The usual guard function is for MonadPlus, but we can make something equivalent for Alternative.

> guard' :: Alternative f => Bool -> f ()
> guard' True  = pure ()
> guard' False = empty

And now for the meat of the example. parseArbitrary takes an arbitrary predicate on Strings built from lowercase letters and turns it into a parser. The created parser will accept Strings for which the predicate evaluates to True (returning ()) and fail for any other string.

> parseArbitrary :: (String -> Bool) -> Parser ()
> parseArbitrary p =

If we encounter eof, we simply ensure that the predicate holds of the empty string.

>       (eof <* guard' (p [])) 

Otherwise, we choose between 26 alternatives based on the next character in the input. If the character c is encountered, we make a recursive call to parseArbitrary (p . (c:)). The remainder of the input must satisfy (p . (c:)), that is, it must consist of some String s such that (c:s) satisfies the predicate p.

>   <|> foldr (<|>) parserZero 
>         (map (\c -> char c *> 
>                     parseArbitrary (p . (c:))
>              ) 
>              ['a'..'z']
>         )

For any given predicate p, you can think of parseArbitrary p as an infinite tree with a 26-way branch at each node. Each node "remembers" the path taken to reach it from the root of the tree, in the form of prepend functions composed with the original predicate. We have constructed an infinite grammar: each node in the tree corresponds to a production, one for every possible input prefix.

Let’s try it out. Here’s a function which only accepts Strings of the form "aaabbbccc", with an equal number of a’s, b’s, and c’s. This is a well-known example of a language which is not context-free (easily shown using the pumping lemma for context-free languages).

> f :: String -> Bool
> f s 
>   | [('a',na), ('b',nb), ('c',nc)] 
>     <- map (head &&& length). group $ s
>     = na == nb && nb == nc
>   | otherwise = False

Now we make f into a parser and test it on some example inputs:

> p = parseArbitrary f
> main = do
>   parseTest p "aaa"
>   parseTest p "aaabbbcc"
>   parseTest p "aaaabcccc"
>   parseTest p "aaaaabbbbbccccc"

The last test succeeds by returning (). For the first three, we get error messages like this:

parse error at (line 1, column 4):
unexpected end of input
expecting "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y" or "z"

Obviously, these are not very helpful. But what were you expecting? After all, this is one of those things that is interesting in theory, but in practice amounts to an awful hack that no one would ever want to use in real code.

In the end, it’s still true that Applicative can only parse context-free languages as long as we restrict ourselves to finite grammars—which any sensible programmer would do anyway.

[ETA: it looks like using infinite grammars is not as impractical as I thought—see the comments below!]

Posted in haskell | Tagged , , , , , , | 11 Comments

Typeclassopedia v2: moving my efforts to the Haskell wiki

I said back in April that I was going to be working on a second edition of the Typeclassopedia. Since then I have worked on it some, but mostly I’ve dithered trying to work out a toolchain for publishing it in multiple formats including PDF and (commentable) HTML. But it’s become increasingly clear to me that this is really just getting in the way of the important work of actually updating the content, and I ought to just choose a simple solution, imperfect though it may be, and get on with it.

In the meantime, it turns out that Henk Krauss (geheimdienst) has been doing the tedious work of porting the Typeclassopedia to the Haskell wiki (thanks!). It looks much nicer than I thought possible for a wiki page, allaying one particular misgiving I had about publishing in wiki format. And this format obviously makes it extremely easy for others to contribute.

The path forward is clear: I hereby declare that the Haskell wiki version of the Typeclassopedia is now the official version, and from now on I will be focusing my efforts there. You can expect a bunch of updates, improvements, and expansions over the next few months. If you have any improvements to suggest, you can make them directly, or leave a comment on the talk page.

Posted in haskell, projects, writing | 5 Comments