It’s time to start writing a user manual for diagrams. Haddock documentation is great when you have only, say, forgotten the type of the frobnitz function. However, it is woefully inadequate when you are just trying to figure out how to wibble your fromps (it turns out, of course, that frobnitz is quite general and can be used for, among other things, wibbling fromps; but how would you know to look in the Diagrams.Frobnostication module in the first place?).
So I’m looking for tools I can use to help write and publish the manual. For something short like a tutorial, pandoc works like a charm, but the user manual is going to be a much larger and more complex beast. Here are my requirements. I want to:
- write in Markdown (or some similar sort of markup language)
- have the ability to generate not just a single HTML file, but multiple ones with appropriate next/previous links, a table of contents, and so on
- include Haskell code and generate a syntax-highlighted version in the output; the source (or something easily generated from the source) should also be an actual, valid literate Haskell file that people can load in
ghci - have good support for internal hyperlinks, so different parts of the manual can easily link to one another.
And some "nice-to-have" features:
- I’d like easy hooks into the build process, so I can (for example) replace certain snippets of Haskell code with links to the images generated by the code.
- I’d like to be able to incorporate
doc-review, or something like it, so people can comment on each paragraph in the manual. - I want to mark some regions as "collapsible" so they will initially be displayed as just a "placeholder" of some sort which can be expanded and collapsed by clicking. These could include things like extra technical information which many readers will not care about, extra explanation for users unfamiliar with Haskell, and so on.
So — any ideas? Perhaps I really want the system used for the Yesod wiki, but I’m assuming it would not be cheap. The most viable thing I have considered so far is using pandoc to generate docbook, and from there generating chunked HTML. This is nice in some ways but not ideal: pandoc actually can’t do syntax highlighting when writing anything other than HTML; I don’t know how I would include collapsible sections; and you can only use two heading levels if you want valid literate Haskell.
It’s quite likely that there’s no single thing that will do exactly what I want, and I’m quite willing to do some “glue” work to put some pieces together. But I’d like to avoid as much wheel-reinvention as possible. Any ideas?
blog :: Brent -> [String] 
Have you considered emacs org-mode? It has a very simple markup language and many of the features you describe (export to many formats, html, LaTeX, and others, internal and external links, collapsible headings).
With org-babel, you can also include code blocks (in many languages, haskell among them) that have syntax highlighting. You can extract the code blocks to different files, or even include the results of executing a code block into the document.
I have used it to write documentation and found it very useful for the task.
Hmm, interesting idea! I use org-mode all the time for organization, but I hadn’t thought of using it for this purpose. I will look into it.
In our experience around Opa, asciidoc is acceptable, but far from perfect.
You might look into Jekyll.. It’s more geared towards websites, but I believe it can do any of the things you say (except maybe “have good support for internal hyperlinks, so different parts of the manual can easily link to one another”, since I’m not sure what you mean by that). It’s worth a look at least.
There are a lot of other similar tools, including some that are written in Haskell.
Have you looked at Sphinx? http://sphinx.pocoo.org/
It is based on ReStructuredText, which is a format supported by pandoc, and is ‘Markdown-like’ i.e. mainly plain text with conventions for formatting.
Sphinx adds various extensions to ReST to help with source code, and various extension points to add your own things. Most of the Python world are standardising on using Sphinx for docs.
It supports syntax highlighting for Haskell via Pygments – http://pygments.org/demo/15582/
It has excellent support for internal hyperlinks.
Converting to a literate Haskell file would be harder, but should be possible using a bit of glue.
I found this directive that might help with literate Haskell:
http://sphinx.pocoo.org/markup/code.html#directive-literalinclude
You would then have to put your Haskell in a separate file, which might be what you want.
You should be able to do everything you want using pandoc + a little scripting uisng the pandoc API:
http://johnmacfarlane.net/pandoc/scripting.html
Note that pandoc already allows markdown+lhs as an input format.
Yes, I’ve used the pandoc API before and it’s fantastic. I also use the markdown+lhs input format all the time. But as I noted above, pandoc currently doesn’t do syntax highlighting when outputting docbook, and the inability to have more than two heading levels when using markdown+lhs is a problem for me as well. Do you have any ideas for getting around these issues? I’d also be willing to contribute to pandoc to get it to a place where it will work well for me, if the way forward is relatively clear.
If you’re writing a bit of Haskell glue, you can easily split the Pandoc document into several documents by sections — just split the Block list on Header 1 elements. You can then write a separate HTML document for each section. (You’d probably want to add some Prev/Next/Up links, and you’d also have to adjust any existing internal links to point to the proper files, but this shouldn’t be too hard using the API.)
More than two heading levels in LHS: I’m less sure what to do here. You can’t start headings with ‘#’, since that has a special meaning in column 1 in literate Haskell files, and setext headers only give you two levels. But here’s a workaround: put a space before your ‘###’. This will be parsed as a Para, not a Header 3, but you can then go through with generics and transform every Para (Str “#” : Str “#” : Str “#” : xs) to a Header 3 xs.
For Haskell code sections that link to pictures, you can use delimited code blocks with a special class (say, picturelink). Then have your script find code blocks with this attribute, run the code, create the image, and insert a link.
You can use pandoc as a library and implement the desired functionality yourself. (Hyperlinking is a bit problematic, though; last time I tried this, pandoc doesn’t give you access to the unique identifiers.) I converted my operational monad tutorial from markdown to LaTeX with this method.
Whether this is a good use of your time, I don’t know, but that’s probably how I would do it. Also, I would be interested in contributing to / reusing such a project.
A piece of wisdom concerning the markup: in contrast to markdown, restructured text is extensible. It’s extremely likely that you run into a situation where you want to include some extended attribute (like “collapsed”) and then you have a problem with markdown. Maybe John McFarlane has some ideas here. I don’t really know whether ReST is really better here, though.
(That seems to be a general problem: If an existing system does not do everything you want, you’re screwed. That’s why I like the pandoc-as-a-library approach.)
Ah, thanks for the wisdom re: extended attributes. I had considered that problem but wasn’t aware that ReST was extensible, which is good to know; I’ll take a look at it. And you’re right about the pandoc-as-a-library approach.
Skribe/Scribe, Skribilo / lout / Nonpareil ? (No personal experience with these though).
I may be an old fogey but I still don’t understand what’s wrong with texinfo. It has simple semantic markup, it’s easy to edit with a text editor, it can generate hypertext or real typeset manuals (using TeX) with chapters and sections, and it’s been around for decades so it’s free of trendy crap. I still use it.
Hello Brent,
any update on your quest for a good tool? Sooner or later, I have to write a tutorial for my reactive-banana library, hence my interest.
Some time ago, I have even come up with a layout that features a wide margin for annotations.
Since it hasn’t been mentioned here: you could also look at whether you can make dexy do what you want.
Note that Pandoc (now released v1.9, major enhancements) has supported a pretty complete subset of reST as both input and output, so it’s not an either-or. A followup post show what you ended up doing here would be great BTW. . .
Pandoc 1.9 looks awesome! I do love pandoc and use it for quite a lot. However, pandoc still does not support the “user-extensible” portions of reST, such as user-defined roles and containers. These features are critical for me, since I want to be able to invent new “semantic markup” on a per-document basis and then write custom processors for it.
A followup post explaining what I ended up doing is a good idea. For now, the short answer is that I use standard python docutils tools to parse reST to XML, and then hxt for transforming the XML into various other formats.