The algebra of species: primitives

[This is the fifth in a series of posts about combinatorial species. Previous posts: And now, back to your regularly scheduled combinatorial species; Decomposing data structures; Combinatorial species definition, Species definition clarification and exercises.]

Recall that a species is a functor from , the category of finite sets and bijections, to ,¹ the category of finite sets and total functions. (Equivalently, species are endofunctors on , but in this post I’m going to want to think about them as the former.) That is, a species is a mapping sending every set of labels to a set of structures , which also lifts relabelings to functions in a way that respects the compositional structure of bijections.

However, as I hinted in a previous post, it’s inconvenient to work directly with this definition in practice. Instead, we use an algebraic theory that lets us compositionally build up certain species from a collection of primitive species and species operations. (It’s important to note that it does not allow us to build all species, but it does allow us to build many of the ones we care about.)

In this post we’ll begin by examining a few natural species to take as primitive.

• The zero or empty species, denoted , is the unique species with no structures whatsoever; that is,

  

  and

  .

  Of course, will turn out to be the identity element for species sum (which I’ll define in my next post, though it’s not hard to figure out what it should mean).

• The unit species, denoted , is defined by

  

  That is, there is a unique -structure indexed by the empty set of labels, and no -structures with any positive number of labels. lifts bijections in the obvious way, sending every bijection to the appropriate identity function.

  Some people initially find this definition surprising, expecting something like for all instead. That is indeed a valid species, and we will meet it below; but as I hope you will come to see, it doesn’t deserve the name .

  Of course we should also verify that this definition satisfies the requisite functoriality properties, which is not difficult.

  More abstractly, for those who know some category theory, it’s worth mentioning that can be defined as , that is, the covariant hom-functor sending each finite set to the (finite) set of bijections . (This is why I wanted to think of species as functors . I learned this fact from Yeh (1986).) There is, of course, a unique bijection and no bijections for nonempty , thus giving rise to the definition above.

  As you might expect, will be the identity element for species product. Like itself, species product isn’t defined quite as most people would initially guess. If you haven’t seen it before, you might like to try working out how product can be defined in order to make an identity element.

• The singleton species, denoted , is defined by

  

  with lifting of bijections defined in the evident manner. That is, there is a single -structure on a label set of size (which we identify with the label itself, though we could have also defined when ), and no -structures indexed by any other number of labels.

  As with , we may equivalently define as a hom-functor, namely .

  It’s worth noting again that although and do “case analysis” on the label set , they actually only depend on the size of ; indeed, as we noted previously, by functoriality this is all they can do.

• The species of bags², denoted , is defined by

  ,

  that is, there is a single -structure on any set of labels , which we usually identify with the set of labels itself (although we could equivalently define ). The idea is that an -structure consists solely of a collection of labels, with no imposed ordering whatsoever.

  If you want to abuse types slightly, one can define as a hom-functor too, namely . (Yeh (1986) actually has , but that’s wrong.)

As a summary, here’s a graphic showing -, -, -, and -structures arranged by size (i.e., the size of the underlying set of labels ): a dot indicates a single structure, and the size of the label set increases as you move to the right.

Just as a teaser, it turns out that and are identity elements for certain binary operations on species as well, though you’ll have to wait to find out which ones!

Next up, addition!

References

Yeh, Yeong-Nan. 1986. “The calculus of virtual species and K-species.” In Combinatoire énumérative, ed. Gilbert Labelle and Pierre Leroux, 1234:351–369. Springer Berlin Heidelberg. http://dx.doi.org/10.1007/BFb0072525.

---

1. Last time I called this category , but is more concise and matches the species literarure.↩

2. The species literature calls this the species of sets, but that’s misleading to computer scientists, who expect the word “set” to imply that elements cannot be repeated.↩

Diagrams 0.6

I am pleased to announce the release of version 0.6 of diagrams, a full-featured framework and embedded domain-specific language for declarative drawing. Check out the gallery for examples of what it can do!

Highlights of this release include:

• Diagrams now comes with a native-Haskell SVG backend by default. If you were holding off on trying diagrams because you couldn’t install cairo, you no longer have an excuse!

• Proper support for subdiagrams: previous versions of diagrams-core had a mechanism for associating names with a pair of a location and an envelope. Now, names are associated with actual subdiagrams (including their location and envelope, along with all the other information stored by a diagram). This enables cool techniques like constructing a diagram in order to position its subelements and then taking it apart again, or constructing animations via keyframing.

• Traces: in addition to an envelope, each diagram now stores a “trace”, which is like an embedded raytracer: given any ray (represented by a base point and a vector), the trace computes the closest point of intersection with the diagram along the ray. This is useful for determining points on the boundary of a diagram, e.g. when drawing arrows between diagrams.

• The core data structure underlying diagrams has been completely refactored and split out into its own separate package, dual-tree.

• Support for GHC 7.6.

• Many more new features, bug fixes, and improvements! See the release notes for complete details, and the diagrams wiki for help migrating from 0.5 to 0.6.

Try it out

For the truly impatient:

cabal install diagrams

Diagrams is supported under GHC 7.0 through 7.6, with the exception that the cairo and gtk backends do not build under GHC 7.0 (but the SVG backend does), and the gtk backend does not build under GHC 7.6.

To get started with diagrams, read the quick tutorial, which will introduce you to the fundamentals of the framework.

For those who are less impatient and want to really dig in and use the power features, read the user manual.

Get involved

Subscribe to the project mailing list, and/or come hang out in the #diagrams IRC channel on freenode.org for help and discussion. Make some diagrams. Fix some bugs. Submit your cool examples for inclusion in the gallery or your cool code for inclusion in the diagrams-contrib package!

Happy diagramming!

Brought to you by the diagrams team:

• Michael Sloan
• Ryan Yates
• Brent Yorgey

with contributions from:

• Sam Griffin
• Niklas Haas
• Peter Hall
• Claude Heiland-Allen
• Deepak Jois
• John Lato
• Felipe Lessa
• Chris Mears
• Ian Ross
• Vilhelm Sjöberg
• Jim Snavely
• Luite Stegeman
• Kanchalai Suveepattananont
• Michael Thompson
• Scott Walck

Species definition clarification and exercises

[This is the fourth in a series of posts about combinatorial species. Previous posts: And now, back to your regularly scheduled combinatorial species; Decomposing data structures; Combinatorial species definition.]

In my previous post I neglected to mention something quite crucial, namely, that (at least for now) we are only talking about finite sets of labels and finite sets of structures. That is, we do not consider structures with infinitely many labels, and given a particular finite set of labels, there must be only a finite number of structures with those labels. The category is the category of finite sets and bijections, not of all sets as I said previously.

Of course, in practice we do want to think about infinite sets of structures and labels, especially in relation to a non-strict language like Haskell! But the theory was invented by mathematicians interested in counting things. I do intend to explore ways to extend the theory to encompass infinite structures, but for now we’ll stick to the finite.

Before moving on to talk about the algebraic approach to species, I also want to point out a few simple implications of the formal definition. Instead of spelling them out in detail, I will pose them as exercises: you can either take them on faith, or try working through the exercises to deepen your understanding.

1. Let denote the set ¹, and write as an abbreviation for , i.e. the application of the species to the label set . Show that given we can determine for any with .

(This shows that “size is all that matters”: in some sense species are really indexed not by sets of labels but by size.)

2. [BLL² Exercise 1.1.2] Show that we get an equivalent definition if we take species to be functors from to (the category of finite sets and (total) functions) instead of endofunctors on .

(Apparently one can generalize the notion of species by replacing with other categories, but at present I am not sure of the implications.)

---

1. The species literature uses , but (as every good computer scientist knows) counting ought to begin at zero.↩

2. I will use “BLL” to refer to Bergeron, Labelle, and Leroux, Combinatorial Species and Tree-Like Structures (see the references at the end of the previous post for the full citation).↩

Teaching abstraction

I’m just beginning to prepare for the third incarnation of CIS 194, Introduction to Haskell in the spring. It’s occasioned some general thoughts on teaching abstraction which seemed worth writing down.

Abstractions, of course, are everywhere in CS. By abstraction I mean anything with multiple “levels”, where upper levels hide details of lower levels. An operating system abstracts over the details of hardware; objects abstract over their internal state; functions abstract over their implementation; and so on. Let me just state up front my main thesis:

Students should be explicitly taught to think on multiple levels of abstraction.

Stated like that, perhaps it seems obvious; allow me to elaborate.

On the face of it, when learning about some abstraction, there are two things one can learn. One can learn how the abstraction is implemented (that is, the details of the lower level); and one can also learn how to use the abstraction (that is, how to work with the upper level while ignoring the details of the lower level).

In many cases, students already know one of the levels and learn about the other. For example, once they get around to learning about how compilers work, students already know how to write programs. Or perhaps they learn both levels as part of the same course, but at different times: for example, they might first learn about logic gates, and then later move on to talking about latches and adders.

It’s when teaching both levels at once that one must be extremely careful. Such a situation might arise, for example, when teaching a concept that is completely new or difficult to motivate—since it’s best to start with the concrete and move to the abstract, one might wish to begin with the lower level, but referencing the upper level is necessary to motivate the lower level in the first place. The problem with such a situation is that it is really easy for students to get the levels confused.

In particular, I have in mind the Applicative functor abstraction (and Monad as well). The last time teaching CIS 194, I didn’t do a very good job with this. I gave my students a homework assignment which had them first implement an Applicative interface to parser combinators, and then use it to write some parsers. Most students missed the point, however, and kept on using low-level implementation details when writing their parsers, instead of working in terms of the Applicative interface.

It’s tempting to just say “don’t do that”, i.e. don’t confuse students by teaching multiple levels at once! But I think the situation actually presents a great opportunity. In the “real world”, one seldom has the luxury of thinking on only one level at a time. Being able to explicitly switch between levels, and keep multiple levels straight, is an important real-world skill. I think the real solution is to explicitly teach students to think on multiple levels of abstraction: that is, be explicit about the fact that there are multiple levels, and teach them to consider carefully at each point which level they are thinking on, and why. I plan to do this in the spring and will report back on how it goes!

Combinatorial species definition

Continuing from my previous post, recall that the goal of species is to have a unified theory of containers with labeled¹ locations. So, how do we actually specify such things (leaving aside for the moment the question of how we compute with them)?

We might imagine specifying them by:

• using any arbitrary set to represent some family of labeled structures (e.g. the set of labeled binary tree structures, the set of labeled list structures, …), together with
• a function that takes a structure and computes its set of labels.

On the face of it this seems quite natural (at least, it does to me). However, it works better to instead use a function from sets of labels to the subset of all structures containing precisely those labels.

In my experience teaching people about species, this often seems to be a source of confusion—it seems “backwards”. More generally, when thinking about a set indexed by some other set (in this case, structures indexed by their sets of labels), one might think to model this by a function (which tells us the index), but it actually works better to model it by a function , which takes each “index” to the set of all things indexed by it.² Hopefully as we work through the rest of the definition you’ll get a sense for why it works better this way. For now, I think the best advice is don’t assign computational significance to these functions from labels to structures. Just think of them as a convenient technical device to keep track of shapes indexed by labels.

In any case, the first half of the definition is:

• A species is a mapping from sets of labels to sets of structures.

(I deliberately chose the word mapping instead of function to emphasize, again, that we don’t particularly want to assign it computational significance.) Of course, the fact that a species takes sets “of labels” as input and outputs sets “of structures” doesn’t matter; any sets will do, so we might as well just say that a species maps sets to sets. We write for the species applied to a set of labels , and call the set of “-structures with labels drawn from ”, or simply “-structures on ”, or even (when is clear from context) just “-structures”.

So far, however, this is rather uninteresting, and moreover it fails to adequately capture our intuition for what “structures” are. Intuitively, the labels are incidental, just like the variable names used in lambda terms are incidental: we must use them to be able to distinguish locations, but the precise objects we use as labels really “shouldn’t matter”. That is, given two sets of labels of the same size, we ought to have “the same” family of structures indexed by each. Of course they can’t be literally the same, because they have different labels! But they should be the same “up to relabeling”. We want to rule out the ability to have two same-size sets of labels indexing wildly different sets of structures: a species shouldn’t be able to “look at” the individual labels in order to “decide” what sort of structures to produce, just like a polymorphic type in Haskell can’t “look at” its type argument. The major difference is that species are allowed to “look at” the size of the label set.

Making this intuition precise is the clever part, and is really the pivotal point around which the whole theory revolves. Here’s how we do it. We don’t work with sizes of label sets directly; instead we work with bijections between label sets. (Of course, if there is a bijection between two finite sets then they are necessarily the same size.)

Given two label sets and which are related by the bijection (sometimes referred to as a relabeling), there must be a relationship between and —in particular they must also be in bijection. Here, then, is the second part of the definition:

• Given a bijection , a species must also “lift” to a bijection .

(Note that we’re recycling notation here, using for the action of species on both label sets and bijections.) However, this still isn’t quite enough: we don’t want to be just any bijection between and . It really should be the specific bijection that “applies” to the labels contained within the structures in . For example, it would be weird if the identity relabeling, when lifted through , resulted in some nontrivial reshuffling of the structures in . It would also be strange if didn’t respect composition, that is, if there were some such that , since intuitively “applying then applying ” ought to be the same as “applying ”. So we add these as conditions:

• must map every identity bijection to the identity , and
• must preserve composition of bijections, that is, .

Of course, all of this may look rather familiar if you know some basic category theory. Consider the category whose objects are sets and whose morphisms are bijections. Then all of the above can be summed up by the pithy

A species is an endofunctor on .

Whew! We finally made it to the definition. However, working directly with the definition is not very convenient. In my next post I’ll begin explaining the more usual algebraic approach.

At this point I should mention that species were first introduced by André Joyal in his thesis (1981). Unfortunately it is in French, which I cannot read. Fortunately Bergeron, Labelle, and Leroux (1998) wrote an excellent reference text on the subject of species. Unfortunately it is in French too. Fortunately, Margaret Readdy translated it into English!

References

Bergeron, F., G. Labelle, and P. Leroux. 1998. Combinatorial species and tree-like structures. Trans. Margaret Readdy. Encyclopedia of Mathematics and its Applications. Cambridge: Cambridge University Press.

Joyal, André. 1981. “Une Théorie Combinatoire des Séries Formelles.” Advances in Mathematics 42: 1–82.

---

1. A note on spelling: generally, “labeled” is the American spelling and “labelled” British (though “labelled” is also in common American usage, according to Merriam-Webster). I try to consistently use the American spelling, but will probably slip up occasionally, and you should use whichever spelling makes you happiest.↩

2. I’ve seen this pattern show up multiple times in different category-theoretic contexts, but I don’t feel qualified to comment on it more generally. If you have any pointers to more general discussion of this idea/phenomenon I’d appreciate it.↩

foldr is made of monoids

> import Data.Monoid

In his recent blog post What is foldr made of?, Tom Ellis made the clever observation that foldr is equivalent in power to the combination of map and compose, where

> compose :: [a -> a] -> a -> a
> compose []     = id
> compose (f:fs) = f . compose fs

We can then write

> foldr' :: (a -> b -> b) -> b -> [a] -> b
> foldr' g z xs = compose (map g xs) z

which (as Tom proves) is exactly equivalent to the usual foldr.

This is a really neat idea which I don’t remember ever seeing before. But now that I have, I realized that I could expand on it a bit—compose doesn’t come out of thin air quite as much as it might first appear.

Consider the standard function mconcat (from Data.Monoid), which combines a list of values from some Monoid:

mconcat :: Monoid m => [m] -> m
mconcat []     = mempty
mconcat (m:ms) = m `mappend` mconcat ms

I claim that in the presence of map, compose and mconcat are equivalent. Why is that? First, it’s easy to see that compose can be implemented in terms of mconcat—we just instantiate it to the monoid of endofunctions (where mempty = id and mappend = (.)), with a little bit of syntactic noise due to the need to convert in and out of the Endo newtype:

> compose' :: [a -> a] -> a -> a
> compose' = appEndo . mconcat . map Endo

Proving that compose' is the same as compose is not hard; I leave it to you as an exercise.

Implementing mconcat in terms of compose is a bit more interesting:

> mconcat' :: Monoid m => [m] -> m
> mconcat' = ($mempty) . compose . map mappend

The key idea is that we can turn any value from some monoidal type m into a function m -> m by partially applying mappend; composing these functions then corresponds to combining the original values, and the final value can be recovered by applying the resulting function to mempty.¹ That is,

(where I have used and to represent mappend and mempty, respectively). Written out this way, I hope you can see why the equality holds by thinking about what the composition on the right evaluates to (and remembering the right identity law, ).

So we can also say that foldr = map + mconcat! This gets at the idea that lists are free (or initial) monoids, which intuitively means that of all monoids, they “preserve the most information”—up to the monoid laws, combining lists preserves all the information about the original lists and how you have combined them. This also means that there is a canonical way to “convert” a list into any other Monoid: that is, given a mapping f :: a -> m, there is a canonical way to take a list [a] and turn it into an m, namely, mconcat . map f.

Let’s make the connection to foldr even more explicit. First, let’s swap around the order of arguments and add some parentheses which aren’t strictly necessary:

foldr'' :: (a -> (b -> b)) -> [a] -> (b -> b)

Hmm, b -> b… why does this look so familiar? I claim we can actually pull a similar trick² as with compose/mconcat, and replace b -> b with Monoid m => m to come up with a function equivalent to foldr'' (and hence foldr as well):

fold :: Monoid m => (a -> m) -> [a] -> m

Hmm, so how would this be implemented? Let’s see…

> fold :: Monoid m => (a -> m) -> [a] -> m
> fold f = mconcat . map f

So actually, fold itself (and hence, equivalently, foldr) is the canonical mapping from a list down to any monoid which I mentioned earlier! And here we can see quite directly that fold is indeed equivalent to mconcat + map.

In summary: foldr is equivalent to map plus compose/mappend, because lists are free monoids.

---

1. Incidentally, this amounts to saying that mappend itself is a monoid homomorphism (a structure-preserving map between monoids), and this is where difference lists come from. For more, see my recent paper, Monoids: Theme and Variations.↩

2. I leave this as an exercise too.↩

Using multiple versions of GHC in parallel with GNU stow

Do any of the following apply to you?

• You sometimes hack on GHC.
• You sometimes want to try out an unreleased GHC version.
• You maintain a library and want to make sure it builds with several different versions of GHC.
• You want to try out a newly released version of GHC but without blowing away your existing (working!) GHC install.

As you can see, these are roughly in order from most to least “hard core”, and I expect that the last two are fairly common, even if the first two are not. The common thread, of course, is having multiple versions of GHC installed simultaneously.

The solution

A solution that I’ve found which works quite well (on Linux, and I expect it would work on OSX as well) and deserves to be better known is GNU stow. Here’s how it works: instead of installing things directly in /usr/local (or $HOME/local, or whatever), you instead install in sandboxed locations like /usr/local/stow/ghc-7.6.1. You then tell stow that you want to “install” ghc-7.6.1, and it creates all the necessary symlinks to mirror the directory structure under the ghc-7.6.1 directory into /usr/local. But of course it tracks everything so that you can easily uninstall all these symlinks as well. (And yes, it’s smart about symlinking an entire directory when possible, instead of all the individual files, and later “splitting up” such big-scale links when necessary.)

The pros of this sort of scheme should be clear:

1. Once you’ve “installed” a particular version you just use it as normal—no need to pass the right directories as flags to tools like cabal or ghc-pkg.
2. Switching to a different version of GHC is relatively quick (see below).
3. Installing new versions is pain-free; it’s just as easy to have twenty different versions as it is to have two.

Of course, there are a few downsides as well:

1. Trusting a Perl script to munge your /usr/local. (Though for what it’s worth, I have been using it heavily for about a year now—I currently have six different versions of GHC installed—and have never once encountered a single problem.)
2. You can’t use several versions of GHC at once—there is always exactly one “current” version. So if you want to, e.g., do test builds of a library under several versions of GHC in parallel, you need a different solution.

A step-by-step guide

So here’s how this works in more detail. First, of course, you have to install stow itself; I’ll assume you can get it from your OS package manager or know how to build it from source. What you get is simply an executable called stow.

Now, when installing GHC (whether a binary distribution or from source), give the configure step an extra prefix argument, for example:

./configure --prefix=/usr/local/stow/ghc-7.6.1

Just take the directory where you would usually install GHC, and add stow and then an additional subdirectory for the particular version you are installing. This directory doesn’t have to already exist; the installer will create it if it doesn’t.

(Note that if you already have GHC installed, you’ll have to uninstall it first and then reinstall it with stow—stow won’t work unless it can manage all your GHC installs.)

After the installation process completes, you have an unusable ghc because, of course, .../stow/ghc-7.6.1 isn’t on any relevant paths. All you have to do to finish installing it is

cd /usr/local/stow && stow ghc-7.6.1

This instructs stow to make symlinks into ghc-7.6.1 from /usr/local. After this completes, that’s it! You can use your new GHC as usual.

To uninstall, follow the same procedure but give stow the -D option:

cd /usr/local/stow && stow -D ghc-7.6.1

To switch from one version of GHC to another just uninstall the current version and then install the new one. I actually have a script which automates this process: it looks in the stow directory to see which versions are availble, prompts me to choose one from a list, then uninstalls the current version (obtained with ghc --numeric-version) and installs the newly selected version. On my laptop this process takes just a couple seconds.

That’s it! Happy stowing!