The (Real) Reason I Like Elixir

There are lots of excellent reasons to learn Elixir. Thanks to Erlang, it feels like technology from the future. Learning the language, learning new ways of approaching problems, and interacting with the community all feel enjoyable.

But think about the songs, books and movies that are "guilty pleasures" to you. Why do you like them? Often, it's something strange and difficult to pin down, not a weighty virtue, that resonates.

The real reason, the guilty pleasure reason that I like Elixir so much is it has Lisp-style macros in a language with syntax. I love the way Lisp's macros let me think about problems (though Lisp's expressive power isn't always spoken of as a positive1), but I also love languages with clean, block-structured syntax.

To some, this will sound odd. Most programmers fall in love with Lisp at some point, but why single out a language feature that's rarely-used in your day-to-day programming? After all, the number one rule of macro writing is: don't write a macro. You don't need to write macros at all to use Elixir very productively.

Well ... believe it or not, "macros outside of Lisp" are something I once wanted so much that I built them. I've been following and experimenting with macros outside of Lisp years, and I appreciate Elixir more because of it.

Experiments In Macros Outside of Lisp

In the summer of 2011, at the tail end of 9 months in South America, I wrote a prototype implementation of Lisp-style macros in CoffeeScript2 in under 100 lines of code.

Laptop and Hammock

The idea was that we could implement macros without requiring the language to look like a mass of parentheses, by altering the AST but leaving tokenizing/parsing of a language's distinctive syntax alone — in Lisp terms, macros without read-macros.

I was pretty excited at the time, and did a lightning talk at a monthly programming language User Group. I presented3 the project, and demoed two examples:

  1. "list comprehensions in Spanish" — so that instead of for x in [1..10] when x > 3 -> x, you could write para x en [1..10] si (x > 3) hazte eso pues -> x
  2. cc() — a control-flow macro that turned synchronous code async, by reaching down its throat and turning it inside-out. (Just callbacks, not continuations).

There's something in the air.

I hasten to add: that idea wasn't new! Dylan probably doesn't count, as it was Lisp with block syntax. And there were doubtless efforts to use 'Lisp-style' macros in non-lisps earlier in the history of programming.

But for recent examples, I learned that MetaLua had done this in 2007, and I had known that Potion could do macros in 2009 or so. In late 2011 Rust added them in their first form, and in 2012 Julia appeared. Later in 2012 Scala added them as experimental.

Notice anything interesting in that progression? It's that macros moved from toy languages, or toy language forks, to "big" languages. Rust. Scala. Language designers have begun to realize that you can include macros in your language without your users burning everything down. Macros, without read-macros.

While obvious to language designers, I was shocked to realize that there wasn't usually a hard technical reason4 why macros couldn't be added to an existing language as an AST transform step.

Another year of volunteer work in South America intervened, but I finally cleaned up the project a bit, made the code readable with Docco so that it could serve as a reference to others, and presented it in a non-lightning-talk format.

Unsolicited Skin Graft

Although a fork of CoffeeScript (BlackCoffee) was initially based on my prototype, and garnered some attention from @jashenkas, it was clear that any closer integration was unlikely. As the creator of a little language with macros, 'GorillaScript,' mentioned to me, once you have macros it makes sense to write as much of the language as possible with them.

But who would want to rewrite a language to use macros retroactively? In theory, some form of true macros could be bolted onto our favorite languages as just a transform step5. In practice, it makes the most sense to build a language on top of macros.

Obviously, languages shouldn't try to change their nature willy-nilly. So why even consider adding macros to existing languages? Because there weren't then, to my knowledge, any non-Lisps that had macros and were good for general-purpose programming, and had an active code ecosystem/package manager. The languages I mentioned above were validating macros outside of Lisp, but none of them6 were good choices for general-purpose programming (things like UTF, localization, good protocol/networking support, code sharing, etc).

A block-structured language with flexible syntax, Lisp-style macros and an active community seemed like a perfect language for everyday programming. But my idea to bolt macros on top of existing languages sucked. And I lacked in-depth knowledge of language implementations, the free time, and (realistically) the skills to consider anything else.

Macros are the Hammer that Jose Used When He Freaking Nailed It

So when it began to dawn on me the extent to which Jose Valim had nailed it with Elixir, it was a little bit like living in a dream. Everything just made so much sense.

Of course, the things that are hardest to learn about Elixir to a non-Erlangist, the genuinely new ways of thinking, don't really have to do with the language. They have to do with Beam, and OTP, and the bits of Erlang that Elixir dispatches to — not language features like pattern matching, recursion, multiple functions heads, or even Lisp-style macros.

But Elixir makes a lot of bold decisions for a language trying to be accessible and productive. Processes, modules and functions only — no records, classes or objects! Macros make it work; they're such a useful compile-time means of abstraction and combination that stateful, runtime meta-programming can be merrily marginalized by language designers. They can say "you ain't gonna need it," secure in the knowledge that they have macros to fall back on.

It's paid off everywhere: code-generating UTF8 support; replacing 'macro methods' with actual macros; relational query languages that are languages; 'gettext' support. Protocols.

Elixir inherits the most from its Erlang heritage, and it's that heritage we credit for feeling that we've been gifted some kind of advanced alien technology. When I killed a supervised process tree in an OTP application, and for the first time I watched my system route around the damage and repair itself successfully, I didn't thank Lisp particularly.

But when I read Elixir code, I feel as though a lot of Lisp people have a reason to smile.

  1. The Lisp Curse, written April 2011, claims that Lisp's expressivity makes programmers less dependent on community, and posits a relative chilling effect on code sharing. The phrase is sort of a shorthand for Lisp's less-active library/package ecosystem. Personally, I think Lisp is doing fine (CL, Racket, Clojure, Joxa(!), and of course Emacs), but I think that Elixir will be as successful as the languages people compare Lisp to when they talk about the 'Lisp Curse.'

  2. The original code, which I put on github when I got back to the States, was dense and terrible. It only ran in the browser, as that was all I had access to while developing.

  3. A pretty cheesy-looking presentation

  4. ... because software is mostly social. However, there are 'soft technical reasons' it's easier in some languages: (1) an AST that can easily be serialized and deserialized makes 'quote' easier to implement, (2) for macro calls, a flexible syntax lets macros look good and blend in, and (3) for macro writers, pattern matching helps with the complex shape of non-Lisp quoted expressions.

  5. People have tried to retroactively add macros to already-popular languages, but empirically they seem to result in capable but little-used forks (MetaLua, Template Haskell). I'm not sure if Scala (whose macros were added in 2012 though still namespaced 'experimental') counts, as Scala in 2012 was still pretty malleable. For Rubyists, a project called RubyMacros made an effort in an earlier version of the language, and later Rubinius (the third-most-popular implementation of Ruby) supported parse transforms. But in Ruby's world, where run-time meta-programming reigns, it's never been clear why macros would be worth the disruption.

  6. One huge shoutout to Factor, a capable and brain-bending Forth-inspired language that can do macros (among many, many other things). However, there was no formalized code sharing in packages that would have supported a package manager at the time; last I heard they still didn't have one.