Union Types with Clojure.Spec

Elm and other statically typed languages have a great feature called Union Types (also called Sum Types or Algebraic Data Types).

Here’s an example taken from Elm. Suppose your system used to represent users as integers, maybe just an auto-incrementing primary key, but then switched to UUIDs represented as strings.

To correctly model this situation, you need a way to create a type that can be either an integer or a string, that’s what union types give you.

type UserID = OldID Int | NewID String

Now each time you want to do something with a user id, you’ll have to add code to cover all these cases. This you can do with Elm’s case ... of. The interesting thing is that if you miss one of the cases, then the compiler will complain. Say in the future you decide to use a reified UUID type, now the code won’t compile until you’ve handled that case everywhere you’re dealing with user ids.

toNewID : UserID -> String
toNewID userID =
    case userID of
      OldID number -> toString number
      NewID string -> string

With clojure.spec we get the same power of expression, here’s how a ::user-id spec would look like.

(require [clojure.spec :as s])

(s/def ::user-id (s/or :old-id int?
                       :new-id string?))

Consuming this is straightforward, especially with core.match.

(require [core.match :refer [match]])

(defn format-id [user-id]
  (match [(s/conform ::user-id user-id)]
    [:old-id id] (str "Old id: " id)
    [:new-id id] (str "New id: " id)
    :else "Invalid id"))

So we get pretty much the same functionality, even the syntax is still pretty clean, but there’s one crucial difference: we don’t get compile time, or even runtime checks to make sure we’re covering all cases.

I thought it would be a fun excercise to implement a case-of macro, that during macro-expansion checks to see if all cases are covered. This is what it looks like in action.

(require '[lambdaisland.uniontypes :refer [case-of])

(s/def ::availability (s/or :sold-out  #{:sold-out}
                            :in-stock  pos-int?
                            :reordered (s/tuple pos-int? pos-int?)
                            :announced string?))

(defn format-availability [a]
  (case-of ::availability a
    :sold-out _
    "Sold out."

    :in-stock amount
    (str "In stock: " amount " items left.")

    :reordered [min max]
    (str "Available again in " min " to " max "days" )

    :announced date
    (str "Will be available on: " date)))

The first argument is the name of a spec, the second is the value to check. After that you give the name of a case in the s/or spec, a binding form that will receive the conformed value, you can also use destructuring, as in the :reordered case, and finally the code that handles the given case.

If you have too many or too few cases, your code will not compile, and you’ll get an error message that looks like this.

The cases in this `case-of` are different from the ones in the spec:

    (s/def :lambdaisland.uniontypes-test/availability
      (s/or :sold-out #{:sold-out}
            :in-stock pos-int?
            :reordered (tuple pos-int? pos-int?)
            :announced string?))

Add a case for :announced.
Remove the case :foo.

If the given value does not conform to the spec, it’ll throw an exception. You can override that behavior by adding a special :spec/invalid case. (Note: that’s not:clojure.spec/invalid for… reasons).

The given variable will in this case be bound to the result of clojure.spec/explain-data.

The code is on Github as well as Clojars.

[lambdaisland/uniontypes "0.1.0"]

Use cases

It might not be immediately obvious why this compile time check is so important, so let me give another example.

You have a chat application, each channel contains messages, and notifications like people coming and going. In your application state this is modeled as a single heterogeneous sequence. While rendering you need to loop over this sequence, and render the right component depending on the type of each entry.

The same kind of data is used to display notifications, to update a search index, and to aggregate statistics.

The new product manager decides you really need picture uploads, so that becomes a new type of entry. Now you need to update all of the places in your code where you deal with entries. Normally you would try to grep, or just do it from memory, and hope for the best. If you were using uniontypes, you could be sure you weren’t forgetting any of these cases.

ClojureScript support

So does it work on ClojureScript you ask… good question! Yes and no. I tried hard to make it compatible with ClojureScript, and that threw more than one spanner in the works. I got it working though, in the sense that lein doo phantom tells me all the tests pass on ClojureScript (there aren’t many, but enough to test at least a happy path and some error conditions).

Any attempt to actually use it on a ClojureScript project went awry, unfortunately.

One of the reasons it’s hard to do this on ClojureScript is that the spec is needed both at Compile time, to check that you have all the branches, and at runtime, to validate and conform the value. That means the spec provided by the end user needs to be available both on Clojure and ClojureScript, in other words, it’s necessary to put your specs in cljc files.

That alone isn’t always enough to also make sure the spec is loaded during macroexpansion, so the library will try to load the namespace based on the name of the spec. So if you do (case-of :foo.bar/baz), then during macroexpansion this will try to load foo.bar. This worked to get the tests going, but might also be what’s causing issues. It’s not the greatest hack as far as hacks go.

Another “interesting” issue I ran into was the difference between clojure.spec and cljs.spec. ClojureScript has gotten smarter about this, in the sense that when you (require '[clojure.spec :as s]) it will actually require cljs.spec, but when writing macros you need to make sure manually that the symbols you’re generating are in the right namespace.

What I ended up doing was detecting if the macro was being expanded for use in ClojureScript or not using the (:ns &env)trick, and then doing a clojure.walk/prewalk over the generated code, to replace all symbols and keywords in the clojure.spec namespace with their cousins in cljs.spec.


I thought this would be one evening hack, it turned out to be a bit more involved than that. I’m a bit bummed that ClojureScript support isn’t fully there yet, as I think this could be especially useful in UI programming. The fact that it’s working for the restricted case of running tests on Phantom.js does mean that it’s compatible in principle, if anyone cares to do the final stretch of troubleshooting.

I need to dogfood this some more to see how useful it will be in practice, but I think it does show that there are still a lot of imaginative uses of clojure.spec left to be uncovered.

More blog posts

Fork This Conference

Last weekend Heart of Clojure took place in Leuven, Belgium. As one of the core organizers it was extremely gratifying to see this event come to life. We started with a vision of a particular type of event we wanted to create, and I feel like we delivered on all fronts.

For an impression of what it was like you can check out Malwine’s comic summary, or Manuel’s blog post.

It seems people had a good time, and a lot of people are already asking about the next edition. However we don’t intend to make this a yearly recurring conference. We might be back in two years, maybe with another Heart of Clojure, maybe with something else. We need to think about that.

Advice to My Younger Self

When I was 16 I was visited by a man who said he had come from the future. He had traveled twenty years back to 1999 to sit down with me and have a chat.

We talked for an hour or so, and in the end he gave me a few pieces of advice. I have lived by these and they have served me well, and now I dispense this advice to you.

Become allergic to The Churn

ClojureScript logging with goog.log

This post explores goog.log, and builds an idiomatic ClojureScript wrapper, with support for cljs-devtools, cross-platform logging (by being API-compatible with Pedestal Log), and logging in production.

This deep dive into GCL’s logging functionality was inspired by work done with Nextjournal, whose support greatly helped in putting this library together.

Clojure’s standard library isn’t as “batteries included” as, say, Python. This is because Clojure and ClojureScript are hosted languages. They rely on a host platform to provide the lower level runtime functionality, which also allows them to tap into the host language’s standard library and ecosystem. That’s your batteries right there.

The Art of Tree Shaping with Clojure Zippers

This is a talk I did for the “Den of Clojure” meetup in Denver, Colorado. Enjoy!

Captions (subtitles) are available, and you can find the transcript below, as well as slides over here.

For comments and discussion please refer to this post on r/Clojure.

Test Wars: A New Hope

Yesterday was the first day for me on a new job, thanks to Clojurists Together I will be able to dedicate the coming three months to improving Kaocha, a next generation test runner for Clojure.

A number of projects applied for grants this quarter, some much more established than Kaocha. Clojurists Together has been asking people through their surveys if it would be cool to also fund “speculative” projects, and it seems people agreed.

I am extremely grateful for this opportunity. I hope to demonstrate in the coming months that Kaocha holds a lot of potential, and to deliver some of that potential in the form of a tool people love to use.

Two Years of Lambda Island, A Healthy Pace and Things to Come

It’s been just over two years since Lambda Island first launched, and just like last year I’d like to give you all an update about what’s been happening, where we are, and where things are going.

To recap: the first year was rough. I’d been self-employed for nearly a decade, but I’d always done stable contracting work, which provided a steady stream of income, and made it easy for me to unplug at the end of the day.

Lambda Island was, as the Dutch expression goes, “a different pair of sleeves”. I really underestimated what switching to a one-man product business in a niche market would mean, and within months I was struggling with symptoms of burnout, so most of year one was characterised by trying to keep things going and stay afloat financially, while looking after myself and trying to get back to a good place, physically and mentally.

D3 and ClojureScript

This is a guest post by Joanne Cheng (twitter), a freelance software engineer and visualization consultant based in Denver, Colorado. She has taught workshops and spoken at conferences about visualizing data with D3. Turns out ClojureScript and D3 are a great fit, in this post she’ll show you how to create your own visualization using the power of D3 and the elegance of ClojureScript.

I use D3.js for drawing custom data visualizations. I love using the library, but I wanted to try one of the several compile to JavaScript options, and I decided to look into ClojureScript. It ended up working out well for me, so I’m going to show you how I created a D3.js visualization using ClojureScript!

What we’re visualizing

Reloading Woes

Setting the Stage

When doing client work I put a lot of emphasis on tooling and workflow. By coaching people on their workflow, and by making sure the tooling is there to support it, a team can become many times more effective and productive.

An important part of that is having a good story for code reloading. Real world projects tend to have many dependencies and a large amount of code, making them slow to boot up, so we want to avoid having to restart the process.

The Bare Minimum, or Making Mayonnaise with Clojure

Making Mayonnaise

Imagine you have a grandfather who’s great at making mayonnaise. He’s been making mayonnaise since before the war, and the result is truly excellent. What’s more, he does this with a good old fashioned whisk. He’s kept his right arm in shape throughout decades just by beating those eggs and oil and vinegar.

Now he’s bought himself a handheld electric mixer after hearing his friend rave about hers, but after a few tries he gives up and goes back to his whisk. He says he just can’t get the same result. This seems slightly odd, so the next time you go over you ask him to show you how he uses the mixer.

Clojure Gotchas: "contains?" and Associative Collections

When learning a programming language we rarely read the reference documentation front to back. Instead someone might follow some tutorials, and look at sample code, until they’re confident enough to start a little project for practice.

From that point on the learning process is largely “just in time”, looking up exactly the things you need to perform the task at hand. As this goes on you might start to recognize some patterns, some internal logic that allows you to “intuit” how one part of the language works, based on experience with another part.

Developing this “intuition” — understanding this internal logic — is key to using a language effectively, but occasionally our intuition will be off. Some things are simply not obvious, unless someone has explained them to us. In this post I will look at something that frequently trips people up, and attempt to explain the underlying reasoning.

Dates in Clojure: Making Sense of the Mess

Update 2018-11-27: while most of this article is still relevant, I no longer recommend using JodaTime as the main date/time representation for new projects. Even existing projects that aren’t too invested in JodaTime/clj-time should consider migrating to java.time and clojure.java-time across the board.

Update 2 2019-05-29: Also check out the talk Cross Platform DateTime Awesomeness by Henry Widd, given at Clojure/north 2019

You can always count on human culture to make programming messy. To find out if a person is a programmer just have them say “encodings” or “timezones” and watch their face.

Clojure Gotchas: Surrogate Pairs

tl;dr: both Java and JavaScript have trouble dealing with unicode characters from Supplementary Planes, like emoji 😱💣.

Today I started working on the next feature of lambdaisland/uri, URI normalization. I worked test-first, you’ll get to see how that went in the next Lambda Island episode.

One of the design goals for this library is to have 100% parity between Clojure and ClojureScript. Learn once, use anywhere. The code is all written in .cljc files, so it can be treated as either Clojure or ClojureScript. Only where necessary am I using a small amount of reader conditionals.

Simple and Happy; is Clojure dying, and what has Ruby got to do with it?

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure failure stories. They were pointed at this talk from RubyConf 2016, which seemed to hit a lot of people right in the feels, and sparked a subthread with a life of its own.

Meanwhile Ray, one of the hosts of the defn podcast reacted to the original tweet: “I’m calling it: Clojure is alive and well with excellent defaults for productive and sustainable software development.” This sparked another big thread.

Loading Clojure Libraries Directly From Github

Did you ever fix a bug in an open source library, and then had to wait until the maintainer released an updated version?

It’s happened to me many times, the latest one being Toucan. I had run into a limitation, and found out that there was already an open ticket. It wasn’t a big change so I decided to dive in and address it. Just a little yak shave so I could get on with my life.

Now this pull request needs to be reviewed, and merged, and eventually be released to Clojars, but ain’t nobody got time for that stuff. No sir-ee.

Lambda Island Turns One, The Story of a Rocky Ride

One year ago to date I launched Lambda Island, a service that offers high quality video tutorials on web development with Clojure and ClojureScript. It’s been quite a ride. In this post I want to look back at the past year, provide some insight into how this experience has been for me, and give you a glimpse of what the future has in store.

This story really starts in December 2015. After three years of doing contract work for Ticketsolve I decided it was time for a change. I have been self-employed for many years, but I knew that sooner or later I wanted to try my hand at selling a product, rather than selling my time.

In January and February I took some time for soul-searching, and recharging. I went to speak at RubyConf Australia, and got to hang out with some old friends around Australia and New Zealand. Once back in Berlin I got busy creating Lambda Island.

Writing Node.js scripts with ClojureScript

In the two most recent  Lambda Island episodes I covered in-depth how to create command line utilities based on Lumo, how to combine them with third party libraries, and how to deploy them to npmjs.com.

However there’s a different way to create tools with ClojureScript and distribute them through NPM, without relying on Lumo. In this blog post I want to quickly demostrate how to do just that.

To recap, Lumo is a ClojureScript environment based on Node.js, using bootstrapped (self-hosted) ClojureScript. This means the ClojureScript compiler, which is written in Clojure and runs on the JVM, is used to compile itself to JavaScript. This way the JVM is no longer needed, all you need is a JavaScript runtime to compile and run ClojureScript code, which in this case is provided by Node.js. On top of that Lumo uses nexe, so Lumo can be distributed as a single compact and fast executable binary.

Announcing lambdaisland/uri 1.0.0

I just released lambdaisland/uri, a pure Clojure/ClojureScript URI library. It is available on Github and Clojars.

This is a small piece of the code base that powers lambdaisland.com. It’s inspired by Ruby’s Addressable::URI, the most solid URI implementation I’ve seen to date, although it only offers a small part of the functionality that library offers.

It’s written in pure Clojure/ClojureScript, with only minimal use of .cljc reader conditionals to smooth over differences in regular expression syntax, and differences in core protocols. It does not rely on any URI functionality offered by the host, such as java.net.URL, so it’s usable across all current and future Clojure implementations (Clojure, ClojureScript, ClojureCLR).

re-frame Subscriptions Got Even Better

Up until recently, to use re-frame subscriptions in Reagent views, you had to use a form-2 component.

A form-2 component is a function that returns another function, which does the actual rendering of the component to hiccup. In contrast, a form-1 component renders the hiccup directly.

;; form-1
(defn todo-item [todo]
   [todo-checkbox (:id todo) (:completed todo)]
   [:label {:unselectable "on"} title]
   [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]])

;; form-2
(defn todo-item [todo]
  (fn [todo]
     [todo-checkbox (:id todo) (:completed todo)]
     [:label {:unselectable "on"} title]
     [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]]))

Game Development with Clojure/ClojureScript

This weekend it’s Ludum Dare again, the world’s longest running game jam. The idea is that, alone or with a team, you build a game in a weekend based on a certain theme.

We got a little team together here in Berlin, and so I’ve been reviewing what options there are for someone wanting to build a game in Clojure or Clojurescript.

The good news is there are plenty of options, as you’ll see from the list below. You can do desktop games, browser based games with canvas or webgl, and you can even create Unity 3D games, all from your comfortable Clojure parentheses.