Reloading Woes

Setting the Stage

When doing client work I put a lot of emphasis on tooling and workflow. By coaching people on their workflow, and by making sure the tooling is there to support it, a team can become many times more effective and productive.

An important part of that is having a good story for code reloading. Real world projects tend to have many dependencies and a large amount of code, making them slow to boot up, so we want to avoid having to restart the process.

Restarting the process is done when things get into a bad state. This can be application state (where did my database connection go?), or Clojure state (I’m redefining this middleware function, but the webserver isn’t picking it up.)

tools.namespace and component/integrant

We can avoid the cataclysm of a restart if we can find some other way to get to a clean slate. To refresh Clojure’s view of the world, there’s, which is able to unload and then reload namespaces.

To recreate the app state from scratch there’s the “reloaded workflow” as first described by Stuart Sierra, who created Component for this reason.

Combining these two allows you to tear down the app state, reload all your Clojure code, then recreate the app state, this time based on the freshly loaded code. Combining them like this is important because when you use tools.namespace to reload your code, it effectively creates completely new and separate versions of functions, records, multimethods, vars, even namespace objects, and any state that is left around from before the reload might still be referencing the old ones, making matters even worse than before.

So you need to combine the two, which (naively) would look something like this.

(ns user
  (:require [com.stuartsierra.component :as component]
            [ :as c.t.n.r]))

(def app-state nil)

(defn new-system []
  (component/system-map ,,,))

(defn go []
  (alter-var-root! app-state (fn [_]
                               (-> (new-system)

(defn reset
  "Stop the app, reload all changed namespaces, start the app again."
  (component/stop app-state)

  ;; Clojure tools namespace takes a symbol, rather than a function, so that it
  ;; can resolve the function *after code reloading*. That way it's sure to get
  ;; the latest version.
  (c.t.n.r/refresh :after 'user/go))

This pattern is encoded in reloaded.repl; System also contains an implementation. I recommend using an existing implementation over rolling your own, as there are some details to get right. It’s also interesting to note that reloaded.repl uses suspendable, an extension to Component that adds a suspend/resume operation.

To dig deeper into this topic, check out the Lambda Island episodes on Component and System.

The author of reloaded.repl, James Reeves, eventually became dissatisfied with Component, and created Integrant as an alternative, with integrant-repl as the counterpart of reloaded.repl.

With reloaded.repl your own utility code now looks like this:

(ns user
  (:require reloaded.repl

(defn new-system []
  (com.stuartsierra.component/system-map ,,,))

(reloaded.repl/set-init! new-system)

Now you start the system with (reloaded.repl/go). To stop the system, reload all changed namespaces, and start the system again, you do (reloaded.repl/reset).

With integrant-repl things look similar.

(ns user
  (:require integrant.repl

(defn system-config []
  (-> "system_config.edn" slurp clojure.edn/read-string))

(integrant.repl/set-prep! system-config)

Now (integrant.repl/go) will bring your system up, and with (integrant.repl/reset) you can get back to a clean slate.


CIDER (Emacs’s Clojure integration) supports tools.namespace through the cider-refresh command. This only does code reloading though, but you can tell it to run a function before and after the refresh, to achieve the same effect.

;; emacs config

;; reloaded.repl
(setq  cider-refresh-before-fn "reloaded.repl/suspend")
(setq  cider-refresh-after-fn "reloaded.repl/resume")

;; integrant-repl
(setq  cider-refresh-before-fn "integrant.repl/suspend")
(setq  cider-refresh-after-fn "integrant.repl/resume")

You can also configure this on a per-project basis, by creating a file called .dir-locals.el in the project root, which looks like this.

((nil . ((cider-refresh-before-fn . "integrant.repl/suspend")
         (cider-refresh-after-fn . "integrant.repl/resume"))))

Now calling cider-refresh (Spacemacs: , s x) will again stop the system, reload all namespaces, and restart.

Plot twist

So far so good, this was a lengthy introduction, but I wanted to make sure we’re all on the same page, now let’s look at some of the things that might spoil our lovely reloading fun.


Clojure has a nifty feature called AOT or “ahead of time compilation”. This causes Clojure to pre-compile namespaces to Java classes and save those out to disk. This can be a useful feature as part of a deployment pipeline, because it can speed up booting your app. It has some serious drawbacks though, as it messes with Clojure’s dynamic nature.

What tends to happen is that at some point people find out about AOT, they think it’s amazing, and enable it everywhere. A bit later errors start popping up during development that just make no sense.

AOT should only be used for deployed applications. Don’t use it during development, and don’t use it for libraries you publish.

Even when following this advice you can get into trouble. Say you have aot enabled as part of the process of building an uberjar for deployment.

(defproject my-project "0.1.0"
  :profiles {:uberjar
             {:aot true}})

You do a lein uberjar to test your build locally. This will create AOT compiled classes and put them on the classpath (under target/ to be precise). Next time you try to (reset) it will tell you it can’t find certain Clojure source files, even though they’re right there. I have lost hours of my life figuring this out, and have more than once found my own Stackoverflow question+answer when googling for this. Watch out for the ghost of AOT!

By the way, here’s a handy oneliner: git clean -xfd. It removes any files not tracked by git, including files in .gitignore. It’s the most thorough way of cleaning out a repository. Do watch out, this might delete files you still want to keep! With git clean -nxfd you can do a dry run to see what it plans to delete.

defonce won’t save you

Ideally all your state is inside your system, but maybe there’s something else that you want to carry over between reloads. No need to judge, life is compromise.

You might think, “I know, I’ll just defonce it!”, but defonce won’t save you. tools.namespace will completely remove namespaces with everything in them before loading them again, so that var created by defonce the first time is long gone by the time defonce gets called again, and so it happily defines it anew.

What you can do instead is add some namespace metadata telling tools.namespace not to unload the namespace.

(ns my-ns
  (:require [ :as c.t.n.r]))

(c.t.n.r/disable-unload!) ;; this adds false
                          ;; to the namespace metadata

(defonce only-once (Date.))

This namespace will still get reloaded, so if you have functions in there then you’ll get the updated definitions, but it won’t be unloaded first, so defonce will work as expected. This does also mean that if you remove (or rename) a function, the old one is still around.

To completely opt-out of reloading, use (c.t.n.r/disable-reload!). This implies disable-unload!.

cljc + ClojureScript source maps

When ClojureScript generates source maps, it needs a way to point the browser at the original source files, so it copies cljs and cljc files into your web server document root. (your “public/” directory). Seems innocent enough, but it can confuse tools.namespace.

It is quite a common practice to search for files that are requested over HTTP in the classpath, so the “public/” directory tends to be on the classpath as well. tools.namespace scans the classpath for Clojure files (clj or cljc) and finds those cljc files that ClojureScript copied there, but their namespace name doesn’t correspond with their location, and things breaks. There is a JIRA ticket for this for tools.namespace: TNS-45, with several proposed patches, but no consensus yet about the right way forward.

The easiest way around this is to limit tools.namespace to only scan your own source directories.

(c.t.n.r/set-refresh-dirs "src/clj")

CIDER does its own thing

cider-refresh roughly does the same thing as, and it uses tools.namespace under the hood, but it reimplements refresh using lower level tools.namespace primitives. This leads to subtle differences.

Currently cider-refresh does not honor set-refresh-dirs. It honors the unload/load metadata, but does not follow the convention that false implies false.

I proposed a patch to address both of these issues. In general if cider-refresh seems to have issues, then try to use (reset) directly in the REPL.

More blog posts

Fork This Conference

Last weekend Heart of Clojure took place in Leuven, Belgium. As one of the core organizers it was extremely gratifying to see this event come to life. We started with a vision of a particular type of event we wanted to create, and I feel like we delivered on all fronts.

For an impression of what it was like you can check out Malwine’s comic summary, or Manuel’s blog post.

It seems people had a good time, and a lot of people are already asking about the next edition. However we don’t intend to make this a yearly recurring conference. We might be back in two years, maybe with another Heart of Clojure, maybe with something else. We need to think about that.

Advice to My Younger Self

When I was 16 I was visited by a man who said he had come from the future. He had traveled twenty years back to 1999 to sit down with me and have a chat.

We talked for an hour or so, and in the end he gave me a few pieces of advice. I have lived by these and they have served me well, and now I dispense this advice to you.

Become allergic to The Churn

ClojureScript logging with goog.log

This post explores goog.log, and builds an idiomatic ClojureScript wrapper, with support for cljs-devtools, cross-platform logging (by being API-compatible with Pedestal Log), and logging in production.

This deep dive into GCL’s logging functionality was inspired by work done with Nextjournal, whose support greatly helped in putting this library together.

Clojure’s standard library isn’t as “batteries included” as, say, Python. This is because Clojure and ClojureScript are hosted languages. They rely on a host platform to provide the lower level runtime functionality, which also allows them to tap into the host language’s standard library and ecosystem. That’s your batteries right there.

The Art of Tree Shaping with Clojure Zippers

This is a talk I did for the “Den of Clojure” meetup in Denver, Colorado. Enjoy!

Captions (subtitles) are available, and you can find the transcript below, as well as slides over here.

For comments and discussion please refer to this post on r/Clojure.

Test Wars: A New Hope

Yesterday was the first day for me on a new job, thanks to Clojurists Together I will be able to dedicate the coming three months to improving Kaocha, a next generation test runner for Clojure.

A number of projects applied for grants this quarter, some much more established than Kaocha. Clojurists Together has been asking people through their surveys if it would be cool to also fund “speculative” projects, and it seems people agreed.

I am extremely grateful for this opportunity. I hope to demonstrate in the coming months that Kaocha holds a lot of potential, and to deliver some of that potential in the form of a tool people love to use.

Two Years of Lambda Island, A Healthy Pace and Things to Come

It’s been just over two years since Lambda Island first launched, and just like last year I’d like to give you all an update about what’s been happening, where we are, and where things are going.

To recap: the first year was rough. I’d been self-employed for nearly a decade, but I’d always done stable contracting work, which provided a steady stream of income, and made it easy for me to unplug at the end of the day.

Lambda Island was, as the Dutch expression goes, “a different pair of sleeves”. I really underestimated what switching to a one-man product business in a niche market would mean, and within months I was struggling with symptoms of burnout, so most of year one was characterised by trying to keep things going and stay afloat financially, while looking after myself and trying to get back to a good place, physically and mentally.

D3 and ClojureScript

This is a guest post by Joanne Cheng (twitter), a freelance software engineer and visualization consultant based in Denver, Colorado. She has taught workshops and spoken at conferences about visualizing data with D3. Turns out ClojureScript and D3 are a great fit, in this post she’ll show you how to create your own visualization using the power of D3 and the elegance of ClojureScript.

I use D3.js for drawing custom data visualizations. I love using the library, but I wanted to try one of the several compile to JavaScript options, and I decided to look into ClojureScript. It ended up working out well for me, so I’m going to show you how I created a D3.js visualization using ClojureScript!

What we’re visualizing

The Bare Minimum, or Making Mayonnaise with Clojure

Making Mayonnaise

Imagine you have a grandfather who’s great at making mayonnaise. He’s been making mayonnaise since before the war, and the result is truly excellent. What’s more, he does this with a good old fashioned whisk. He’s kept his right arm in shape throughout decades just by beating those eggs and oil and vinegar.

Now he’s bought himself a handheld electric mixer after hearing his friend rave about hers, but after a few tries he gives up and goes back to his whisk. He says he just can’t get the same result. This seems slightly odd, so the next time you go over you ask him to show you how he uses the mixer.

Clojure Gotchas: "contains?" and Associative Collections

When learning a programming language we rarely read the reference documentation front to back. Instead someone might follow some tutorials, and look at sample code, until they’re confident enough to start a little project for practice.

From that point on the learning process is largely “just in time”, looking up exactly the things you need to perform the task at hand. As this goes on you might start to recognize some patterns, some internal logic that allows you to “intuit” how one part of the language works, based on experience with another part.

Developing this “intuition” — understanding this internal logic — is key to using a language effectively, but occasionally our intuition will be off. Some things are simply not obvious, unless someone has explained them to us. In this post I will look at something that frequently trips people up, and attempt to explain the underlying reasoning.

Dates in Clojure: Making Sense of the Mess

Update 2018-11-27: while most of this article is still relevant, I no longer recommend using JodaTime as the main date/time representation for new projects. Even existing projects that aren’t too invested in JodaTime/clj-time should consider migrating to java.time and across the board.

Update 2 2019-05-29: Also check out the talk Cross Platform DateTime Awesomeness by Henry Widd, given at Clojure/north 2019

You can always count on human culture to make programming messy. To find out if a person is a programmer just have them say “encodings” or “timezones” and watch their face.

Clojure Gotchas: Surrogate Pairs

tl;dr: both Java and JavaScript have trouble dealing with unicode characters from Supplementary Planes, like emoji 😱💣.

Today I started working on the next feature of lambdaisland/uri, URI normalization. I worked test-first, you’ll get to see how that went in the next Lambda Island episode.

One of the design goals for this library is to have 100% parity between Clojure and ClojureScript. Learn once, use anywhere. The code is all written in .cljc files, so it can be treated as either Clojure or ClojureScript. Only where necessary am I using a small amount of reader conditionals.

Simple and Happy; is Clojure dying, and what has Ruby got to do with it?

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure failure stories. They were pointed at this talk from RubyConf 2016, which seemed to hit a lot of people right in the feels, and sparked a subthread with a life of its own.

Meanwhile Ray, one of the hosts of the defn podcast reacted to the original tweet: “I’m calling it: Clojure is alive and well with excellent defaults for productive and sustainable software development.” This sparked another big thread.

Loading Clojure Libraries Directly From Github

Did you ever fix a bug in an open source library, and then had to wait until the maintainer released an updated version?

It’s happened to me many times, the latest one being Toucan. I had run into a limitation, and found out that there was already an open ticket. It wasn’t a big change so I decided to dive in and address it. Just a little yak shave so I could get on with my life.

Now this pull request needs to be reviewed, and merged, and eventually be released to Clojars, but ain’t nobody got time for that stuff. No sir-ee.

Lambda Island Turns One, The Story of a Rocky Ride

One year ago to date I launched Lambda Island, a service that offers high quality video tutorials on web development with Clojure and ClojureScript. It’s been quite a ride. In this post I want to look back at the past year, provide some insight into how this experience has been for me, and give you a glimpse of what the future has in store.

This story really starts in December 2015. After three years of doing contract work for Ticketsolve I decided it was time for a change. I have been self-employed for many years, but I knew that sooner or later I wanted to try my hand at selling a product, rather than selling my time.

In January and February I took some time for soul-searching, and recharging. I went to speak at RubyConf Australia, and got to hang out with some old friends around Australia and New Zealand. Once back in Berlin I got busy creating Lambda Island.

Writing Node.js scripts with ClojureScript

In the two most recent  Lambda Island episodes I covered in-depth how to create command line utilities based on Lumo, how to combine them with third party libraries, and how to deploy them to

However there’s a different way to create tools with ClojureScript and distribute them through NPM, without relying on Lumo. In this blog post I want to quickly demostrate how to do just that.

To recap, Lumo is a ClojureScript environment based on Node.js, using bootstrapped (self-hosted) ClojureScript. This means the ClojureScript compiler, which is written in Clojure and runs on the JVM, is used to compile itself to JavaScript. This way the JVM is no longer needed, all you need is a JavaScript runtime to compile and run ClojureScript code, which in this case is provided by Node.js. On top of that Lumo uses nexe, so Lumo can be distributed as a single compact and fast executable binary.

Announcing lambdaisland/uri 1.0.0

I just released lambdaisland/uri, a pure Clojure/ClojureScript URI library. It is available on Github and Clojars.

This is a small piece of the code base that powers It’s inspired by Ruby’s Addressable::URI, the most solid URI implementation I’ve seen to date, although it only offers a small part of the functionality that library offers.

It’s written in pure Clojure/ClojureScript, with only minimal use of .cljc reader conditionals to smooth over differences in regular expression syntax, and differences in core protocols. It does not rely on any URI functionality offered by the host, such as, so it’s usable across all current and future Clojure implementations (Clojure, ClojureScript, ClojureCLR).

re-frame Subscriptions Got Even Better

Up until recently, to use re-frame subscriptions in Reagent views, you had to use a form-2 component.

A form-2 component is a function that returns another function, which does the actual rendering of the component to hiccup. In contrast, a form-1 component renders the hiccup directly.

;; form-1
(defn todo-item [todo]
   [todo-checkbox (:id todo) (:completed todo)]
   [:label {:unselectable "on"} title]
   [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]])

;; form-2
(defn todo-item [todo]
  (fn [todo]
     [todo-checkbox (:id todo) (:completed todo)]
     [:label {:unselectable "on"} title]
     [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]]))

Game Development with Clojure/ClojureScript

This weekend it’s Ludum Dare again, the world’s longest running game jam. The idea is that, alone or with a team, you build a game in a weekend based on a certain theme.

We got a little team together here in Berlin, and so I’ve been reviewing what options there are for someone wanting to build a game in Clojure or Clojurescript.

The good news is there are plenty of options, as you’ll see from the list below. You can do desktop games, browser based games with canvas or webgl, and you can even create Unity 3D games, all from your comfortable Clojure parentheses.

Union Types with Clojure.Spec

Elm and other statically typed languages have a great feature called Union Types (also called Sum Types or Algebraic Data Types).

Here’s an example taken from Elm. Suppose your system used to represent users as integers, maybe just an auto-incrementing primary key, but then switched to UUIDs represented as strings.

To correctly model this situation, you need a way to create a type that can be either an integer or a string, that’s what union types give you.