The Bare Minimum, or Making Mayonnaise with Clojure

Making Mayonnaise

Imagine you have a grandfather who’s great at making mayonnaise. He’s been making mayonnaise since before the war, and the result is truly excellent. What’s more, he does this with a good old fashioned whisk. He’s kept his right arm in shape throughout decades just by beating those eggs and oil and vinegar.

Now he’s bought himself a handheld electric mixer after hearing his friend rave about hers, but after a few tries he gives up and goes back to his whisk. He says he just can’t get the same result. This seems slightly odd, so the next time you go over you ask him to show you how he uses the mixer.

Imagine your face when he takes one of the mixer’s beaters and starts beating the stuff by hand. Turns out he actually had no idea how to use the mixer, so he just took the bit that seemed familiar, and started using it the way he was used to.

What does this have to do with Clojure?

A far fetched story perhaps, but it’s the best metaphor I could come up with to illustrate the difference between “writing Clojure”, and “doing Clojure”.

Sometimes people approach Clojure in a similar way. They’ve heard good things about it, and decide to try it out, while sticking to their familiar tools and workflow. They hear Clojurists talk about their Emacs and other exotica, and write it off as a strange fetish for obscure technology. They know coding. They already have an editor they know and love, thank you very much. All they want to know is how this code in $LANGUAGE translates to Clojure.

I made this same mistake some years ago when I decided to give SmallTalk a try. I looked at a couple of implementations, but they all seemed to come with this massive environment, including a UI, editor, and something called “images” that just did not make sense. All I wanted was to learn a language, all I needed (I thought), was a way to stick some code into a source file, compile and run it.

But SmallTalk, like Clojure, isn’t just a language. It’s a complete philosophy about creating software, with specific ideas about how programmers will interact with their environment. This makes it strange and unfamiliar, but if you really want to learn one of these languages you need to dive in wholesale. You will never be able to assess and appreciate what makes them stand apart by skirting around the edges of your comfort zone.

The sad part is that this stuff isn’t made very explicit. It’s the kind of knowledge that’s passed down from mentor to mentee, taught by example. If you’re learning Clojure all on your own, you may never realize what you’re missing.

So I’m here to tell you what the bare minimum setup is you need to not just be “writing Clojure”, but to actually be “doing Clojure”.

In short, there are two things that you simply can not do without. I’ll call these “managed parentheses”, and “in-place evaluation”. Take note that I did not say “use Emacs” (or Vim or ed(1) or whatever). Many editors and IDE’s can provide these features, but you do have to make sure your environment has these features, and you have to learn how to use them.

Managed Parentheses

Managed parentheses means that your editor makes it near impossible to have unbalanced parentheses in your code. You should never ever have to manually insert a closing parenthesis.

There are several ways to achieve this. The most well known approach is called “paredit”, originally built for Emacs, but with equivalent packages for most major editors. A different approach that’s been gaining popularity, and with less of a learning curve is Parinfer. The general term for this is “structural editing”.

You can find eplanations of how these work in detail elsewhere. Just take it from me that they manage your parentheses, so you don’t have to. What they really do is cause a mental shift, where you no longer think about parentheses, but instead think about blocks of code (also called forms, expressions, or sexps).

Clojure is a LISP. This in itself is a superpower. For decades people have been saying that treating programs as text is backwards, that we need higher level tools that allow us to manipulate the structure of the program directly. LISP drags you halfway there. It takes getting used to, but it pays off massively.

Don’t just take it from me, this is what user bbrinck had to say on ClojureVerse:

I have found that many beginners don’t realize that they will be much, much more productive once they learn paredit or parinfer. I worked in Clojure for a few years in my spare time without structural editing. I knew paredit existed, but I didn’t have a good sense of the time investment necessary vs the productivity difference.

It was only when I pair programmed with some experienced Clojurists that I realized how much time I was wasting on bugs related to parentheses that they never experienced.

In place evaluation

Most languages have REPLs, command line tools that let you enter and evaluate program code one line at a time. They’re great for exploration, for quickly trying things out, or for running commands on the fly.

In the LISP family of languages (as well as SmallTalk) this concept is taken one step further by integrating this REPL deeply into the editor experience. This allows you to put your cursor next to a piece of code in a source file, and evaluate it right there.

Because of LISP’s uniform structure you can unambigously single out any single sub-expression, and send that to the editor-connected-REPL, immediately seeing the result. This brings coding to a new level of interactivity. You are interacting with a full, running system, which you can inspect, update, and alter. You can even connect to your production system, to look at its state, or to quickly redefine a function to patch a bug.

You can also evaluate a piece of code, and insert the result back into your source file. People often say that in Clojure code is data, focusing on the manipulation of code structure with things like macros, but the equivalence goes both ways. Data is code, so any Clojure data structure, when printed, is a valid Clojure expression. Depending on where you’re coming from this will be either a no-brainer or a mind bender. In any case it’s extremely powerful. I said it before, LISP is a superpower.

What else?

These two are the absolute minimum. If you honestly want to give Clojure a try, then make sure you’re using an environment that has these features, and learn how to use them. Some popular options include

  • CIDER (Emacs)
  • Cursive (IntelliJ)
  • ProtoREPL (Atom)
  • fireplace.vim

There are others. The point is to not blindly adopt what someone tells you, but to be able to assess yourself whether the tool you’re using clears this bar.

These two things are only the absolute bare minimum. For a real world setup I would recommend a few more things. For all of these

  • check that your environment can do it
  • learn how to use it
  • make it part of your workflow

Automatic indentation

LISP, when properly indented, looks a lot like Python. There’s general concensus about how to indent Clojure, and so it’s something that easily can, and should be automated.

Either have a way to automatically correct the indentation of a line (e.g. by pressing TAB), or use something that automatically keeps the indentation of the whole file up to date (I recommend the latter, for Emacs this is called “agressive-indent-mode”).

Jump to definition

Before coming to Clojure I did a lot of Ruby. In Ruby when you’re looking at a method invocation it can be hard to figure out where that implementation lives, because it depends on runtime types. Not so in Clojure, the system always knows exactly where a given function is defined. Your editor can ask its integrated REPL where that is, so you can jump to the right file and line.

This is great for navigating your own project, as well for jumping into library code to better understand how it works. And while you’re there you could put a couple println statements in that library code, and re-evaluate it. Never second guess again what values are flowing through your system.

Show function signature

When your cursor is on a function name, your editor should show somewhere unintrusively what the signature of that function looks like. This little bit of information, the arity and argument names of a function, are 80% of the documentation you’ll need. Having it right there will help you stay in a state of flow.

Look up documentation

Functions in Clojure come with built-in documentation. All of core has these docs, and most libraries do as well. You really want to be able to pull these up with the touch of a button.

Rainbow parentheses

More of a nice to have, but worth a mention. This feature gives matching parentheses matching colors. I really like this, but it’s probably not essential.

Staying in the flow

If you have all of the above you’re already ahead of most programmers in the industry when it comes to tooling and workflow. Consider yourself lucky. The next step is to make sure you can stay in the flow as much as possible, to eradicate events that upset and interrupt your REPL based workflow.

Reloadable system state

A downside of dynamically working with a long running process is that eventually state will accrue, so that the in-memory state of your Clojure process no longer reflects the source files on disks. There could be functions that you’ve deleted or renamed, but that still exist in memory. You could have application state that no longer reflects a valid state of the program. There are a bunch of gotchas in Clojure that can cause these errors, for instance redefining a defrecord, or a defmulti.

This can cause code to work fine, but fail as soon as you restart the process, or it can cause things to fail even though they really should work. In both cases what you need is a clean slate.

Initially you’ll just take this in your stride and restart the REPL process. Once you have felt enough of this pain though you’ll look for a solution. What you’ll end up with will likely be a component-style library to manage your application state, something like Component, Mount, or Integrant, combined with clojure.tools.namespace for intelligent unloading and reloading of all your namespaces.

Hot loading of new dependencies

When developing applications, especially greenfield apps, you’ll regularly be adding more third party libraries as you go. Each time you add one of these to project.clj, deps.edn, or build.boot, you need to restart your Clojure process.

This can be solved through libraries like Pomegranate, which will install and hot-load new dependencies. Configure them in ~/.lein/profiles.clj or ~/.clojure/deps.edn so you have them available on every project.

Further reading

More blog posts

Two Years of Lambda Island, A Healthy Pace and Things to Come

It’s been just over two years since Lambda Island first launched, and just like last year I’d like to give you all an update about what’s been happening, where we are, and where things are going.

To recap: the first year was rough. I’d been self-employed for nearly a decade, but I’d always done stable contracting work, which provided a steady stream of income, and made it easy for me to unplug at the end of the day.

Lambda Island was, as the Dutch expression goes, “a different pair of sleeves”. I really underestimated what switching to a one-man product business in a niche market would mean, and within months I was struggling with symptoms of burnout, so most of year one was characterised by trying to keep things going and stay afloat financially, while looking after myself and trying to get back to a good place, physically and mentally.

D3 and ClojureScript

This is a guest post by Joanne Cheng (twitter), a freelance software engineer and visualization consultant based in Denver, Colorado. She has taught workshops and spoken at conferences about visualizing data with D3. Turns out ClojureScript and D3 are a great fit, in this post she’ll show you how to create your own visualization using the power of D3 and the elegance of ClojureScript.

I use D3.js for drawing custom data visualizations. I love using the library, but I wanted to try one of the several compile to JavaScript options, and I decided to look into ClojureScript. It ended up working out well for me, so I’m going to show you how I created a D3.js visualization using ClojureScript!

What we’re visualizing

Reloading Woes

Setting the Stage

When doing client work I put a lot of emphasis on tooling and workflow. By coaching people on their workflow, and by making sure the tooling is there to support it, a team can become many times more effective and productive.

An important part of that is having a good story for code reloading. Real world projects tend to have many dependencies and a large amount of code, making them slow to boot up, so we want to avoid having to restart the process.

Clojure Gotchas: "contains?" and Associative Collections

When learning a programming language we rarely read the reference documentation front to back. Instead someone might follow some tutorials, and look at sample code, until they’re confident enough to start a little project for practice.

From that point on the learning process is largely “just in time”, looking up exactly the things you need to perform the task at hand. As this goes on you might start to recognize some patterns, some internal logic that allows you to “intuit” how one part of the language works, based on experience with another part.

Developing this “intuition” — understanding this internal logic — is key to using a language effectively, but occasionally our intuition will be off. Some things are simply not obvious, unless someone has explained them to us. In this post I will look at something that frequently trips people up, and attempt to explain the underlying reasoning.

Dates in Clojure: Making Sense of the Mess

You can always count on human culture to make programming messy. To find out if a person is a programmer just have them say “encodings” or “timezones” and watch their face.

In Java, and hence Clojure, there are half a dozen different ways to represent dates and times, which can lead to confusion, needless type coercion, and inconsistent code. In this post I’ll give you a quick lay of the land, and some tips to make it all a bit easier to deal with.

Representing Time

Clojure Gotchas: Surrogate Pairs

tl;dr: both Java and JavaScript have trouble dealing with unicode characters from Supplementary Planes, like emoji 😱💣.

Today I started working on the next feature of lambdaisland/uri, URI normalization. I worked test-first, you’ll get to see how that went in the next Lambda Island episode.

One of the design goals for this library is to have 100% parity between Clojure and ClojureScript. Learn once, use anywhere. The code is all written in .cljc files, so it can be treated as either Clojure or ClojureScript. Only where necessary am I using a small amount of reader conditionals.

Simple and Happy; is Clojure dying, and what has Ruby got to do with it?

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure failure stories. They were pointed at this talk from RubyConf 2016, which seemed to hit a lot of people right in the feels, and sparked a subthread with a life of its own.

Meanwhile Ray, one of the hosts of the defn podcast reacted to the original tweet: “I’m calling it: Clojure is alive and well with excellent defaults for productive and sustainable software development.” This sparked another big thread.

Loading Clojure Libraries Directly From Github

Did you ever fix a bug in an open source library, and then had to wait until the maintainer released an updated version?

It’s happened to me many times, the latest one being Toucan. I had run into a limitation, and found out that there was already an open ticket. It wasn’t a big change so I decided to dive in and address it. Just a little yak shave so I could get on with my life.

Now this pull request needs to be reviewed, and merged, and eventually be released to Clojars, but ain’t nobody got time for that stuff. No sir-ee.

Lambda Island Turns One, The Story of a Rocky Ride

One year ago to date I launched Lambda Island, a service that offers high quality video tutorials on web development with Clojure and ClojureScript. It’s been quite a ride. In this post I want to look back at the past year, provide some insight into how this experience has been for me, and give you a glimpse of what the future has in store.

This story really starts in December 2015. After three years of doing contract work for Ticketsolve I decided it was time for a change. I have been self-employed for many years, but I knew that sooner or later I wanted to try my hand at selling a product, rather than selling my time.

In January and February I took some time for soul-searching, and recharging. I went to speak at RubyConf Australia, and got to hang out with some old friends around Australia and New Zealand. Once back in Berlin I got busy creating Lambda Island.

Writing Node.js scripts with ClojureScript

In the two most recent  Lambda Island episodes I covered in-depth how to create command line utilities based on Lumo, how to combine them with third party libraries, and how to deploy them to npmjs.com.

However there’s a different way to create tools with ClojureScript and distribute them through NPM, without relying on Lumo. In this blog post I want to quickly demostrate how to do just that.

To recap, Lumo is a ClojureScript environment based on Node.js, using bootstrapped (self-hosted) ClojureScript. This means the ClojureScript compiler, which is written in Clojure and runs on the JVM, is used to compile itself to JavaScript. This way the JVM is no longer needed, all you need is a JavaScript runtime to compile and run ClojureScript code, which in this case is provided by Node.js. On top of that Lumo uses nexe, so Lumo can be distributed as a single compact and fast executable binary.

Announcing lambdaisland/uri 1.0.0

I just released lambdaisland/uri, a pure Clojure/ClojureScript URI library. It is available on Github and Clojars.

This is a small piece of the code base that powers lambdaisland.com. It’s inspired by Ruby’s Addressable::URI, the most solid URI implementation I’ve seen to date, although it only offers a small part of the functionality that library offers.

It’s written in pure Clojure/ClojureScript, with only minimal use of .cljc reader conditionals to smooth over differences in regular expression syntax, and differences in core protocols. It does not rely on any URI functionality offered by the host, such as java.net.URL, so it’s usable across all current and future Clojure implementations (Clojure, ClojureScript, ClojureCLR).

re-frame Subscriptions Got Even Better

Up until recently, to use re-frame subscriptions in Reagent views, you had to use a form-2 component.

A form-2 component is a function that returns another function, which does the actual rendering of the component to hiccup. In contrast, a form-1 component renders the hiccup directly.

;; form-1
(defn todo-item [todo]
  [:div.view
   [todo-checkbox (:id todo) (:completed todo)]
   [:label {:unselectable "on"} title]
   [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]])

;; form-2
(defn todo-item [todo]
  (fn [todo]
    [:div.view
     [todo-checkbox (:id todo) (:completed todo)]
     [:label {:unselectable "on"} title]
     [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]]))

Game Development with Clojure/ClojureScript

This weekend it’s Ludum Dare again, the world’s longest running game jam. The idea is that, alone or with a team, you build a game in a weekend based on a certain theme.

We got a little team together here in Berlin, and so I’ve been reviewing what options there are for someone wanting to build a game in Clojure or Clojurescript.

The good news is there are plenty of options, as you’ll see from the list below. You can do desktop games, browser based games with canvas or webgl, and you can even create Unity 3D games, all from your comfortable Clojure parentheses.

Union Types with Clojure.Spec

Elm and other statically typed languages have a great feature called Union Types (also called Sum Types or Algebraic Data Types).

Here’s an example taken from Elm. Suppose your system used to represent users as integers, maybe just an auto-incrementing primary key, but then switched to UUIDs represented as strings.

To correctly model this situation, you need a way to create a type that can be either an integer or a string, that’s what union types give you.