Lambda Island

To be effective at programming, you have to understand your tools. In this episode we’ll peel a few layers away, and use the ClojureScript compiler without any help from Leiningen, Cljsbuild or Figwheel. You’ll learn about compiler options, levels of optimization, the role of the Google Closure Library, and how to confgure the compiler for development and production use.

Transcript

If you’re just starting out with ClojureScript, either with or without tinkering with Clojure first, it’s easy to get lost in ClojureScript’s setup and tooling.

There are various projects and tools that try to make your life easier, but which ones do you really need, and do the benefits outweigh the added complexity?

In this and following episodes we’ll start with the most basic setup, and work our way up to more complex tools. If you get stuck with some of the more advanced tools, try taking a step back. Having less layers involved can make it more clear what’s going on.

For this first episode we’ll use the standalone cljs.jar. You can find it on ClojureScript’s Github site under “releases”.

curl -L https://github.com/clojure/clojurescript/releases/download/r1.9.36/cljs.jar > cljs.jar

JAR is short for “Java Archive”. Don’t worry though, no Java code has been used in the making of this video, but we will make use of the Java Runtime Environment, since that is needed to power Clojure.

A JAR is essentially a zip file containing code and other resources. This particular JAR is great to play around with, because it bundles both Clojure and ClojureScript, so it has everything to get you started.

For example, you can use it to start a Clojure REPL by invoking Clojure’s “main” class.

cp ~/Downloads/cljs.jar .
java -cp cljs.jar clojure.main

You can also use this JAR to execute a Clojure source file, by passing the filename as an extra argument.

You might wonder what that “-cp” stands for. It specifies the “classpath”, which is Java’s way of specifying locations to look for stuff. It’s like the “load path” or “library path” of other languages.

An entry in the classpath can be a directory path, but it can also be a JAR file, in which case Java will look for resources inside the JAR. Here we tell it that to find clojure.main, in needs to look inside cljs.jar

mkdir clj
echo '(doseq [i (range 1 7)] (println i "mississippi"))' > clj/mississippi.clj
java -cp cljs.jar clojure.main clj/mississippi.clj

To show you I have nothing up my sleaves, I’m starting from a new and empty directory. My ClojureScript code will go in the cljs directory. The entry point of this app will be the demo.main namespace. I will follow the Clojure naming conventions by putting the code for this namespace in a main.cljs file in the demo directory.

mkdir -p cljs/demo
emacsclient -n cljs/demo/main.cljs

(ns demo.main)

(js/alert "it's working")

Now two things need to happen: we need to compile this code to JavaScript, and we need to deliver it to the browser.

The ClojureScript compiler is written in Clojure, so you can run it simply by calling a Clojure function. Let’s write a script to compile our code.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options {:output-to "compiled.js"})

(cljs/build "cljs" compiler-options)

Let’s see what we have so far, I have a build script written in Clojure, which will compile the ClojureScript code to JavaScript. There’s a ClojureScript source file, which needs to be compiled, and then there’s the JAR containing Clojure and ClojureScript.

$ tree
.
├── clj
│   └── build.clj
├── cljs
│   └── demo
│       └── main.cljs
└── cljs.jar

Now we can use Clojure to run the build script, which kicks off the ClojureScript compilation. We set the Java classpath to be the cljs JAR. We tell Java to run clojure.main, and pass the name of our build script as an argument to clojure.main. Clojure will compile and run our script, which then compiles our ClojureScript code.

java -cp cljs.jar clojure.main clj/build.clj

Woah there’s a lot more going on now. There’s compiled.js, which we asked for, but under the out directory there’s all kinds of other stuff. ClojureScript didn’t just compile our one namespace, but also cljs.core, which contains the ClojureScript built-in functions and data structures. And cljs.core has its own dependencies, which also need to be compiled.

You might wonder what that goog stuff is. ClojureScript makes use the Google Closure Library (with an “s”, not with a “j”, I know it’s confusing). The Google Closure Library or GCL provides us with three things,

A collection of useful functions, a bit like a standard library
A module and dependency tracking system
An optimizing JavaScript-to-JavaScript compiler

In fact, ClojureScript namespaces are compiled to GCL modules. That way ClojureScript doesn’t need to implement its own dependency tracking, and it allows GCL to do advanced optimizations, like “dead code elimination”.

$ tree
.
├── clj
│   └── build.clj
├── cljs
│   └── demo
│       └── main.cljs
├── cljs.jar
├── compiled.js
└── out
    ├── cljs
    │   ├── core.cljs
    │   ├── core.js
    │   └── core.js.map
    ├── cljs_deps.js
    ├── demo
    │   ├── main.cljs
    │   ├── main.cljs.cache.json
    │   ├── main.js
    │   └── main.js.map
    └── goog
        ├── array
        │   └── array.js
        ├── asserts
        │   └── asserts.js
        ├── base.js
        ├── debug
        │   └── error.js
        ├── deps.js
        ├── dom
        │   └── nodetype.js
        ├── object
        │   └── object.js
        └── string
            ├── stringbuffer.js
            └── string.js

Now all that’s left for us to do is ship this to the browser. Create this small HTML file, and open your browser. Hmmm… not quite what we expected. The problem is that while ClojureScript compiled our code, it’s not getting loaded. Instead we get an error message saying goog is undefined.

Some tutorials will tell you now to add some extra stuff to your index.html so goog is no longer undefined, but that would mean adding stuff that you would have to strip out again when deploying to production. Instead there’s an easier solution.

<html>
  <!-- index.html -->
  <body>
    <script type="text/javascript" src="compiled.js"></script>
  </body>
</html>

If we tell ClojureScript what our main namespace is, then it will also inject some extra code to make sure this namespace and all its dependencies get loaded.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options {:output-to "compiled.js"
                       :main 'demo.main})

(cljs/build "cljs" compiler-options)

Recompile…

java -cp cljs.jar clojure.main clj/build.clj

And now it works

Now if you peak under the covers, you’ll notice a whole lot of extra stuff got added to the DOM. Because we compiled our code without any optimizations, each ClojureScript namespace and GCL module is loaded as a separate JavaScript file, plus ClojureScript injected some code to bootstrap our app.

Also notice that ClojureScript helpfully generated source maps, so we see ClojureScript code when using the browser’s debugger.

<html>
  <head></head>
  <body>
    <script type="text/javascript" src="compiled.js"></script>
    <script src="out/goog/base.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/deps.js"></script>
    <script src="out/cljs_deps.js"></script>
    <script>if (typeof goog != "undefined") { goog.require("demo.main"); } else { console.warn("ClojureScript could not load :main, did you forget to specify :asset-path?"); };</script>
    <script type="text/javascript" src="file:///tmp/out/goog/string/string.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/object/object.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/string/stringbuffer.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/debug/error.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/dom/nodetype.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/asserts/asserts.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/array/array.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/../cljs/core.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/../demo/deps.js"></script>
    <script type="text/javascript" src="file:///tmp/out/goog/../demo/core.js"></script>
  </body>
</html>

Let’s have another look at the build script. We passed the compiler two arguments: the path of the directory with source files to compile, and a map of “compiler options”. It’s worth it to focus on those compiler options for a bit, because even when using other tools like lein-cljsbuild or Figwheel you will still be dealing with these exact same compiler options.

There are too many options to cover them all, plus many of them you won’t have to care about. I’ll go over the ones that matter most, either for a development setup or for a production build. The ClojureScript wiki contains a complete reference of all recognized options.

You can see all intermediate JavaScript files in the out/ directory.

Building a ClojureScript project happens in two steps, first the ClojureScript compiler turns each ClojureScript namespace into a JavaScript module, then these modules are fed to the Google Closure Compiler, to produce a single, optimized, minimized JavaScript file.

I call these intermediate JavaScript files “modules” because that’s what they are to Google Closure. They follow specific conventions to take part in Google Closure’s module system.

$ find out -name *.js
out/demo/main.js
out/cljs/core.js
out/goog/object/object.js
out/goog/debug/error.js
out/goog/asserts/asserts.js
out/goog/array/array.js
out/goog/dom/nodetype.js
out/goog/string/string.js
out/goog/string/stringbuffer.js
out/goog/deps.js
out/goog/base.js

Here’s what our compiled demo.main namespace looks like.

Each module specifies its name by calling goog.provide. If a module depends on other modules, it can specifiy those dependencies with goog.require. Our namespace declaration didn’t specifiy dependencies, but we still depend on the core ClojureScript functions being available.

There’s also a magic comment at the end to point the browser at the source map.

// Compiled by ClojureScript 1.9.36 {}
goog.provide('demo.main');
goog.require('cljs.core');
alert("it's working!");

//# sourceMappingURL=main.js.map%

Now normally the Google Closure Compiler would take all these modules, analyze them, and output a single optimized JavaScript file, but we didn’t specify an optimization level, which means we got the default value, which is :none.

:none is the recommended setting for development. With this setting ClojureScript does not invoke the Google Closure Compiler. Instead it loads all the module files individually, which is why we got all those extra <script> tags added to the DOM.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options {:output-to "compiled.js"
                       :main 'demo.main
                       :optimizations :none})

(cljs/build "cljs" compiler-options)

Opting out of optimizations, as we’ve done so far, is useful for development because it allows ClojureScript to only recompile the namespaces that have changed. It is also a requirement for Figwheel to be able to do its automatic code reloading.

For production use we probably do want some level of optimization. The Google Closure Compiler gives us three options: whitespace, simple, and advanced. I’ll leave a link to this documentation in the show notes, it’s a good idea to read through it once when you have a moment.

levels of optimizations:

Any optimization level (besides none, or no optimization at all) will cause all modules to be packed together into a single JavaScript file. This means we no longer need to specify a main namespace, any top level forms in the compiled namespaces will be part of this single compiled JavaScript file.

The simplest option :whitespace, will only strip out comments and whitespace to reduce the file size, but that’s it. The resulting file is over a megabyte!

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options '{:output-to "compiled.js"
                        :optimizations :whitespace})

(cljs/build "cljs" compiler-options)

$ rm -rf compiled.js out
$ java -cp cljs.jar clojure.main clj/build.clj

-rw-rw-r-- 1 arne arne 1014K Jun 15 14:54 compiled.js

But something else changed as well, we no longer have source maps. ClojureScript only enables source maps when not using optimizations, but we can easily turn them on again.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options '{:output-to "compiled.js"
                        :source-map "compiled.js.map"
                        :optimizations :whitespace})

(cljs/build "cljs" compiler-options)

Oh yeah, we also need to specify the output directory or this won’t work. So far we’ve used the default output directory, called “out”, we can just stick to that. When compiling without optimizations you would have to set this output-dir to wherever your webserver looks for files to serve.

If you do that and you’re still having trouble loading these files, you might have to fiddle with the asset-path setting. I won’t show that here, you can find it documented in the ClojureScript wiki.

With optimizations enabled the output directory is only used for intermediate files, so it doesn’t matter much what you call it or where you put it. Another commonly used name for it is “target”.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options '{:output-to "compiled.js"
                        :source-map "compiled.js.map"
                        :output-dir "out"
                        :optimizations :whitespace})

(cljs/build "cljs" compiler-options)

One step up from :whitespace optimization is :simple. This is where the Google Closure Compiler starts doing some of its magic. It will rename variables to make them shorter, but only in ways that can not possibly change the meaning of the program. This mainly means it limits itself to variables inside functions. It’s somewhat limited, but still managed to shave a quarter off the final file size.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options '{:output-to "compiled.js"
                        :source-map "compiled.js.map"
                        :output-dir "out"
                        :optimizations :simple})

(cljs/build "cljs" compiler-options)

-rw-rw-r-- 1 arne arne 727K Jun 15 15:30 compiled.js

Finally we get to :advanced optimization. This is the real deal. GCL will not only do renaming of functions, variables, and constants across your complete code base, it will also remove any code that isn’t being used. This is known as “dead code elimination”. If you require a big library, but only end up using one or two functions, then this dramatically reduces the impact on your final file size.

This is also true for cljs.core. If you’re not using all the ClojureScript core functions, then you’re not paying for them.

;; clj/build.clj
(require '[cljs.build.api :as cljs])

(def compiler-options '{:output-to "compiled.js"
                        :source-map "compiled.js.map"
                        :optimizations :advanced})

(cljs/build "cljs" compiler-options)

And, no joke, our original megabyte has shrunk to just over a kilobyte. Factor in HTTP gzip compression, and you’re looking at 648 bytes over the wire. That’s not bad!

Keep in mind that advanced optimization is invasive. It will assume your code sticks to the Google Closure Library conventions. Only if that’s the case can it perform these advanced optimizations correctly.

$ ls -lh compiled.js
-rw-rw-r-- 1 arne arne 1.3K Jun 15 15:37 compiled.js

$ cat compiled.js | gzip | wc -c
648

If you’re only depending on other ClojureScript code, then that’s not an issue, but it’s a different story if you rely on third party JavaScript libraries.

Consuming third party JavaScript code is a big topic, but I’ll give you a few rules of thumb. First check http://cljsjs.org. Many popular libraries have been packaged for use with ClojureScript. These packages are carefully prepared so they contain everything needed to be compatible with ClojureScript and GCL.

If there’s no CLJSJS package, and you don’t care too much about file size, you can always choose for :optimizations :simple. It’s not as nice, but it always works.

If that’s not good enough for you, you will have to learn about externs. The ClojureScript wiki has documentation to help you get started. I’ll leave some links in the show notes.

In the next episode we’ll learn about browser repls, and we’ll look at two popular tools for compiling ClojureScript: lein-cljbuild and Figwheel.

The ClojureScript Compiler