Lambda Island

Learn how to run tests with Figwheel, PhantomJS, Node, doo, and Karma.

Running Clojure tests once they are written is pretty straightforward, but for ClojureScript there are a few more things to consider. Your code needs to be compiled first, and then needs to run on the JavaScript engine of your choice. In this episode you’ll learn how to set up a ClojureScript project from scratch using Figwheel, how to run tests from the REPL, how to create test runners for PhantomJS and Node, and how to use doo and Karma to run your tests on a wide variety of browsers simultaneously.

Transcript

Writing tests for ClojureScript isn’t all that different from writing tests for Clojure. There are a few small differences in what the test library provides, but for the most part your code will look exactly the same.

When it comes to running ClojureScript tests however, there’s a bit more to be said, because you need to be concerned both with the compiler, and with a JavaScript runtime environment.

Let’s get a feel for things by setting up a small ClojureScript project. To show you I have nothing up my sleave I’m going to start with a completely new and empty project, generated with lein new.

$ lein new ep33testcljs

First open project.clj to set up dependencies and configure a ClojureScript build. The alphas of Clojure 1.9 are pretty solid so I tend to just use whatever is the most recent one. Then also get the latest ClojureScript. There have been some Compiler performance improvements lately so it’s good to be up to date.

My only concession to minimalism is that I’m using Figwheel, because really why wouldn’t you? This is by far the fastest and easiest way to get a ClojureScript REPL going, which is what I want as a starting point.

You also need to set up a ClojureScript build. This is all pretty standard stuff. Tell Figwheel it should manage this build, tell it to find its source files in the “src” and “test” directories, and finally configure the ClojureScript compiler.

:main is the top level namespace, all parts of your app need to be explicitly or implicitly required from this namespace, otherwise they won’t be included in the compiled result.

To be compatible with Figwheel, optimizations needs to be set to :none, this means each namespace will get compiled to an individual JavaScript file under :output-dir. Notice that they’re being output under resources/public, this way they are accessible from the browser.

The browser still needs to be able to find them though, with :asset-path you configure the url-prefix needed to correctly locate a compiled file. Finally you set the :output-to file, which is just a small stub in this case that loads up all the actual compiled namespaces.

(defproject ep33testcljs "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.9.0-alpha17"]
                 [org.clojure/clojurescript "1.9.671"]]

  :plugins [[lein-figwheel "0.5.11"]]

  :cljsbuild {:builds
              [{:id "dev"
                :figwheel true
                :source-paths ["src" "test"]
                :compiler {:main testcljs.core
                           :optimizations :none
                           :output-dir "resources/public/js/out"
                           :asset-path "js/out"
                           :output-to "resources/public/js/app.js"}}]})

Let’s get rid of the Clojure files that leiningen created, and instead just create a single ClojureScript namespace so Figwheel has something to chew on.

rm -rf src/* test/*
mkdir src/testcljs
echo '(ns testcljs.core)'> src/testcljs/core.cljs

Ok, now start lein figwheel. This takes a moment, I’m just going to speed that way up. There.

Now open your browser at http://localhost:3449. It should say “Figwheel Server: Resource not found. Keep on figwheelin’”. Which I certainly intend to, but first Figwheel is going to need a little HTML file that loads up the ClojureScript, so let’s add that.

This is under resources/public, you create a new file called index.html. This is just the absolute minimum of HTML5 boilerplate, plus a script tag to load up the app.

Now refresh the browser. You’ll get a blank page, that’s expected, but in the browser console you can see that Figwheel is getting fruity already. And indeed if you switch back to the terminal you’ll see that the Figwheel REPL has appeared.

<!doctype html>
<html lang="en">
  <head>
    <meta charset='utf-8'>
  </head>
  <body>
    <script src="js/app.js"></script>
  </body>
</html>

That’s all I ever wanted, a REPL. Not that I intend to actually type anything into that REPL like some troglodyte. No, of course I want to set up my trusty editor so I can work from a buffer. I use CIDER for a lot of stuff, but for the sake of simplicity I’m going to use inf-clojure. If you’re using Emacs you should be able to follow along, there’s also a separate episode on inf-clojure if you’re feeling a bit fuzzy about what its all about. For other environments the Figwheel Wiki contains documentation on how to get a REPL connected to your editor. There are instructions there for Vim, Cursive, Lighttable, as well as generic instructions for using Figwheel on top of nREPL.

For me I’m just launching inf-clojure, telling it to run lein figwheel to start the REPL. Again this takes a while, in the meantime I’m going to enable inf-clojure-minor-mode in this source buffer.

Cutting some fruit… and… there’s the REPL prompt.

M-: (inf-clojure "lein figwheel")

With that I can type some code in the buffer, hit C-x C-e, and see the result pop up in the REPL buffer.

Ok now let’s write some code, I’ll be generating mice emoticons like it’s 1999. Some will have short tails, some will be longer, but they are all equally adorable and deserving of love.

Let’s give that a try. Ok, cool.

(ns testcljs.core)

(defn mouse [n]
  (apply str "<:3)" (repeat n "~")))

(mouse 5)
;;=> "<:3~~~~~"

And now we get to the fun part, the tests. Create a testcljs.core-test namespace inside the test directory. Require the code you want to test, as well as clojure.test. This is just an alias for cljs.test, a ClojureScript port of clojure.test. You could also require cljs.test directly. I like to do it this way though for consitency. The added benefit is that this will work seamlessly when writing cross-platform CLJC files.

You do have to refer the various test functions explicitly, since ClojureScript doesn’t have a :refer :all. Most of these are actually macros, and in older versions of ClojureScript you had to use :require-macros or :refer-macros, but in this case it also works with a regular refer. This is because cljs/test.cljs already requires its own macro macro namespace. Thanks to an improvement in ClojureScript that landed about a year ago this means downstream callers can refer to the macros the same way as they would for regular functions.

(ns testcljs.core-test
  (:require [testcljs.core :refer [mouse]]
            [clojure.test :refer [deftest testing is are async run-tests]]))

The test itself is straightforward. Use deftest, followed by a name, I also like to use testing to add a bit of a description. I’m using are which I described in the previous episode.

Just as in Clojure this deftest will create a function which runs the test. You can also use (run-tests), which runs all tests in the given namespaces, defaulting to the current namespace, or you can run all defined tests with (run-all-tests). This all isn’t news. Everything we’ve done so far works exactly the same way in Clojure.

(deftest mouse-test
  (testing "it has a cute tail"
    (are [n m] (= (mouse n) m)
      0 "<:3)"
      1 "<:3)~"
      2 "<:3)~~"
      10 "<:3)~~~~~~~~~~")))

(comment
  (mouse-test)
  (run-tests))

I do have a bug in my rodent, so let me fix that. The mouse is missing its bum. Ok, now the tests run without errors.

Now suppose you want to run these tests from the command line. For Clojure tests you could just use lein test. In project.clj you configure one or more :test-paths, by default that’s just ["test"]. lein test will load all namespaces it finds in there, and then call run-tests on all of them.

For ClojureScript things are a little different. You first need to create a build that includes all your tests. The result is a single compiled JavaScript file, which you then pass to a JavaScript runtime, such as your browser.

To decide what goes into a build, ClojureScript takes the :main namespace, and then starts adding required namespaces from there. To make sure all test namespaces get added, create a new test-runner namespace, which requires all the different test namespaces.

So this namespace loads the one test namespace we have so far, as well as the run-all-tests macro.

Make sure output is printed on the browser console with enable-console-print!, so you can see the test results. Now that the test namespaces are included in the build, call run-all-tests at the top level, so they run whenever this code gets loaded.

(ns testcljs.test-runner
  (:require [testcljs.core-test]
            [clojure.test :refer [run-all-tests]]))

(enable-console-print!)

(run-all-tests)

Finally update project.clj to use this test-runner as the main namespace.

(defproject
  ,,,
  :cljsbuild {:builds
              [{:id "dev"
                ,,,
                :compiler {:main testcljs.test-runner
                           ,,,}}]})

Because you change the build configuration to use a different :main namespace, you have to tell Figwheel to reload and restart, using (reload-config)

;; on the Figwheel REPL
dev:cljs.user=> (reload-config)
Figwheel: Stopped watching build - dev
Figwheel: Reloading build config information
Figwheel: Cleaning build - dev
Figwheel: Watching build - dev
Figwheel: Cleaning build - dev
Compiling "resources/public/js/app.js" from ["src" "test"]...
Successfully compiled "resources/public/js/app.js" in 5.835 seconds.

Refresh your browser, open the JavaScript console, and you should see the test results. And thanks to Figwheel any code change will cause them to reload and run again, so this already works alright as a test runner.

The next step is running the tests from the command line. This is especially useful for setting up Continuous Integration, say with Travis or Circle CI.

For this first add the lein-cljsbuild Leiningen plugin to your project. This allows you to build your code without Figwheel.

(defproject
  ,,,

  :plugins [[lein-figwheel "0.5.11"]
            [lein-cljsbuild "1.1.6"]]
  ,,,)

So far I’ve been using this single dev build to run the tests, but it might be better to introduce a separate test build.

Change the :main namespace of the dev build back to testcljs.core. Now add a test build, which uses testcljs.test-runner as the main namespace. You won’t be using Figwheel in this case, so remove the :figwheel flag.

For the dev build I used :optimizations :none, the lowest level of optimizations. With this setting each namespace gets turned into a separate JavaScript file, meaning the JavaScript environment needs to know how to resolve and load namespaces. This means a HTTP server needs to be involved, to serve up the individual JavaScript files, that’s also why you need that :asset-path setting.

For the tests you can go one level up and :optimizations :whitespace. The only difference is that with this setting, all code gets concatenated into a single file. Each build needs to have a separate :output-dir, so let’s just add test to this path, and finally choose a different name for the final build artifact.

  :cljsbuild {:builds
              [{:id "dev"
                :figwheel true
                :source-paths ["src" "test"]
                :compiler {:main testcljs.core
                           :optimizations :none
                           :output-dir "resources/public/js/out"
                           :asset-path "js/out"
                           :output-to "resources/public/js/app.js"}}

               {:id "test"
                :source-paths ["src" "test"]
                :compiler {:main testcljs.test-runner
                           :optimizations :whitespace
                           :output-dir "resources/public/js/out/test"
                           :output-to "resources/public/js/test.js"}}]}

Ok, save that, and now run lein cljsbuild to create test.js. Give it a moment to do its magic. Ok, now there should be a single test.js file that contains all code and tests.

$ lein cljsbuild once test
Compiling ClojureScript...
Compiling "resources/public/js/test.js" from ["src" "test"]...
Successfully compiled "resources/public/js/test.js" in 8.788 seconds.

You could still just load that in your browser by updating index.html, but let’s not do that. Instead I’ll run it on the command line with PhantomJS. Phantom is a complete WebKit-based browser, but it’s “headless”, meaning it doesn’t have a GUI.

The Phantom site has installation instructions, for what follows I’m assuming you have PhantomJS installed and working on your system.

$ phantomjs --version
2.1.1

Now you just pass Phantom the path to test.js, and there you go! Notice though that the process doesn’t exit, it’s just hanging there until you hit Ctrl-C. We’ll have to do something about that.

$ phantomjs resources/public/js/test.js

Testing cljs.user

Testing cljs.core

Testing clojure.string

Testing cljs.pprint

Testing cljs.test

Testing clojure.template

Testing testcljs.core

Testing testcljs.core-test

Testing testcljs.test-runner

Ran 1 tests containing 4 assertions.
0 failures, 0 errors.

If for some reason this didn’t work, then try deleting resources/public/js, and then recompile. Sometimes lein cljsbuild gets confused due to old build artifacts lying around.

$ rm -rf resources/public/js

To get phantom to exit, you call phantom.exit, optionally passing it an exit code for the process. Let’s check though if that phantom object is present first, so that this code doesn’t cause problems on other platforms. ClojureScript has an exists? predicate that lets you check if an identifier is defined.

(ns testcljs.test-runner
  (:require [testcljs.core-test]
            [clojure.test :refer [run-all-tests]]))

(enable-console-print!)

(run-all-tests)

(when (exists? js/phantom)
  (js/phantom.exit))

This works, but it has two problems: the first is that when a test fails, Phantom should return a non-zero exit status. Otherwise your CI will think that all is green when it really isn’t. So we’ll have to get access to the test results in some way.

The second problem is that when using asynchronous tests, this will kill the process before the tests have had a chance to finish.

We can solve both of these problems by plugging into the cljs.test/report multimethod. This method lets you monitor the test suite’s lifecycle, it gets called when a test starts or finishes, when an assertion succeeds or fails, and it gets called all the way at the end after the test run has finished.

This method dispatches on a two element vector, the first element is the type of test reporter being used for this test run. Normally that will be :cljs.test/default, you can abbreviate that using a double colon and the namespace alias.

The second element is the type of event being handled. This can be something like :begin-test-ns, or :fail, we’re interested in :end-run-tests.

The argument given to report is a map, containing :type, :fail, :error, :pass, and :test. I’m going to mimic what lein test does, but adding together the number of tests that failed or raised an error, and use that as the exit code. The exit code should be zero if all is good, and non-zero otherwise, so that works.

(ns testcljs.test-runner
  (:require [testcljs.core-test]
            [clojure.test :as t :refer [run-all-tests]]))

(enable-console-print!)

(defmethod t/report [::t/default :end-run-tests] [m]
  ;;{:type :end-run-tests, :fail 0, :error 0, :pass 4, :test 1}
  (when (exists? js/phantom)
    (js/phantom.exit (+ (:fail m) (:error m)))))

(run-all-tests)

Cool, there you have your PhantomJS test runners in a handful of lines of ClojureScript.

If you like to use Node instead then that’s also possible. Beware that Node is just a JavaScript runtime, it does not contain a browser, and so doesn’t provide a DOM or other browser-specific APIs. It’ll depend on your project if this is an issue for you. Luckily for us we can generate cute mice emoticons on Node just fine.

Your first attempt would probably be to run that same test.js on Node, which does not work. Node is different enough from other environments that ClojureScript has a custom mode for outputting code compatible with Node.

$ node resources/public/js/test.js
/home/arne/ep33testcljs/resources/public/js/test.js:79
"String))throw 1;for(const a of[2,3]){if(a\x3d\x3d2)continue;function "+"f(z\x3d{a}){let a\x3d0;return z.a}{function f(){return 0;}}return f()"+"\x3d\x3d3}";return evalCheck('(()\x3d\x3e{"use strict";'+es6fullTest+"})()")});addNewerLanguageTranspilationCheck("es6-impl",function(){return true});addNewerLanguageTranspilationCheck("es7",function(){return evalCheck("2 ** 2 \x3d\x3d 4")});addNewerLanguageTranspilationCheck("es8",function(){return evalCheck("async () \x3d\x3e 1, true")});return requiresTranspilation};goog.provide("goog.string");goog.provide("goog.string.Unicode");goog.define("goog.string.DETECT_DOUBLE_ESCAPING",false);goog.define("goog.string.FORCE_NON_DOM_HTML_UNESCAPING",false);goog.string.Unicode={NBSP:" "};goog.string.startsWith=function(str,prefix){return str.lastIndexOf(prefix,0)==0};goog.string.endsWith=function(str,suffix){var l=str.length-suffix.length;return l>=0&&str.indexOf(suffix,l)==l};


TypeError: Cannot set property 'Unicode' of undefined
    at Object.<anonymous> (/home/arne/ep33testcljs/resources/public/js/test.js:79:720)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:393:7)
    at startup (bootstrap_node.js:150:9)
    at bootstrap_node.js:508:3

In the ClojureScript compiler options add :target :nodejs. Also change the optimizations level to :simple, this is the lowest level of optimizations that work with Node.

:compiler {:main testcljs.test-runner
           :optimizations :simple
           :target :nodejs
           ,,,}

Now clean up the compiled JavaScript again, to force cljsbuild to do a full rebuild. Now you can run the tests with Node.

$ rm -rf resources/public/js
$ lein cljsbuild once test
Compiling ClojureScript...
Compiling "resources/public/js/test.js" from ["src" "test"]...
Successfully compiled "resources/public/js/test.js" in 13.249 seconds.
$ node resources/public/js/test.js

Testing cljs.user

Testing cljs.core

Testing clojure.string

Testing cljs.pprint

Testing cljs.test

Testing clojure.template

Testing testcljs.core

Testing testcljs.core-test

Testing testcljs.test-runner

Ran 1 tests containing 4 assertions.
0 failures, 0 errors.

In this case Node does exit immediately, not because of our own exit code, since that only works for Phantom, but because Node is not a browser, it just runs the script you give it and then exits. This does mean that once again the exit code doesn’t reflect the status of the test run.

That’s easy enough to fix, the equivalent of phantom.exit is called process.exit, but it does highlight that you have to do extra work for every environment you’re targeting.

(defmethod t/report [::t/default :end-run-tests] [m]
  (when-let [exit (cond (exists? js/phantom) js/phantom.exit
                      (exists? js/process) js/process.exit)]
    (exit (+ (:fail m) (:error m)))))

Luckily there’s a project calls doo which makes running tests on different environments a whole lot easier.

Add lein-doo as a plugin, and change the test build back so it’s no longer targeting node, and it has :optimizations :whitespace

:plugins [[,,,]
          [lein-doo "0.1.7"]]


:cljsbuild {:builds
            [,,,

             {:id "test"
              :source-paths ["src" "test"]
              :compiler {:main testcljs.test-runner
                         :optimizations :whitespace
                         :output-dir "resources/public/js/out/test"
                         :output-to "resources/public/js/test.js"}}]}

In order for doo to do its magic you need to use some custom macros in the test runner. doo-tests in the doo.runner namespace replaces run-tests, and doo-all-tests replaces run-all-tests.

(ns testcljs.test-runner
  (:require [testcljs.core-test]
            [doo.runner :refer-macros [doo-tests doo-all-tests]]))

(doo-tests 'testcljs.core-test)

Now you can invoke doo from the command line with with lein doo. When running lein doo, you pass it the runner it should use, and the name of the build. So for instance lein doo phantom test uses PhantomJS to run the build named test. A cool thing is after running the tests, doo will start watching you code for changes. As soon as you change a file, it re-runs the tests, so you get a nice and speedy feedback loop.

$ lein doo phantom test

Doo supports an impressive list of environments, including Chrome, Firefox, Safari, PhantomJS, Slimer, and the JavaScript engines that come bundled with Java: Rhino and Nashorn.

$ lein doo nashorn test

To be able to support all these different browsers, doo leverages Karma, a fairly advanced JavaScript test runner that came out of the Angular project. This does mean that to run for instance on Firefox or Chrome, you need to first install Karma, using npm.

To begin with you need the karma-cli executable, which doo recommends you install globally with the -g flag.

$ npm install -g karma-cli

You also need Karma itself, as well as a couple of plugins. One for each browser you want to test with, and one to support ClojureScript tests. These you can install inside your project directory. They will get install under node_modules, which you don’t have to check into source control, although you could provide a package.json so others can more easily install these.

npm install karma karma-firefox-launcher karma-cljs-test

Now when you do lein doo firefox test, it will compile the code, start the Karma server, and open Karma’s url in Firefox: http://localhost:9876. Now each time you change some code, it re-runs all your tests in Firefox, and shows you the result.

I did notice that it doesn’t always run the tests when it starts up, but as soon as you touch a file it does re-run them.

The really cool thing about Karma is that you can open this URL in as many browsers as you like. So here I opened it in Firefox, Chrome, and on my phone. Now when I change some code the tests will run on each of these browsers, and you get the see the results for each of them. Pretty cool, right?

Show Notes

browse source code

Useful links and further reading

ClojureScript Wiki: Compiler Options I refer to this page all the time. Definitely one for the bookmarks.
Figwheel Wiki For when you need help getting things hooked up to your editor.
doo The README has elaborate instructions on how to set things up to test against various browser backends.

Cheat sheet

Compile ClojureScript once

lein cljsbuild <build-id> once

Compile ClojureScript and watch for changed

lein cljsbuild <build-id> watch

Run doo

lein doo <environment> <build-id>

Extras

Something I didn’t mention in the episode is that lein-cljsbuild allows you to configure your test runners, so you can invoke them with lein cljsbuild test <runner>. This will also make sure your code is compiled before starting the test run.

It doesn’t do anything you can’t do directly from the command line, and it doesn’t support watching tests for changes, but it can be handy when integrating into a Leiningen based work.

(defproject lein-cljsbuild-example "1.2.3"
  :plugins [[lein-cljsbuild "1.1.6"]]
  :cljsbuild {:test-commands
              ;; run with `lein cljsbuild test my-test`
              {"my-test" ["phantomjs" "resources/public/js/test.js"]}}
  :hooks [leiningen.cljsbuild])

Running ClojureScript Tests