2 Clojure REPLs
Clojure's Built-in REPL
Starting the REPL
A natural starting point is the REPL that comes with Clojure itself. This is
not the REPL you get with lein repl
though! To see Clojure's built-in REPL
all we need is the single JAR file which contains Clojure. If you've done any
Clojure before you probably already have it on your system, you can find it
with this command
find ~/.m2 -name clojure-1.*.jar
If that doesn't yield anything, then download Clojure from the official download page, extract the zip file, and you should find the JAR in there.
Now run this command in your terminal, and you should find yourself looking at a beautiful Clojure REPL.
java -jar /path/to/clojure-x.y.z.jar
For example
$ java -jar clojure-1.8.0.jar Clojure 1.8.0 user=> (+ 1 1) 2 user=>
If you're curious you can find the implementation in clojure.main/repl.
REPL Special Features
When this REPL starts we are in the user
namespace. Besides the usual stuff
from clojure.core
there are a few extra functions available to you in this
namespace. These are all very handy to have around during development, so
make sure you acquaint yourself with them.
From clojure.repl
source
- Show the source code of a function.
(source clojure.main/repl)
. apropos
- Given a string or regular expression, find all functions that match.
(apropos "some")
doc
- Show the documentation for a function, macro, or special form.
(doc map-indexed)
dir
- Shows a list of functions and other vars in a certain namespace.
(dir clojure.string)
pst
- Given an exception, print its message and stacktrace.
(try (/ 1 0) (catch Exception e (pst e)))
find-doc
- Searches all docstrings for a string or regex pattern.
(find-doc "join")
From clojure.java.javadoc
javadoc
- Opens the documentation for a Java class in your browser.
(javadoc java.util.regex.Pattern)
From clojure.pprint
pprint
- Show a "pretty" formatted representation of a value.
(pprint (mapv #(vec (range % 10)) (range 10)))
pp
- Pretty print the previous result, same as
(pprint *1)
There are also a few "magic" variables available
*1
,*2
,*3
- The result of the last, penultimate, and third to last evaluation
*e
- The last uncaught exception
To exit the REPL, type Ctrl-D.
Line Editing and History Support
When you try to use the arrow keys in the REPL you might end up with
gibberish ^[[A
. Clojure's REPL does not come with editing or history
support out of the box.
Most REPLs rely on the GNU Readline library for these features, and in the Java world there's JLine which emulates Readline.
There's a nifty little tool called rlwrap
, which can add Readline support
to programs that don't have it themselves. So if you want a Clojure REPL with
full editing and history support, you can start it like this:
rlwrap java -jar clojure-1.8.0.jar
Socket REPL
When a REPL "reads" and "prints", where does it read from, or print to? The answer is: standard input, and standard output.
Every process gets a byte stream that it can read from called "standard
input", also called STDIN
, System.in
in Java, or clojure.core/*in*
in
Clojure. If you start a program from a terminal, then this input stream
receives your keyboard's keystrokes.
Each process also starts with two output streams, "standard output" and
"standard error". Standard output (STDOUT
, System.out
,
clojure.core/*out*
) is where the REPL prints its results, and so they show
up in your terminal.
So what if you hook something else up as input and output stream? For instance, a network socket. Would that still work?
Turns out it does, and this is what Clojure's Socket REPL can do for you. The
name Socket REPL is actually misleading, because there is nothing
REPL-specific about this feature. Instead it's a general facility that causes
Clojure to start a TCP server, and wait for connections. Whenever a
connection is received, Clojure will connect *in*
and *out*
to the
network connection's input and output streams, and then pass control over to
a chosen function.
If that function happens to be clojure.main/repl
, well, then you've got
yourself a network REPL!
You tell Clojure to start a socket REPL using the clojure.server.repl
property, which you can configure like this:
java -cp clojure-1.8.0.jar -Dclojure.server.repl="{:port 5555 :accept clojure.core.server/repl}" clojure.main
When you run that command you will see a REPL appear, but that's not the
socket REPL we are talking about, it's just the regular REPL started by
clojure.main
. To see the socket REPL in action, open a second terminal, and
connect to it with Netcat (alternatively you can use telnet)
nc localhost 5555
You can even add rlwrap
to the mix to get a feature rich REPL experience.
The great thing about this socket REPL is that
- it works over the network, so you can connect to an app running somewhere else
- it doesn't require any source code changes, you can have an app that's already fully compiled and deployed, and give it REPL support just by adding a command line flag.
Don't expose this socket REPL to the internet though! Make sure your firewall blocks outside traffic to this port, then set up an SSH tunnel between the server and your local machine to connect to it.
Creating Your Own "Socket REPLs"
We told Clojure to call the clojure.core.server/repl
function each time it
receives a new connection. This is just a small variant of
clojure.main/repl
with some extra initialization. We can also tell Clojure
to use a different function.
Create a file called wrapple.clj
in the current directory.
(ns wrapple) (defn echo-time [] (println (.toString (java.util.Date.))))
Now start a "socket REPL" using this function:
java -cp clojure-1.8.0.jar:. -Dclojure.server.repl="{:port 5555 :accept wrapple/echo-time}" clojure.main
And you've got yourself a network service that echos the time!
$ nc localhost 5555 Thu Jul 07 16:22:17 CEST 2016
We can even make our own REPL loop, like this one, which will turn anything you give it into uppercase.
(ns wrapple (:require [clojure.string :as str])) (defn prompt-and-read [] (print "~> ") (flush) (read-line)) (defn uprepl [] (loop [input (prompt-and-read)] (-> input str/upper-case println) (recur (prompt-and-read))))
java -cp clojure-1.8.0.jar:. -Dclojure.server.repl="{:port 5555 :accept wrapple/uprepl}" clojure.main
$ nc localhost 5555 ~> isn't this awesome? ISN'T THIS AWESOME? ~>
nREPL
nREPL stands for "network REPL", and while this may sound pretty similar to a "socket REPL", they are completely different animals.
The REPLs we've seen so far are stream based, they read lines from an input stream, and write the result (and any output from side effects) to an output stream. This makes them conceptually simple, but tedious to communicate with programmatically.
nREPL seeks to fix this by being message-based, putting program-to-program first through a client-server architecture, where the client is your editor or IDE, and the server is the nREPL "REPL".
The client initiates the interaction by sending a message to the server, and as a response the server will send one or more messages back to the client.
The easiest way to start an nREPL server is either through Leiningen (lein
repl
) or Boot (boot repl
). In either case the first line of output will
contain a port number that an nREPL client can connect to.
Messages
A message might look something like this.
{:id "10" :op "eval" :code "(+ 1 1)" :ns "user"}
nREPL supports a number of "operations", the quintessential being "eval", which simply evaluates some code.
Conceptually a message is like a Clojure map, a set of key-value pairs. The
precise keys used depend on the type of message. In the case of eval
it
needs to know the code to evaluate, as well as the namespace to use for
context.
The response will look something like this.
{:id "10" :value "2"}
An editor or IDE could provide a Clojure REPL by implementing Read, Print, and Loop, and letting nREPL handle the Evaluate part, but it can do much more than just emulating a traditional REPL. nREPL can power "live" evaluation (sending forms from an editor buffer directly to a Clojure process), it can be used to look up documentation, inspect a running program, and much more.
nREPL provides a range of operations beyond "eval", together they can be used to offer rich Clojure editing support. Through "nREPL middleware" it is possible to add support for new operations.
nREPL supports these operations out of the box
eval
- Evaluate some code, returns the output value
interrupt
- Attempt to interrupt the current
eval
operation describe
- Returns a list of currently supported operations, as well version information
load-file
- (re-)load a complete source file
Sessions
nREPL comes with a built-in "session" middleware. This way you can have multiple REPLs backed by a single nREPL server. The session middleware adds three operations
ls-session
- list the current sessions
clone
- create a new session
close
- close a session
Each message will now carry a session-id. This becomes important when dealing
with the standard in- and output streams, *in*
, *out*
, and *err*
.
A single eval
operation might cause many lines of output, possibly
asynchronously, so they arrive after the eval
has completed. The session
middleware will intercept this output, and send it back to the client with
the same session-id as the original eval
message.
Here's an example interaction, evaluating the code
(dotimes [i 3] (Thread/sleep 1000) (println (str "==> " i)))
I'm using the message representation used by CIDER, with -->
indicating a
message to the server, and <--
being a message sent back to the client.
(--> ns "user" op "eval" session "10299efe-8f84-4b97-814c-21eb6860223b" code "(dotimes [i 3] (Thread/sleep 1000) (println (str \"==> \" i)))\n" file "*cider-repl localhost*" line 53 column 6 id "15" ) (<-- id "15" out "==> 0\n" session "10299efe-8f84-4b97-814c-21eb6860223b" ) (<-- id "15" out "==> 1\n" session "10299efe-8f84-4b97-814c-21eb6860223b" ) (<-- id "15" out "==> 2\n" session "10299efe-8f84-4b97-814c-21eb6860223b" ) (<-- id "15" ns "user" session "10299efe-8f84-4b97-814c-21eb6860223b" value "nil" ) (<-- id "15" session "10299efe-8f84-4b97-814c-21eb6860223b" status ("done") )
Custom Middleware
Through "nREPL middleware" you can even implement your own operations. Some notable "middlewares" are piggieback
, cider-nrepl
, and
nrepl-refactor
. Here's an example of the "eldoc" operation provided by CIDER-nREPL.
(--> op "eldoc" session "ff822558-e885-49ed-8cf6-b5331bc8553b" ns "user" symbol "str" id "10" ) (<-- docstring "With no args, returns the empty string. With one arg x, returns\n x.toString(). (str nil) returns the empty string. With more than\n one arg, returns the concatenation of the str values of the args." eldoc (nil ("x") ("x" "&" "ys")) id "10" name "str" ns "clojure.core" session "ff822558-e885-49ed-8cf6-b5331bc8553b" status ("done") type "function" )
Writing your own middleware isn't hard. Let's make a middleware that adds a "classpath" operation, which returns the current Java classpath. It's trivial but perhaps not altogether useless.
(ns classpath-nrepl.middleware (:require [nrepl [middleware :refer [set-descriptor!]] [transport :as transport]])) (defn wrap-classpath [handler] (fn [{:keys [id op transport] :as request}] (if (= op "classpath") (transport/send transport {:id id :classpath (->> (java.lang.ClassLoader/getSystemClassLoader) .getURLs (map str))}) (handler request)))) (set-descriptor! #'wrap-classpath {:requires #{} :expects #{} :handles {"classpath" {:doc "Return the Java classpath"}}})
At the heart of any nREPL server is the "handler", a function which handles a single incoming message. A middleware is a function which takes the existing handler function and "wraps" it, returning a new handler function.
Our new handler will look at each incoming request, and if the operation is anything but classpath, it simply calls the old handler. It's basically saying, "I don't care about this request, you handle it!"
When it receives a classpath
message however, then it constructs a
response, and sends it back to the client.
As part of the incoming request the handler received a "transport" object, which it needs to use to reply to the client. This is how nREPL achieves asynchronous operation. A handler can send several messages back to the client based on a single incoming message, possibly much later or from another thread. It only needs to make sure to use the transport and id of the original message.
To see this middleware in action, make sure you have nrepl
in your
dependencies, and add some :repl-options
to your Leiningen project.clj
to
insert the middleware into the "middleware stack".
(defproject middleware-test "0.1.0-SNAPSHOT" :dependencies [[org.clojure/clojure "1.8.0"] [nrepl "0.6.0"]] :repl-options {:nrepl-middleware [classpath-nrepl.middleware/wrap-classpath]})
If you're using Boot then you can insert the middleware like this in your build.boot
(require 'boot.repl) (swap! boot.repl/*default-middleware* conj 'classpath-nrepl.middleware/wrap-classpath)
If you're starting your own nREPL server directly instead of using
lein repl
or boot repl
, then you can pass the middleware as an argument
to the "default handler".
(require '[nrepl.server :as nrepl]) (require '[classpath-nrepl.middleware :refer [wrap-classpath]]) (nrepl/start-server :handler (server/default-handler #'wrap-classpath))
Now start a REPL with lein repl
, and note the port number that Leiningen
prints in the first line of output, for example nREPL server started on port
41065 on host 127.0.0.1
. Now you can connect to it, either from the same
process, or from a completely different REPL.
user> (require '[nrepl.core :as repl]) nil user> (def conn (repl/connect :port 41065)) #'user/conn user> (repl/message (repl/client conn 1000) {:id 1 :op "classpath"}) ({:classpath ["file:/home/arne/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar" ,,,], :id 1})
Here's what the interaction looks like:
(--> op "classpath" session "ff822558-e885-49ed-8cf6-b5331bc8553b" id "1" ) (<-- classpath ("/home/arne/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar" ...) id "1" session "ff822558-e885-49ed-8cf6-b5331bc8553b" status ("done") )
- Notable middleware
The CIDER-nREPL README has a list of included middlewares and their supported operations.
Many more operations are added by refactor-nrepl.
Piggieback allows using nREPL with ClojureScript, see the Piggieback section under ClojureScript REPLs.
Protocol
Messages between an nREPL client and server are sent over the wire using "BEncode" (pronounced B-encode). BEncode was originally developed for use in BitTorrent, it's a binary format that supports integers, strings, lists, and dictionaries.
While BEncode is a binary protocol, not intended for human consumption, it is kind of readable if you know what you're looking for.
Take this Clojure Map
{:id 1 :op "eval" :code "(+ 1 1)\n"}
Here's the BEncoding of it
"d4:code8:(+ 1 1)\n2:idi1e2:op4:evale"
The way to read this is
d
dictionary4:code
a 4 byte string:"code"
8:(+ 1 1)\n
an 8 byte string:"(+ 1 1)\n"
2:id
a 2 byte string:"id"
i1e
an integer,1
- …
e
end of dictionary
Clients
It should be clear by now that communicating with nREPL is best done through a dedicated client library. The nREPL package contains both the nREPL server and a client implementation that you can use from Clojure.
CIDER-nREPL contains an nREPL client implemented in Emacs Lisp. If you want to experiment with nREPL from Emacs you can try this snippet
(nrepl-send-request '("op" "classpath") (lambda (&more) ) (car (cider-connections)))
And inspect the
*nrepl-messages*
buffer- OCaml: grenchman contains an nREPL client
- Objective-C: LVReplClient
- Python: python-nrepl-client
- Ruby: experimental client
- Node.js: nrepl-client on npm
The nREPL manual lists editors and tools that support nREPL.
ClojureScript clients for nREPL
Advanced, you can safely skip this section.
This section is a stub, you can help by expanding it.
This section is about connecting to an nREPL server from a ClojureScript client. If instead you want your nREPL server to use a ClojureScript environment to do evaluation, see the section on Piggieback.
By default nREPL communicates over TCP sockets, using Bencode as its data format. Raw TCP sockets are not available from a browser, but they are available in Node.js. You can use the nrepl-client package on NPM to connect from Node.js to an nREPL server.
The nREPL transport is pluggable, so a more browser friendly alternative would be to use HTTP over JSON, which is what Drawbridge provides. It is implemented as a Ring handler, so you need to use it in conjunction with a Ring server adapter (Jetty, Http-kit, Aleph, …)
Drawbridge contains an nREPL client that uses the same transport, but that one is for Clojure, not ClojureScript. For a ClojureScript client you can use drawbridge-cljs.