Lambda Island

In this second installment of our beginner friendly introduction to Ring the API goes from being read-only to being read-write. You also learn about juxt, slurp, and how to parse an incoming request body.

Transcript

<– 001 –>

In the previous episode we started building an API for an address book application using Ring. This is what the code looks like so far. We use an atom containing a map as our database of contacts. The Ring handler so far only handles GET requests to the root path. We’re using the Ring Jetty adapter to talk to the outside world.

(ns rolodex.core
  (:require [ring.adapter.jetty :refer [run-jetty]]))

(defn random-uuid []
  (java.util.UUID/randomUUID))

(def contacts (atom (let [id (random-uuid)]
                      {id {:id id
                           :full-name "Trish Turtle"
                           :skills ["LISP" "Lambda Calculus"]}})))

(defn handler "request -> response"
  [req]
  (if (and (= (:request-method req) :get) (= (:uri req) "/"))
    {:status 200
     :headers {"Content-Type" "application/edn; charset=UTF-8"}
     :body (prn-str (vals @contacts))}))

(defonce server
  (run-jetty #'handler {:port 3000
                        :join? false}))

<– 002 –>

Looking at project.clj you can I see I’ve kept the dependencies to a minimum. There’s just Clojure itself and then the Ring Jetty adapter. In real a world project you should pull in some libraries containing middleware and routing functionality, and we’ll cover some of those later in the series, but first I want you to grok the concepts of Ring, it will help a lot once you start combining these third party pieces.

(defproject rolodex "0.1.0"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [ring/ring-jetty-adapter "1.5.0"]])

If you want to follow along, which you should, then grab the Rolodex repository from GitHub. Just do a git clone, then cd into it, and then checkout ring-part-1 branch. This is the code we ended up with at the end of the previous episode.

git clone https://github.com/lambdaisland/rolodex.git
cd rolodex
git checkout ring-part-1

Building an API is a lot more fun if there’s an app that can use it, so I created this Reagent app that you can use to test your API. It’s also on Github in the rolodex-gui repository. Just clone it and then start the server with lein run, now you can open it on http://localhost:3449.

git clone https://github.com/lambdaisland/rolodex-gui.git
cd rolodex-gui
lein run

Another piece of supporting code that you get for free is this contact generator. It contains a list of names, and turns them into contact maps, with an id, twitter handle, and email address. It’s just something to have some data to play around with. If you cloned the repo from GitHub, then you can find the generator in the origin/ring-part-2 branch. Add it to your current branch, by doing a git checkout, adding the specific path you want to check out. Now generator.clj has been added to the git staging area as a new file, and you can commit it. This is a neat trick for when you want to copy a file from one branch to another.

git checkout ring-part-2 -- src/rolodex/generator.clj
git commit

Alright, let’s get coding. I have the GUI running in a terminal. For the API I’ll start CIDER with cider-jack-in (C-c M-j) so I can easily tinker with it from Emacs. You don’t have to use Emacs of course, but I still recommend some kind of interactive setup so you can easily redefine functions, otherwise you’ll be starting and stopping the server a lot.

Evaluate the rolodex.core namespace (that’s C-c C-k) to start the server.

To use the contact generator first add it to the namespace declaration, aliased to gen.

(ns rolodex.core
  (:require [clojure.edn :as edn]
            [ring.adapter.jetty :refer [run-jetty]]
            [rolodex.generator :as gen]))

The generator’s main function, contact, takes a name and returns a contact map, for example:

(gen/contact "Master Sammy")
;;=> {:id #uuid "f36392f5-9b8e-4b1a-ab1d-6935edf43841", :full-name "Master Sammy", :email "mastersammy@hotmail.com", :twitter "@sammy91"}

There’s also a predefined list of names, although of course you can come up with your own wacky address book. With these two in hand we can map the contact function over the names to get an initial list of contacts.

(map gen/contact gen/names)

To make it easy to find a contact by its unique id we use a map with ids as keys and the contact map as the associated value. Getting data into the right shape is something Clojure excels at, all we need is a dash of functional programming.

The first ingredient we’ll use is the juxt function. “juxt” is short for “juxtapose”, which just means “putting things side by side”. juxt is a “function combinator”, it takes a number of functions, and combines them to create a new function.

Calling this function that juxt created is like calling each of these functions with the same arguments, and collecting the results in a vector.

Let me demonstrate that with the inc (increment) and dec (decrement) functions. If you increment 5, you get 6, and when you decrement 5 you get 4. When you pass inc and dec to juxt you get a new function which is the juxtaposition of inc and dec. Calling that function with the argument 5 yields both results in a vector.

(inc 5)              ;;=> 6
(dec 5)              ;;=> 4
(juxt inc dec)       ;;=> #function[clojure.core/juxt/fn--4742]
((juxt inc dec) 5)   ;;=> [6 4]

Keywords are also a kind of function, so we can use these with juxt as well, which is great for pulling values out of a bunch of maps.

Say you have this list of contacts, if you’re only interested in the names, then you can map using the :full-name keyword, the same goes for twitter handles. If you want to extract both, then you use juxt.

(def contacts [{:id #uuid "352610a7-3e74-4c83-9d99-8f4c741cdaa0",
                :full-name "Trish Turtle",
                :email "trish.turtle@hotmail.com",
                :twitter "@horse_trish"}
               {:id #uuid "a6c5315e-ff5d-493e-bbe7-dbfede911a0a",
                :full-name "Master Sammy",
                :email "mastersammy@gmail.com",
                :twitter "@master_ebooks"}
               {:id #uuid "20e9efae-cede-4e1a-b88c-95d6518c8805",
                :full-name "Brock Samson",
                :email "brocksamson@gmail.com",
                :twitter "@_samson_"}])

(map :full-name contacts)
;;=> ("Trish Turtle" "Master Sammy" "Brock Samson")

(map :twitter contacts)
;;=> ("@horse_trish" "@master_ebooks" "@_samson_")

(map (juxt :full-name :twitter) contacts)
;;=> (["Trish Turtle" "@horse_trish"] ["Master Sammy" "@master_ebooks"] ["Brock Samson" "@_samson_"])

If you want to get a value from a map, but you also want to hang on to the map itself, then you can combine juxt with identity. The identity function always returns its argument, so if we (juxt :id identity), then we get a vector containing the contact id, and the contact itself.

If you look at the first result of mapping over the contacts you can see it’s a pair of the id and the contact itself. These are exactly the key-value pairs we want in our contacts map, all we need to do is put them “into” a map with the into function. Et voila, a map of contacts indexed by their id.

(map (juxt :id identity) contacts)
;; => ([#uuid "352610a7-3e74-4c83-9d99-8f4c741cdaa0" {:id #uuid "352610a7-3e74-4c83-9d99-8f4c741cdaa0", :full-name "Trish Turtle" ,,,}] ,,,)

(into {} (map (juxt :id identity) contacts))

So let’s use that as our initial contacts database. We no longer need this random-uuid, the generator will create ids for us. Generate contacts, then map over them extracting the id and the contact itself. Add them all to a map with into, and then wrap the result in an atom again since this will the holder of our mutable application state.

(def contacts (atom (into {} (map (juxt :id identity)
                                  (map gen/contact gen/names)))))

Now that we have some dummy data let’s have a look at that GUI again. The only operation our API supports so far is to do a GET to the root of the API to retrieve the list of contacts. That seems to work. You can click in the list to see the contact details.

The other operations work on individual contacts, hover over them if you’re not sure which is which. POST creates a new contact, GET fetches a single contact based on the given id, PUT updates it, and DELETE deletes. Since none of these are implemented yet they all return 404, as you can see at the bottom.

This is what our handler looks like so far. We get a request map, and check its request method and uri, and if it’s a GET to the root then we return the list of contacts.

The next operation we want to implement is a POST request at the same URI, which will create a new contact. So let’s pull that conditional apart, we’ll check for the URI first, and then put in a case statement to switch based on the request method.

(defn handler "request -> response"
  [req]
  (if (= (:uri req) "/")
    (case (:request-method req)
      :get
      {:status 200
       :headers {"Content-Type" "application/edn; charset=UTF-8"}
       :body (prn-str (vals @contacts))})))

To keep the handler lean pull that response into its own function.

(defn- get-all-contacts []
  {:status 200
   :headers {"Content-Type" "application/edn; charset=UTF-8"}
   :body (prn-str (vals @contacts))})

(defn handler "request -> response"
  [req]
  (if (= (:uri req) "/")
    (case (:request-method req)
      :get (get-all-contacts))))

The POST operation will create a new contact, so create a function for that, and hook it up in the handler. create-new-contact needs access to the Ring request so it can find the submitted values. Let’s see how that request map looks like when a form is being submitted by printing it out, for the rest just return a hard-coded 200 OK, we’ll get to that in a bit. I’m using pprint for this, which is a bit like prn but formats the output for readability. Make sure to add the require statement to the ns form.

(defn- create-new-contact [req]
  (pprint req)
  {:status 200})

(defn handler "request -> response"
  [req]
  (if (= (:uri req) "/")
    (case (:request-method req)
      :get (get-all-contacts)
      :post (create-new-contact req))))

To create a new contact, pick a new random uuid, fill in the contact details, and then hit POST. You can see the empty 200 OK coming back.

Now have a look at the REPL and you should find this pretty printed request map. If it’s not there check the *nrepl-server* buffer instead.

{:ssl-client-cert nil,
 :protocol "HTTP/1.1",
 :remote-addr "127.0.0.1",
 :headers
 {"accept" "",
  "connection" "close,keep-alive",
  "user-agent"
  "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0",
  "referer" "http://lol:3449/",
  "host" "localhost:3000",
  "accept-language" "en-US,en;q=0.5",
  "accept-encoding" "gzip, deflate",
  "content-length" "83",
  "content-type" "application/x-www-form-urlencoded;charset=UTF-8"},
 :server-port 3000,
 :content-length 83,
 :content-type "application/x-www-form-urlencoded;charset=UTF-8",
 :character-encoding "UTF-8",
 :uri "/",
 :server-name "localhost",
 :query-string "",
 :body
 #object[org.eclipse.jetty.server.HttpInputOverHTTP 0x13db08ce "HttpInputOverHTTP@13db08ce"],
 :scheme :http,
 :request-method :post}

Now you would expect to find the values you submitted in there, but it doesn’t look like they’re there. So where are they?

If you look at the Ring specification, you’ll see that the request body is a java.io.InputStream, in our case it’s a type of InputStream provided by Jetty. We’ll have to read from this stream to get to the submitted data.

According to ancient LISP tradition, a function that provides this kind of functionality is called slurp. Clojure’s version of slurp is pretty handy because it works with a bunch of different things, like files, urls, and input streams.

So grab the body from the request and slurp it, this returns a string.

(defn- insert-new-contact [req]
  (let [instream (:body req)
        body (slurp instream)]
    ,,,)

  {:status 200})

According to the Ring spec the request body is optional, it may be nil, and trying to slurp from nil will raise an exception, so let’s change that let to an if-let.

We already used EDN format for the responses, and we’ll do the same for the requests, so the “body” string contains unparsed EDN data. To turn that into a Clojure data structure, we use the read-string function provided by clojure.edn.

Note that clojure.core also contains a read-string function, and at first glance it may seem they do the same thing, the version in clojure.core is only intended to read Clojure code, it should never be used to read in data coming from untrusted sources, unless you want to sign up for a dose of remote code execution.

Now we can update the contact atom by associating the submitted contact with its id. We’ll assume the client has already picked a unique random id. Since we’re using UUIDs they should be able to do that without risk of clashing.

Finally we also need to hand back a response map. Since we’ve created a new resource the result is 201 CREATED, instead of just a 200 OK. We’re returning EDN, so make that clear with a Content-Type header, and finally put the contact again in the body.

(ns rolodex.core
  (:require [clojure.edn :as edn]
    ,,,))

(defn- create-new-contact [req]
  (if-let [instream (:body req)]
    (let [body (slurp instream)
          contact (edn/read-string body)]
      (swap! contacts assoc (:id contact) contact)
      {:status 201
       :body (prn-str contact)})))

That’s as far as we’ll go this episode. The API is now read-write, so you can try that out in the GUI. Generate a new id, fill in some values, and hit POST. When you do GET to refresh the list of contacts the new contact should show up in the list.

Next episode we’ll implement our own middleware to deal with the details of reading and writing EDN.

Show Notes

browse source code

At the start of this episode, the code is as found in the ring-part-1 branch. The end result is in the ring-part-2 branch.

rolodex-gui

To test out and play around with the API there’s a web app that you can also grab from github. Make sure your API is running at port 3000, and then start the app by issuing

lein run

and surfing to http://localhost:3449.

To get around cross-origin restrictions in browsers the server part of this app will proxy requests to /api to http://localhost:3000/.

Code

The final API code looks like this

(ns rolodex.core
  (:require [clojure.edn :as edn]
            [ring.adapter.jetty :refer [run-jetty]]
            [rolodex.generator :as gen]))

(defonce contacts (atom (into {} (map (juxt :id identity)
                                      (map gen/contact gen/names)))))

(defn- get-all-contacts []
  {:status 200
   :headers {"Content-Type" "application/edn; charset=UTF-8"}
   :body (prn-str (vals @contacts))})

(defn- create-new-contact [req]
  (if-let [instream (:body req)]
    (let [body (slurp instream)
          contact (edn/read-string body)]
      (swap! contacts assoc (:id contact) contact)

      {:status 201
       :headers {"Content-Type" "application/edn; charset=UTF-8"}
       :body (prn-str contact)})))

(defn handler "request -> response"
  [req]
  (if (= (:uri req) "/")
    (case  (:request-method req)
      :get (get-all-contacts)
      :post (create-new-contact req))))

(defonce server
  (run-jetty #'handler {:port 3000
                        :join? false}))

Ring, part 2: the Two Routes