Lambda Island

We add GET, PUT, DELETE operations to the contact book API. In the process we pull out the EDN handling into its own middleware, and come across some handy Regex tips. This third episode picks up the pace a bit, but by now that shouldn’t be a problem.

Transcript

This episode builds upon episodes 13 and 16 about Ring, although you can watch it by itself if you’re already familiar with the basics of Ring request and response maps.

All code is on Github, in the lambdaisland/rolodex repository. The starting point for this episode is the ring-part-2 branch, the final code can be found in ring-part-3.

There’s a supporting Reagent app that you can use to test the API. You can find instructions in the show notes.

Let’s have another look at the code we have so far. It consists of an atom containing our contact database, and two functions corresponding with the two API operations we already support. Then there’s the handler which takes care of the routing, and finally the invocation to start Jetty.

The assumption that we’re using EDN is encoded in five different places. It determines which Content-Type header to set, how to parse the request, and how to encode the response.

What if we want to switch to JSON or Transit later on? What if we want to support multiple formats through content negotiation? As it stands that will be much harder than it needs to be. Let’s clean this up.

Instead of repeating the content-type in every response, we can just set it once in the handler.

(defn handler "request -> response"
  [req]
  (-> (if (= (:uri req) "/")
        (case  (:request-method req)
          :get (get-all-contacts)
          :post (create-new-contact req)))
      (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")))

Now look at these two response maps, each time we’re calling prn-str to convert the result. There are only two of these now, but we’ll add more soon, so let’s move that prn-str into the handler as well.

We’ll first assign the unencoded response to :body, then convert the body before passing it on.

(defn handler "request -> response"
  [req]
  (-> ,,,
      ,,,
   (update :body prn-str)))

So now the content-type header and encoding of the result are handled in one place, but we’re still manually parsing the request body. Let’s handle that centrally as well.

We’ll add some pre-processing on the request map. Like before check if a request body is present, then slurp and parse it. The result gets added to the request map under :params. If there was no body in the request then do nothing.

:params is not part of the Ring spec, but since Ring requests are just maps we can add new keys to them, it’s an open ended system.

(defn- create-new-contact [req]
  (let [contact (:params req)]
    (swap! contacts assoc (:id contact) contact)
    {:status 201
     :body contact}))

(defn- parse-params [req]
  (if-let [instream (:body req)]
    (assoc req :params (-> instream slurp edn/read-string))
    req))

(defn handler "request -> response"
  [req]
  (let [req (parse-params req)]
    (-> (if (= (:uri req) "/")
          (case  (:request-method req)
            :get (get-all-contacts)
            :post (create-new-contact req)))
        (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")
        (update :body prn-str))))

Take a step back and have a look at this function. It consists of three parts. We get the request map from the Ring adapter, and do some pre-processing. Then comes the code that actually implements the business logic, which works with this new request map.

We get a response map back, and then do some post-processing, making it ready to be passed back to the handler.

Only the middle part implements the logic of our app. The rest is just infrastructure, plumbing. It would be nice if we could pull that out into a separate function that handles all the details of sending and receiving EDN data. However since some of it would have to run at the start of the handler, and some of it at the end, we would need two functions.

The function decorator pattern allows us to do it in a single function. In the context of Ring this kind of function is called a Ring middleware.


(defn handler "request -> response"
  [req]
  (let [req (parse-params req)]

    (-> (if (= (:uri req) "/")
          (case  (:request-method req)
            :get (get-all-contacts)
            :post (create-new-contact req)))

        (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")
        (update :body prn-str))))

So let’s make an EDN middleware. A middleware is a function which takes a handler function and returns a new handler function. You can think of it as “wrapping” the old handler with some extra functionality, which is why their names commonly start with wrap-.

You can have many middlewares wrapped around your handler, like rings of an onion, or Russion dolls. A request will pass through all the middlewares on the way in, and the response passes through them all again in reverse order on the way out.

wrap-edn returns an anonymous function which will be the new handler. All that EDN handling we already had in the old handler, so I’ll copy that over, and just remove the logic bit in the middle.

So what goes in the middle? Well, wrap-edn doesn’t care, we’ll just delegate to the original handler.

Now all of that pre- and post-processing can go, and our handler is back to being nothing but application logic. All that’s left is actually wrapping the old handler.

(defn wrap-edn "handler -> handler"
  [handler]
  (fn [req]
    (let [req (parse-params req)]

      (-> (handler req)

          (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")
          (update :body prn-str)))))

(defn handler "request -> response"
  [req]
  (if (= (:uri req) "/")
    (case  (:request-method req)
      :get (get-all-contacts)
      :post (create-new-contact req))))

(defonce server
  (run-jetty (wrap-edn #'handler) {:port 3000
                                       :join? false}))

There’s one small issue with this code. Normally if a the handler returns nil, then Jetty will turn that into a 404 Not Found repsonse, but this middleware will turn nil into a valid response map, and so it becomes a 200 OK instead.

You can try that out, suppose we have a handler that always returns nil, and then we wrap it with wrap-edn, and call it with a request map. The result should be nil, bit it’s not.

((wrap-edn (constantly nil)) {})
;;=> {:headers {"Content-Type" "application/edn; charset=UTF-8"}, :body "nil\n"}

So get the response from the handler first, and only do the post-processing when the response is not nil.

(defn wrap-edn "handler -> handler"
  [handler]
  (fn [req]
    (let [req (parse-params req)
          res (handler req)]
      (if res
        (-> res
            (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")
            (update :body prn-str))))))

Cool, with that EDN stuff out of the way let’s move on to the next three API endpoints. These all work on a specific contact, identified by putting the UUID in the URI. So let’s try to parse that.

GET /:id
  Retrieve a single contact

PUT /:id
  Update a contact

DELETE /:id
  Delete a contact

We’ll have to check if the current URI path is a slash followed by a UUID, which is represented by sections of hexadecimal digits separated by dashes. This is easy to check with a regex.

Whenever you need to do anything non-trivial with a regular expression, pull up the documentation for the Java regex pattern class. Clojure’s regular expressions are the same as Java’s, and this one page is basically the full manual. A quick search shows that there’s a specific notation for matching hexadecimal digits.

We need 8 hexadecimal digits, then three times 4, then 12, with dashes in between.

\A and \z match the beginning and end of the string, to make sure we match the whole input. We want to match an initial slash, and then the UUID, which I’m putting in a capturing group so it’s easy to get out again. Now match a hex digit, repeated 8 times, then a dash. Next we want 4 - 4 - 4 and 12.

Now let’s try that. Because of the capturing group re-find returns a vector with two strings: the first is the complete match, the second is just the UUID without the leading slash.

(def uuid-uri-pattern #"\A/(\p{XDigit}{8}-\p{XDigit}{4}-\p{XDigit}{4}-\p{XDigit}{4}-\p{XDigit}{12})\z")

(re-find uuid-uri-pattern "/df0bf48f-f02f-4b7f-8aa8-28b16e1d8af1")
;;=> ["/df0bf48f-f02f-4b7f-8aa8-28b16e1d8af1" "df0bf48f-f02f-4b7f-8aa8-28b16e1d8af1"]

Add another let here to already pull the uri out of the request. Now add an else clause to this if, where we check if the uri matches the UUID pattern. re-find returns a vector of two strings, but we’re only interested in the second one.

uuid here contains a String, but Clojure (actually, Java) has a specific UUID type, and that’s what we’ve been using in our contacts database, so we need to convert. Make sure to import the UUID class in your namespace.

Use a case to switch based on the request method like we did earlier. GET will just fetch a single contact, let’s make a function for that.

Also let’s pull this :request-method into the let as well to make this function a little less dense.

Now all that’s left is to return the response map with the contact’s details. Uh yeah we don’t need the whole request here, just the uuid.

I hope you’re starting to see the pattern. What we’re doing in the api handler function is called the “routing” (or “routing”). Based on request method and URI it delegates to functions that will take care of handling the request.

If you’re following along with the GUI then select a contact, and clear all fields except the id field. Now you can do a GET to load the missing data from the API.

(defn- get-contact [uuid]
  {:status 200
   :body (@contacts uuid)})

(defn api "request -> response"
  [req]
  (let [{uri :uri params :params} req]
    (if (= uri "/")
      (case ,,,)
      (if-let [uuid (second (re-find uuid-uri-pattern uri))]
        (let [uuid (UUID/fromString uuid)]
          (case (:request-method req)
            :get (get-contact uuid)))))))

Almost there, the rest is straightforward now. Doing a PUT request will update the contact based on the incoming params, so let’s get those from the request. Next add a case for a :put request, and delegate it to a new update-contact function.

The incoming “params” are the updated contact data. They might already contain the contact id, but we’ll give preference to the id that we found in the URI.

Use swap! to update the contacts atom, and finally return another Ring response, a 200 OK with the contact as its body.

Let’s also clean up this create-new-contact function, it doesn’t need access to the whole request, just the params is enough.

(defn handler "request -> response"
  [req]
  (let [,,,
        params (:params req)]
    (if (= uri "/")
      ,,,
      (if-let [uuid ,,,]
        (case method
          ,,,
          :params (update-contact uuid params))))))

(defn- update-contact [uuid params]
  (let [contact (assoc params :id uuid)]
    (swap! contacts assoc uuid contact)
    {:status 200
     :body contact}))

(defn- create-new-contact [contact]
  (swap! contacts assoc (:id contact) contact)
  {:status 201
   :body contact})

(defn handler [req]
  (let [,,,
        params (:params req)])
  ,,, (create-new-contact params))

Finally we get to delete, add the delete case, delegate to a delete-contact function, which uses dissoc to remove the id and its associated contact from the contacts atom.

We return “204 No Content” since the request was succesful, but there’s nothing to send back.

(defn- delete-contact [uuid]
  (swap! contacts dissoc uuid)
  {:status 204})

We’ve implemented the complete API now, only the full text search is still missing. And we did it with nothing but pure Clojure.

You should be comfortable now working with Ring requests and responses, including writing and using Ring middleware. These are fundamentals that will serve you for the rest of your Clojure career.

Show Notes

browse source code

At the start of this episode, the code is as found in the ring-part-2 branch. The end result is in the ring-part-3 branch.

rolodex-gui

To test out and play around with the API there’s a web app that you can also grab from github. Make sure your API is running at port 3000, and then start the app by issuing

lein run

and surfing to http://localhost:3449.

To get around cross-origin restrictions in browsers the server part of this app will proxy requests to /api to http://localhost:3000/.

Documentation

If you’re still a bit fuzzy on the core Ring concepts, request maps, response maps, handlers, and middleware, then you should go and read the Concepts page in the Ring wiki.

The documentation for Clojure’s regular expression can be found in the JavaDoc for java.util.regex.Pattern

Final code

(ns rolodex.core
  (:require [clojure.edn :as edn]
            [ring.adapter.jetty :refer [run-jetty]]
            [rolodex.generator :as gen])
  (:import java.util.UUID))

(defonce contacts (atom (into {} (map (juxt :id identity)
                                      (map gen/contact gen/names)))))

(def uuid-uri-pattern #"\A/(\p{XDigit}{8}-\p{XDigit}{4}-\p{XDigit}{4}-\p{XDigit}{4}-\p{XDigit}{12})\z")

;; individual routes

(defn- get-all-contacts []
  {:status 200
   :body (vals @contacts)})

(defn- create-new-contact [contact]
  (swap! contacts assoc (:id contact) contact)
  {:status 201
   :body contact})

(defn- get-contact [uuid]
  {:status 200
   :body (@contacts uuid)})

(defn- update-contact [uuid params]
  (let [contact (assoc params :id uuid)]
    (swap! contacts assoc uuid contact)
    {:status 200
     :body contact}))

(defn- delete-contact [uuid]
  (swap! contacts dissoc uuid)
  {:status 204}) ;; 204 = empty response

;; EDN middleware

(defn- parse-params [req]
  (if-let [instream (:body req)]
    (assoc req :params (-> instream slurp edn/read-string))
    req))

(defn wrap-edn "handler -> handler"
  [handler]
  (fn [req]
    (let [req (parse-params req)
          res (handler req)]
      (if res
        (-> res
            (assoc-in [:headers "Content-Type"] "application/edn; charset=UTF-8")
            (update :body prn-str))))))

;; handler performing the routing

(defn handler "request -> response"
  [req]
  (let [uri (:uri req)
        method (:request-method req)
        params (:params req)]
    (if (= uri "/")
      (case method
        :get (get-all-contacts)
        :post (create-new-contact params))
      (if-let [uuid (second (re-find uuid-uri-pattern uri))]
        (let [uuid (UUID/fromString uuid)]
          (case method
            :get (get-contact uuid)
            :put (update-contact uuid params)
            :delete (delete-contact uuid)))))))

(defonce server
  (run-jetty (wrap-edn #'handler) {:port 3000
                                   :join? false}))

Ring, part 3: Return of the Middleware