Lambda Island

With records, Clojure has introduced a way to create struct-like objects, while avoiding some of the problems encountered in object orientation. Records don’t hide their data behind custom interfaces, but instead partake in the abstractions that Clojure’s own data types are built on, so that code can treat treat a record like a regular Clojure data type.

Protocols are Clojure’s take on abstract interfaces. They make it possible to extend preexisting types to new methods, and extend preexisting methods to new types, thus avoiding the expression problem.

Transcript

Many programming languages have some kind of “struct” type, a way of creating values with specific named fields. Clojure calls these records, for example a Book record could have an author, title, and year it was published.

defrecord takes the name of the record type, and then a vector forms with the field names.

(defrecord Book [author title year])

defrecord is an extension of deftype, so everything you know about types also applies to records. You can construct them the same way, with an interop style constructor, or with the arrow constructor function, and you can access fields with dot forms. If you’re not familiar yet with deftype, I recommend checking out episode 23, which covers everything you need to know.

(defrecord Book [author title year])

(Book. "Ursula K. Le Guin" "The Dispossessed" 1974)
(->Book "Ursula K. Le Guin" "The Dispossessed" 1974)

(.title (->Book "a" "b" 1999))

That said, even though records can do everything that types can, you don’t normally use them the same way. That’s because records behave like proper Clojure data types. With deftype in order to have your custom type partake in Clojure’s abstractions you have to manually implement all the necessary interfaces. Records already come with implementations for a dozen of those interfaces, which means you can treat them like any other Clojure value. In particular they behave a lot like maps.

This starts with the constructor. Besides the arrow constructor that takes positional arguments, records also provide a constructor which takes a map, with keywords for keys that correspond with the record’s field names.

(map->Book {:title "The Caves of Steel" :author "Isaac Asimov" :year 1954})

Records have value semantics, so two records with the same field values are considered equivalent. Functional programmers the world over heave a sigh of relief.

(= (map->Book {:author "Pat Cadigan", :title "Fools", :year 1992})
   (map->Book {:author "Pat Cadigan", :title "Fools", :year 1992})) ;;=> true

(= (map->Book {:author "Pat Cadigan", :title "Fools", :year 1992})
   (map->Book {:author "Anne McCaffrey", :title "Dragonflight", :year 1968})) ;;=> false

You can also add metadata to records, something vanilla deftype does not support. You have to use with-meta here instead of the caret notation because that notation only works with literals, and calling a constructor function does not constitute a literal.

This is because metadata applied with the caret shorthand is read and added to the object by the Clojure reader before any code is evaluated. In order to use it the output of the reader needs to be a fully constructed object, rather than a code form that evaluates to the object.

(-> (map->Book {:title "Consider Phlebas" :author "Iain M. Banks" :year 1987})
    (with-meta {:source "Wikipedia"})
    meta)

;; doesn't work
^{:source "Wikipedia"} (map->Book {:title "Consider Phlebas"
                                   :author "Iain M. Banks"
                                   :year 1987})

To get around this you can use so called “reader forms” to build your records. The hash sign should already be a hint that these are processed by the reader. You put the fully qualified name of the record, followed either by a vector form, for positional arguments, or a map form. Unsuprisingly deftype only supports the first.

Note that technically you’re referencing the corresponding Java class, so if your namespace has dashes you need to replace them with underscores.

Now that you’re using literals you can directly attach metadata.

#ep24defrecord.core.Book["Iain M. Banks" "Consider Phlebas" 1987]
#ep24defrecord.core.Book{:author "Iain M. Banks" :title "Consider Phlebas" :year 1987}

(def consider-phlebas
 ^{:source "Wikipedia"}
  #ep24defrecord.core.Book{:title "Iain M. Banks"
                           :author "Consider Phlebas"
                           :year 1987})

(meta consider-phlebas)

Records also behave like maps when it comes to field access. You can use get, or look up a field by using a keyword as a function. To update a field, you use assoc. The result is a new Book record with an updated field.

Since these basic map operations are all implemented you can also use other map functions, like update, update-in, or reduce-kv.

(def book (->Book "Iain M. Banks" "Consider Phlebas" 1987))

(get book :title)    ;;=> "Consider Phlebas"
(:title book)        ;;=> "Consider Phlebas"

(assoc book :title "The Player of Games")
;;=> #ep24defrecord.core.Book{:author "Iain M. Banks", :title "The Player of Games", :year 1987}

(-> book
    (assoc :title "The Player of Games")
    (update :year inc))
;;=> #ep24defrecord.core.Book{:author "Iain M. Banks", :title "The Player of Games", :year 1988}

You can even assoc fields that aren’t defined on the record. If you’ve been paying attention then this should be at least a little surprising, after all fields are implemented as actual object fields on the host level, meaning Java or JavaScript. How can Clojure add fields on the fly to an existing class?

The answer is that it doesn’t. Instead records internally keep an extra map for “extra” keys that don’t have a corresponding “real” field.

(assoc book :series "The Culture Series")
;;=> #ep24defrecord.core.Book{:author "Iain M. Banks", :title "Consider Phlebas", :year 1987, :series "The Culture Series"}

Records are also Seqable, calling seq on a record will yield a seq of map entries, again just like maps, Map entries behaves a lot like two element vectors, so you can call first, second or last on them, but there’s also specific functions for them: key and val.

If you really want to turn your record into a plain map, you can use into.

(seq book)
;;=> ([:author "Iain M. Banks"] [:title "Consider Phlebas"] [:year 1987])

(map first book)     ;;=> (:author :title :year)
(map key book)       ;;=> (:author :title :year)
(map val book)       ;;=> ("Iain M. Banks" "Consider Phlebas" 1987)

(into {} book)       ;;=> {:author "Iain M. Banks", :title "Consider Phlebas", :year 1987}

All in all this looks pretty good. You get a custom type, and you can implement interfaces or protocols to support polymorphism, but at the same time it’s really just a map, and so the code you pass it to never has to know it’s really something else.

There is one small difference that you should be aware of though. Maps behave like functions, you can pass in a keyword and get the corresponding value back. Records don’t do this. This exception shows that Clojure tries to treat the book as a function, and fails.

To look up a value in the record you need to either use a keyword as the function, or explicitly use get.

#_(book :title)
;;=> Exception...

(get book :title)
(:title book)

If necessary it’s easy to work around though. Clojure functions or other callable objects like maps or sets are really just objects that implement the IFn interface, so you can do the same for your record.

Implement invoke so it calls get with the record itself and the passed in keyword as arguments.

(defrecord Book [author title year]
  clojure.lang.IFn
  (invoke [this kw]
    (get this kw)))

(def book (->Book "Iain M. Banks" "Consider Phlebas" 1987))

(book :title) ;;=> "Consider Phlebas"

Let’s take a moment to appreciate this design. Rich Hickey considers certain aspects of object orientation problematic, in particular data hiding and concrete derivation. So while records may look a lot like value objects from OO languages, they have the same transparent and uniform data access that Clojure’s own data structures provide.

And while you can implement interfaces and protocols, you can’t “inherit” from concrete types. None of that “A bird is an animal, a duck is a bird” kind of stuff. Inheritance is a source of tremendous complexity, and so it was wisely left out.

What you do get is fast type based dispatch, either through Java’s interfaces, or through Clojure’s own version of it: protocols.

Let’s look at that through a classic example: geometrical shapes. Each shape has a surface area, but how to calculate that surface area depends on the kind of shape.

So create a Shape protocol, and include a surface-area function.

(defprotocol Shape
  "Protocol for computing properties of geometrical shapes"
  (surface-area [shape]
    "The surface are of this shape."))

The first thing to note is that Shape and surface-area are regular vars. We’re fully in Clojure land now. And because they are vars, they can contain metadata like docstrings, something you couldn’t do with interfaces.

Shape ;;=> {...}
surface-area ;;=> #function[ep24defrecord.core/eval26559/fn--26560/G--26550--26565]

(:doc (meta #'Shape))
(:doc (meta #'surface-area))

Now define some shape records. A Rectangle has a width and a height. Make Rectangle “implement the Shape protocol”, so inside the defrecord write Shape, followed by implementations of the methods in the protocol. In this case that’s just surface-area. The rectangle record will be passed in, but we can also access the width and height fields inside this method definition directly.

Next up is a Circle record. A circle is defined by its radius, and its surface area is the square of the radius multiplied by Pi.

This code can now ask for the surface area of any shape, without having to worry about the kind of shape it’s dealing with.

(defrecord Rectangle [width height]
  Shape
  (surface-area [rectangle]
    (* width height)))

(defrecord Circle [radius]
  Shape
  (surface-area [circle]
    (* Math/PI (* radius radius))))

(surface-area (->Circle 1))          ;;=> 3.141592653589793
(surface-area (->Rectangle 2 3))     ;;=> 6

It’s also possible to implement a protocol for a type that was already defined previously, even if the type is provided by Java or by a third party library. Turns out Java already contains a class to represent rectangles. By extending the type so that it implements the Shape protocol, you can now use objects of type Rectangle in your code transparently. This is a great way to bridge the gap between functional Clojure code and object oriented Java code when working with third party libraries.

(extend-type java.awt.Rectangle
  Shape
  (surface-area [rect]
    (* (.width rect) (.height rect))))

(surface-area (java.awt.Rectangle. 100 50))

In this case I’ve used extend-type to implement a single protocol for this type, but you can also use extend-type to add several protocols to a type at once.

Sometimes you want to do the opposite: implement a single protocol for multiple types. That’s what extend-protocol is for.

Say you’re writing a surface-area library, and you want it to work with every geometrical shape class in Java’s standard library. With a single extend-protocol you can add implementations for every class you can find.

(extend-protocol Shape
  java.awt.Rectangle
  (surface-area [rect]
    (* (.width rect) (.height rect)))

  java.awt.geom.Ellipse2D$Double
  (surface-area [e]
    (* Math/PI (.width e) (.height e))))

(surface-area (java.awt.geom.Ellipse2D$Double. 0 0 100 50))

Note how this makes protocols fully extensible in two directions. You can add new types that implement a protocol, say Triangle or Hexagon, and you can create new protocols and extend them to pre-existing types, all without touching existing code.

Very few languages allow this, for instance Java forces you to declare all interfaces a class implements, you can’t add any later on. This is known as the “expression problem”, and it’s elegantly solved with protocols.

A really useful thing you can do with protocols is swap out parts of your application during testing. Say you rely on a web service to retrieve user info. You might want to have some tests that just use stubbed out responses, instead of hitting the real API.

With a protocol you can create a facade that sits in front all requests to the service. You provide one “real” implementation, and pass this one to any function that needs to access the API.

During testing instead you use an alternative implementation, one that returns fake responses.

So here I have a UserService protocol that I’ll use to wrap access to the real API. The first implementation, RESTUserService, does an actual HTTP call when calling fetch-user-info. I pass an instance of this record to any function that needs retrieve user info.

The TESTUserService also implements UserService, but instead of doing HTTP calls it just returns canned responses.

Now in my test I can call this application code with the test API wrapper, and see what comes out.

(ns ep24defrecord.core
  (:require [clojure.test :refer [deftest is]]))

(defprotocol UserService
  (fetch-user-info [_ user-id] "Fetch info about the given user"))

;; app
(defrecord RESTUserService [endpoint]
  UserService
  (fetch-user-info [_ user-id]
    (http-get (str endpoint "/" user-id))))

(defn user-profile-handler [api user-id]
  ,,,)

(user-profile-handler (->RESTUserInfo "http://...") 32)

;;test
(defrecord TESTUserInfo [mock-data]
  UserService
  (fetch-user-info [_ user-id]
    (get mock-data user-id)))

(deftest profile-test
  (let [mock-api (->TESTUserInfo {32 {:name "John"}})]
    (is (= (user-profile-handler mock-api 32) ,,,))))

Records and protocols are a useful Clojure feature. They work the same in ClojureScript so I didn’t dwell on the differences. If you’re new to Clojure then resist overusing them, often a regular map will do fine. That said, combined with protocols they are a great way to make boundaries in your code explicit.

I hope you enjoyed this episode. Till next time, on Lambda Island!

defrecord and defprotocol