Lambda Island

With the basics of multimethods out of the way it’s time to look at some of the more advanced uses. This episodes explores in depth Clojure’s keyword hierarchy features, some little known aspects of the isa? and type functions, and closes off with some examples that demonstrate the flexibility Clojure provides when modeling data and behavior.

Transcript

In part one I recapped the basics of protocols, and presented an introduction to multimethods. If you’ve never used multimethods before you should watch that one first. Now it’s time to dig into some of their more advanced aspects.

When calling defmulti, after providing the method name and dispatch function, you can supply two more options: :default and :hierarchy.

:default is pretty straightforward, it changes the dispatch value that is used as a fallback. This is useful if the :default keyword already has a certain significance in your domain. It can also be a way to provide a value that more meaningfully communicates its intent.

(defmulti obstacle? :item/type :default ::unkown-item-type)

(defmethod obstacle? ::unknown-item-type [item]
  (log/warn "Missing implementation (obstacle? " (:item/type item) ")")
  false)

At this point I should point out that calling defmulti a second time for the same method name does nothing. This is a safety mechanism that’s built-in, because when you redefine a multimethod it invalidates all previous defmethods. The downside is that when you change the dispatch function or any of the options, that Clojure will not see these changes even after you re-evaluate the defmulti form.

If you do want to force Clojure to pick up the changes, you need to first undefine the multimethod var using ns-unmap. Assigning nil to it also works. After that re-evaluate the defmulti, and any of its defmethods.

(ns-unmap *ns* 'obstacle?)
;; or
(def obstacle? nil)

Doing this unloading/reloading manually is tedious and error-prone, which is why you have tools.namespace. If you’re using a reloaded-style workflow then a call to (reset) should clear things up. The component and integrant episodes can tell you more about how that works.

Note that the defmethods for a given defmulti can be spread across many different namespaces, so having some kind of dependency tracking for smart reloading is no frivolous luxury.

(reloaded.repl/reset)
;; or
(integrant.repl/reset)

The :default option is fairly straightforward, :hierarchy on the other hand will need a bit more of an explanation, but for that we first need to talk about make-hierarchy and friends.

The make-hierarchy function in clojure.core doesn’t take any arguments, it always returns the same map with three keys, :parents, :ancestors, and :descendants.

(make-hierarchy) ;;=> {:parents {} :ancestors {} :descendants {}}

With derive you add things to the hierarchy, creating a parent-child or is-a relationship.

For instance in our game we can say that a :gate is a :door, a :rock-face is a :wall, and a :city-gate is a kind of :gate.

(def my-hierarchy (-> (make-hierarchy)
                      (derive :gate :door)
                      (derive :rock-face :wall)
                      (derive :city-gate :gate)))

Looking at the result you can see Clojure did some book keeping to track the relationships in both directions.

my-hierarchy
;;=>
{:parents     {:gate      #{:door}
               :rock-face #{:wall}
               :city-gate #{:gate}}

 :ancestors   {:gate      #{:door}
               :rock-face #{:wall}
               :city-gate #{:door :gate}}

 :descendants {:door #{:city-gate :gate}
               :wall #{:rock-face}
               :gate #{:city-gate}}}

Once you have your hierarchy in place you can pass it to the (isa?) function. Notice that it correctly handles transitive relations. Because :gate is a :door and :city-gate is a :gate, we can conclude that (in this hierarchy at least) :city-gate is a :door.

(isa? my-hierarchy :gate :door)      ;;=> true
(isa? my-hierarchy :city-gate :door) ;;=> true

But isa? doesn’t stop there. If you pass it two java classes, it will consider Java’s class hierarchy to determine whether the relationship holds.

(isa? my-hierarchy java.sql.Date java.util.Date) ;;=> true

So far we’ve used derive with keywords, you can also use symbols just the same, but the first value passed to derive, the so called “child” can also be an interface or class. This allows you to set up a relationship between any built-in or custom type and a keyword.

(derive my-hierarchy `foo `bar)
(derive my-hierarchy java.util.Date ::datelike)

Suppose the game has a Fence which is represented as a record. I can now establish a relationship between that record type, and the :wall keyword.

Note that hierarchies are just Clojure maps, so immutable. This means that to add to an existing hierarchy you’ll need some kind of mutable container like an atom or var. Using vars for this is quite common, since hierarchies are mostly defined at the top-level, when code is first loaded, so you don’t need the concurrency protections that atoms provide.

(defrecord Fence [])

(derive my-hierarchy Fence :wall)

(alter-var-root #'my-hierarchy derive Fence :wall)

(isa? my-hierarchy Fence :wall)
;; => true

isa? is a sort of equality relationship, if two values are equal then isa? will also consider them equal. This is true for both simple and complex types like maps.

(isa? my-hierarchy 5 5)             ;;=> true
(isa? my-hierarchy {:x :y} {:x :y}) ;;=> true

For vectors isa? has an extra trick up its sleave. If the vectors are of equal size, then isa? will call isa? recursively on their corresponding elements.

(isa? my-hierarchy [:city-gate :rock-face] [:door :wall]) ;;=> true

This works even for nested vectors, or when containing a mix of keywords and types present in the hierarchy, and plain values, so you can check multiple things in a single go, which can be really useful for multimethods, as I’ll show you in a bit.

(isa? my-hierarchy
      [[:city-gate :rock-face] {:x :y}]
      [[:door :wall] {:x :y}])
;;=> true

Clojure comes with a default hierarchy built-in, called clojure.core/global-hierarchy. It’s defined as private, but you can see what’s in there by accessing the var and then dereffing it. So far it’s just an empty hierarchy. You might also see relationships in there defined by libraries you’ve loaded.

@#'clojure.core/global-hierarchy

If you call isa? with only two arguments, so without the first hierarchy argument, then it will use this global hierarchy to do its lookups.

(isa? 5 5)                    ;;=> true
(isa? :gate :door)            ;;=> false

To add to this global hierarchy you call derive without the hierarchy argument. Note that there is a pretty big difference between the two and three argument version of derive. One is a pure function which returns a new hierarchy, the other one modifies a piece of state in place and returns nil. This is a bit of an irregularity in a language that is otherwise very systematic about these things, so watch out for that.

(derive ::gate ::door) ;;=> nil
(isa? ::gate ::door)   ;; => true

Another thing to note is that in this global hierarchy everything must be namespaced. You can’t add unqualified keywords or symbols to the global hierarchy. This is an understandable design decision, it is global state after all, so you want to prevent stepping on others peoples toes by properly qualifying the things you put in there.

(derive :gate :door)
;;=> Exception

At this point I should mention there’s also an underive function, which undoes the effect of derive. Just like derive it has a destructive 2-arity, and a functional 3-arity version.

(underive ::gate ::door)  ;; => nil
(isa? ::gate ::door)      ;; => false

Now that you know all about hierarchies, let’s look again at multimethods. When you call a multimethod Clojure first calls the dispatch function, to determine a dispatch value. Then it goes over the various defmethods, until it finds one where the dispatch value matches.

This match is performed using isa?, using the global hierarchy, so we can use that to our benefit, provided we switch to namespaced keywords, since otherwise we won’t be able to add them to the hierarchy.

(defmulti obstacle? :item/type)

(defmethod obstacle? :default [item]
  false)

(defmethod obstacle? ::door [item]
  (not (:door/open? item)))

(defmethod obstacle? ::wall [item]
  true)

(derive ::gate ::door)

(obstacle? {:item/type ::gate, :door/open? true})   ;;=> false
(obstacle? {:item/type ::gate, :door/open? false})  ;;=> true

Instead of using the global hierarchy you can also bring your own hierarchy, by providing a :hierarchy option to defmulti.

Create a var that will hold the hierarchy, and then add a helper function to update it. With this in place it will be easy to add things to the hierarchy later on, similar to the global hierarchy.

(def obstacle-hierarchy (make-hierarchy))

(defn obstacle-derive! [child parent]
  (alter-var-root #'obstacle-hierarchy derive child parent))

Now you pass this to defmulti. Note that you need to give it the var, and not just the value. This makes sense, Clojure expects to see a reference type since hierarchies are expected to be updated. Providing an atom would also work.

Notice how powerful this is. The obstacle? method can consider a gate to be like a door, but the climb function can consider it to be like a fence.

Do keep in mind the remark I made earlier: if obstacle? was already defined, then you need to unmap it first, or the new option will not take effect.

(defmulti obstacle? :item/type :hierarchy #'obstacle-hierarchy)

(obstacle-derive! ::gate ::door)

So now let’s reexamine the claim that Clojure provides “polymorphism a la carte”.

multimethods can dispatch on any aspect of an object, not just its type
lookup hierarchy is separate from the type/class hierarchy
different methods can have different hierarchies

Sounds pretty a la carte to me!

To wrap it up let’s look at some examples of multimethod usage.

A common feature in statically typed object-oriented languages is to have multiple versions of a function, based on the type and count of the arguments. This is called method overloading.

function attack(Player p, Zombie z) { }
function attack(Player p, Zombie z) { }
function attack(Player p, Creature c1, Creature c2) { }
function attack(Zombie z, Skeleton s) { }

Clojure supports functions with multiple arities well enough, you can supply multiple function bodies in a single function definition, but how about dispatching on types?

(defn attack
  ([attacker target] ,,,)
  ([attacker target1 target2] ,,,))

This is where multimethods come in again. Say you’re writing a game which has a player, and a number of creatures like zombies and skeletons. What you want is a unified interface to add these kinds of entities to the game state.

For this I’ll create a multimethod called into-game. As a dispatch value I’ll use a vector with the types of the arguments. Note the use of mapv, because of the way isa? works it has to be a vector, and not a seq.

(defrecord Player [x y hp])
(defrecord Zombie [x y hp desc])
(defrecord Skeleton [x y hp desc])

(defrecord Game [player creatures])

(defmulti into-game (fn [& args] (mapv type args)))

Now the implementations first list the types of arguments they expect, and then follow with the argument list and function body. I have one version to add a player to the game state, one for Zombies, and finally one for skeletons. This is all fairly elegant, but the last two implementations are basically identical, how could you get rid of that repetition?

(defmethod into-game [Game Player] [game player]
  (assoc game :player player))

(defmethod into-game [Game Zombie] [game zombie]
  (update game :creatures conj zombie))

(defmethod into-game [Game Skeleton] [game skeleton]
  (update game :creatures conj zombie))

Since you’re using records a common technique would be to create an empty protocol that’s just used as a marker. It doesn’t contain any methods, but by adding it to the records they now have a common supertype which you can use for your defmethod.

(defprotocol Creature)

(defrecord Zombie [x y hp desc]
  Creature)
(defrecord Skeleton [x y hp desc]
  Creature)

(defmethod into-game [Game Creature] [game c]
  (update game :creatures conj c))

Notice that you could very well achieve the same thing using the hierarchy. Make Zombie and Skeleton both derive from a qualified ::creature keyword, and then dispatch on that.

(derive Zombie ::creature)
(derive Skeleton ::creature)

(defmethod into-game [Game ::creature] [game creature]
  (update game :creatures conj creature))

Let’s try that, first create an empty game. Now you can add a player and a bunch of creatures to it by repeatedly calling into-game. For each type of game entity into-game will do the right thing, adding it at the right spot in the game state.

(def empty-game (map->Game {}))

(reduce into-game
        empty-game
        [(->Player 10 15 99)
         (->Zombie 1 1 15 "shambler")
         (->Zombie 20 10 15 "fastzom")
         (->Skeleton 25 7 10 "lil bones")])

I’ll store this in a var for later use

(def game (reduce into-game ,,,))

To allow the player to save the game you want to be able to convert the game state to XML, so you implement a multimethod to convert the various types used to the representation of an XML document as used by clojure.xml, meaning a map with :tag, :attrs, and :content keys.

The implementation for Game recursively calls to-xml on the player and the creatures, which then each get their own implementation.

(defmulti to-xml type)

(defmethod to-xml Game [{:keys [player creatures]}]
  {:tag :game
   :content [(to-xml player) (to-xml creatures)]})

(defmethod to-xml Player [{:keys [x y hp]}]
  {:tag :player
   :attrs {:x x :y y :hp hp}})

(defmethod to-xml ::creature [creature]
  {:tag (keyword (.getSimpleName (class creature)))
   :attrs (into {} creature)})

However this doesn’t quite work yet, the list of creatures is a plain vector, and you don’t have an implementation of to-xml for those. You also don’t want to add a defmethod for any vector, because a list of creatures will be represented differently than a list of items, for instance.

(to-xml game)
;;=> Exception

There’s a pretty elegant solution to this. Notice that so far I’ve been using the type function, rather than the class function. The difference is that when an object contains a :type metadata key, type will return that instead of the actual class.

(class [])
;; => clojure.lang.PersistentVector

(type [])
;; => clojure.lang.PersistentVector

(type ^{:type ::creature-list} [])
;; => :user/creature-list

This is really cool, because it means you can still use a plain old vector for the list of creatures, while easily distinguishing it from other vectors. So let’s add that metadata to the creature list in our empty game state, and then recreate the game state based on that.

So now add an implementation to to-xml for ::creature-list, which output a <creatures> xml tag, with the individual creatures as child elements.

Now to-xml no longer throws an exception, and you can pass it to clojure.xml/emit to output the actual XML. And of course this type is still checked against the current hierarchy. Pretty powerful stuff.

(def empty-game (map->Game {:creatures ^{:type ::creature-list} []}))

(defmethod to-xml ::creature-list [l]
  {:tag :creatures
   :content (map to-xml l)})

(def game
  (reduce into-game
          empty-game
          [(->Player 10 15 99)
           (->Zombie 1 1 15 "shambler")
           (->Zombie 20 10 15 "fastzom")
           (->Skeleton 25 7 10 "lil bones")]))

(clojure.xml/emit-element (to-xml game))

That’s all I have for you today. If you want to dig into a good real world example of multimethods and hierarchies then have a look at Integrant, covered in episode 39, and the Integrant-based web framework Duct.

I hope you’ll agree by now that Clojure has top-notch support for polymorphism, and that you can mix and match its ingredients in ways few other languages would allow.

Polymorphism á la carte indeed!

À la Carte Polymorphism, part 2