(→ Stranded Between Parens)

The Lambda Island Blog

Dates in Clojure: Making Sense of the Mess

You can always count on human culture to make programming messy. To find out if a person is a programmer just have them say “encodings” or “timezones” and watch their face.

In Java, and hence Clojure, there are half a dozen different ways to represent dates and times, which can lead to confusion, needless type coercion, and inconsistent code. In this post I’ll give you a quick lay of the land, and some tips to make it all a bit easier to deal with.

Representing Time

Unix Time

The traditional way to represent time, the system that has been handed down to us by The Elders, is known as Unix Time. A Unix timestamp is a number representing the amount of seconds passed since the start of time, which is conveniently defined as midnight, January 1, 1970.

1970's family pic

A happy family shortly after the start of time (credits)

java.util.Date

Some 822 million seconds later Java 1.0 came out. It was the year of the Spice Girls, the Macarena, and mutable date classes.

user> (def date (java.util.Date.))
#'user/date
user> date
#inst "2017-06-09T06:59:40.829-00:00"
user> (.setTime date 822391200000)
nil
user> date
#inst "1996-01-23T10:00:00.000-00:00"
user>

A Date wraps a milli-second based Unix timestamp, meaning it’s not time zone aware.

user>  (java.util.TimeZone/setDefault (java.util.TimeZone/getTimeZone "UTC"))
user> (.toString date)
"Tue Jan 23 10:00:00 UTC 1996"
user> (java.util.TimeZone/setDefault (java.util.TimeZone/getTimeZone "Europe/Berlin"))
user> (.toString date)
"Tue Jan 23 11:00:00 CET 1996"

That’s not all that’s wrong with java.util.Date, so in Java 1.1 they tried to rectify things by deprecating half of the Date API and instead introducing a (obviously mutable) GregorianCalendar.

JodaTime

It’s 2002, Brazil has beaten Germany in the FIFA World Cup, and someone decides the time is ripe for a proper date/time library. In no time at all JodaTime is born!

It doesn’t take long or Joda becomes the de facto time library on the JVM, and in 2010 it becomes the basis for clj-time.

Joda contains two principles classes for representing time, org.joda.time.Instant, which is essentially a Unix timestamp, and org.joda.time.DateTime, which explicitly stores year/month/day/etc.

Having both these classes makes the distinction clear between a “machine time” and “human time”. For machines things are relatively straightforward: time is an ever increasing number. The same instant in time corresponds with the same number, no matter where you are or how you look at it. There are no jumps or gaps, time is strictly monotonous.

The human view of time is a lot more complex. There are time zones, daylight saving time, not to mention different calendar systems and historical time adjustments. For adding timestamps to your logs machine time will work fine, but when your system communicates dates and times with end users you need a richer model.

JSR-310

For years this was the state of things: you used the built-in java.util.Date despite its shortcomings, or you relied on JodaTime as an external dependency. That you needed a library to do something as essential as working with dates and times seemed a bit odd, and so Oracle involved the JodaTime author, Stephen Colebourne, to work on a new Date/Time library for the JDK, affectionately known as JSR-310.

JSR-310 introduces a new java.time package, which includes java.time.Instant and java.time.ZonedDateTime. While clearly inspired by JodaTime, there are also some differences. The author explains in this article that there are some shortcomings in JodaTime that they sought to address.

Development was completed in 2014, and java.time shipped with Java 8. That’s basically last week in Java years, it will be a while before support for this has become widespread.

On the clj-time issue tracker there’s an ongoing discussion about what this will mean for clj-time. While there’s a lot of interest for clj-time to adopt JSR-310, it seems the current concensus is that if they do this will be with a new artifact id (a new library name). The differences between JodaTime and JSR-310 are big enough to make breaking API changes inevitable. By using a different name it will be possible to switch to the new version, while libraries you depend on can keep using the old version.

clojure.java-time

Meanwhile someone has gone ahead and created an idiomatic Clojure library for the java.time.* API, aptly named clojure.java-time, so perhaps this will be what clj-time 2.0 could have been. If you are starting a new project today you might skip clj-time altogether, and instead use clojure.java-time for your date and time needs, assuming you’re at least on Java 8.

If you do go that route you should do your homework though. java.time contains a handful of different date/timestamp classes, and you should know which is which. The clojure.java.time README has several links for further reading.

clojure.instant

It’s also worth pointing out that since Clojure 1.8 there’s a built-in namespace called clojure.instant, which contains a few utility methods for parsing dates.

java.sql.Timestamp

You thought we were done? Guess again! Besides java.util.Date, Java has always included yet another timestamp class: java.sql.Timestamp. The reason is that timestamps in the SQL standard have a nanosecond precision, while java.util.Date only stores milliseconds. So a java.sql.Timestamp inherits from java.util.Date, but adds an extra nanosecond field.

If you’re using JDBC to read or store timestamps, then you’ll run into this class.

Making things bearable

Having all these different types for representing essentially the same thing is a recipe for massive frustration, but with a few small tweaks we can make it all a lot less painful.

The trick is to pick one of the above, and use it across the board. For now org.joda.time.Instant is still a likely candidate, it has widespread library support, and you can combine it with clj-time for writing idiomatic Clojure code. If instead you want to be future proof you can opt for java.time.Instant, in combination with clojure.java-time. Several people have reported they’re already getting good use out of java.time. The snippets below assume you’re sticking with JodaTime.

Whenever time values enter your program, convert them to JodaTime. Maybe you’re reading JSON, EDN, Transit, you’re pulling values off an API or out of a database. Don’t let them propagate through your code as java.util.Dates (or worse, longs representing Unix timestamps), but convert them to org.joda.time.DateTime as soon as possible.

The same goes for outputting values. You don’t want to see an exception any time you encode some JSON with a timestamp, or save a date to the database.

Many libraries can be configured to do this automatically, so you only have to worry about it once.

JDBC

JDBC is Java’s standard for communicating with relational databases, and the clojure.java.jdbc library provides a convenient Clojure wrapper. By default JDBC expects, and outputs, java.sql.Timestamp. But by loading the clj-time.jdbc namespace it switched to JodaTime instead. Nice, that was easy!

;; load/save DB times as joda DateTime
(require 'clj-time.jdbc)

Cheshire

The go-to library for reading and writing JSON from Clojure is called Cheshire. JSON doesn’t have a standard way of encoding time values. If you do need to encode timestamps the common thing to do is to either use a string containing an ISO-8601 formatted timestamp, or to use a number representing a UNIX timestamp.

This snippet configures Cheshire to output org.joda.time.DateTime as a string.

(ns my.ns
  (:require [cheshire.factory]
            [cheshire.generate :as cheshire]
            [clj-time.coerce :as tc]
            [clj-time.core :as t])
  (:import [org.joda.time.DateTime]
           [com.fasterxml.jackson.core.JsonGenerator]
           [java.text.SimpleDateFormat]
           [java.util SimpleTimeZone]))

(defn cheshire-add-jodatime-encoder! []
  (cheshire/add-encoder
   org.joda.time.DateTime
   (fn [^org.joda.time.DateTime d ^JsonGenerator jg]
     (let [sdf (SimpleDateFormat. cheshire.factory/default-date-format)]
       (.setTimeZone sdf (SimpleTimeZone. 0 "UTC"))
       (.writeString jg (.format sdf (tc/to-date d)))))))

(cheshire-add-jodatime-encoder!)

EDN/Clojure

In EDN and Clojure source code you can use an #inst tagged literal to represent timestamps. These by default will be read as java.util.Date.

Having a literal syntax for timestamps can be very handy, one example would be to write out test data containing timestamps.

In my apps I tend to configure Clojure’s reader and printer to represent org.joda.time.DateTime as #joda/inst "...timestamp...", providing me the same convenience. When doing REPL-driven development (directly or inside a source buffer) this is particularly useful, since results that are printed out can be read again, they are proper time literals.

Clojure’s printer can be extended through the print-method and print-dup multimethods.

;; Configure the printer
(defmethod print-method org.joda.time.DateTime
  [dt out]
  (.write out (str "#joda/inst \"" (.toString dt) "\"") ))

(defmethod print-dup org.joda.time.DateTime
  [dt out]
  (.write out (str "#joda/inst \"" (.toString dt) "\"") ))

;; Utility function for the reader
(defn ->joda-inst [time-str]
  (org.joda.time.DateTime/parse time-str))

The reader can be taught about new tagged literals through a data_readers.clj file placed on the classpath. I found it works best to put it under resources, that way Leiningen is smart enough to merge it with other data_readers.clj files provided by libraries when packing up an uberjar.

;; resources/data_readers.clj
{joda/inst lambdaisland.util.time/->joda-inst}

Tu quoque, ClojureScript?

This article has focused on JVM based Clojure, but what about ClojureScript? Things aren’t quite as chaotic on the JavaScript side. It seems in general people are content using js/Date. There is a separate goog.date.DateTime included with the Google Closure library, which in turn is used by cljs-time, so you might run into it.

For a good time

Java’s time/date landscape can be difficult to navigate, but a map of the territory can help you along the way. And with the right tools in your belt it can be a pleasant journey after all.

Do you have your own snippets for making libraries or subsystems play well with either JodaTime or java.time? Send them in and I will add them to the article

A special thanks to Ben Lovell, Tim Gilbert, and Sean Corfield for pointing me to clojure.java-time.

About the author

Arne divides his time between making Clojure tutorial videos for Lambda Island, and working on open-source projects like Chestnut. He is also available for Clojure and ClojureScript training and mentoring. You can support Arne through his Patreon page.

More blog posts

Clojure Gotchas: Surrogate Pairs

tl;dr: both Java and JavaScript have trouble dealing with unicode characters from Supplementary Planes, like emoji 😱💣.

Today I started working on the next feature of lambdaisland/uri, URI normalization. I worked test-first, you’ll get to see how that went in the next Lambda Island episode.

One of the design goals for this library is to have 100% parity between Clojure and ClojureScript. Learn once, use anywhere. The code is all written in .cljc files, so it can be treated as either Clojure or ClojureScript. Only where necessary am I using a small amount of reader conditionals.

Simple and Happy; is Clojure dying, and what has Ruby got to do with it?

The past week or so a lot of discussion and introspection has been happening in the Clojure community. Eric Normand responded to my one year Lambda Island post with some reflections on the size and growth of the community.

And then Zack Maril lamented on Twitter: “I’m calling it, clojure’s dying more than it is growing”. This sparked a mega-thread, which was still raging four days later. A parallel discussion thread formed on Reddit. Someone asked if their were any Clojure failure stories. They were pointed at this talk from RubyConf 2016, which seemed to hit a lot of people right in the feels, and sparked a subthread with a life of its own.

Meanwhile Ray, one of the hosts of the defn podcast reacted to the original tweet: “I’m calling it: Clojure is alive and well with excellent defaults for productive and sustainable software development.” This sparked another big thread.

Loading Clojure Libraries Directly From Github

Did you ever fix a bug in an open source library, and then had to wait until the maintainer released an updated version?

It’s happened to me many times, the latest one being Toucan. I had run into a limitation, and found out that there was already an open ticket. It wasn’t a big change so I decided to dive in and address it. Just a little yak shave so I could get on with my life.

Now this pull request needs to be reviewed, and merged, and eventually be released to Clojars, but ain’t nobody got time for that stuff. No sir-ee.

Lambda Island Turns One, The Story of a Rocky Ride

One year ago to date I launched Lambda Island, a service that offers high quality video tutorials on web development with Clojure and ClojureScript. It’s been quite a ride. In this post I want to look back at the past year, provide some insight into how this experience has been for me, and give you a glimpse of what the future has in store.

This story really starts in December 2015. After three years of doing contract work for Ticketsolve I decided it was time for a change. I have been self-employed for many years, but I knew that sooner or later I wanted to try my hand at selling a product, rather than selling my time.

In January and February I took some time for soul-searching, and recharging. I went to speak at RubyConf Australia, and got to hang out with some old friends around Australia and New Zealand. Once back in Berlin I got busy creating Lambda Island.

Writing Node.js scripts with ClojureScript

In the two most recent  Lambda Island episodes I covered in-depth how to create command line utilities based on Lumo, how to combine them with third party libraries, and how to deploy them to npmjs.com.

However there’s a different way to create tools with ClojureScript and distribute them through NPM, without relying on Lumo. In this blog post I want to quickly demostrate how to do just that.

To recap, Lumo is a ClojureScript environment based on Node.js, using bootstrapped (self-hosted) ClojureScript. This means the ClojureScript compiler, which is written in Clojure and runs on the JVM, is used to compile itself to JavaScript. This way the JVM is no longer needed, all you need is a JavaScript runtime to compile and run ClojureScript code, which in this case is provided by Node.js. On top of that Lumo uses nexe, so Lumo can be distributed as a single compact and fast executable binary.

Announcing lambdaisland/uri 1.0.0

I just released lambdaisland/uri, a pure Clojure/ClojureScript URI library. It is available on Github and Clojars.

This is a small piece of the code base that powers lambdaisland.com. It’s inspired by Ruby’s Addressable::URI, the most solid URI implementation I’ve seen to date, although it only offers a small part of the functionality that library offers.

It’s written in pure Clojure/ClojureScript, with only minimal use of .cljc reader conditionals to smooth over differences in regular expression syntax, and differences in core protocols. It does not rely on any URI functionality offered by the host, such as java.net.URL, so it’s usable across all current and future Clojure implementations (Clojure, ClojureScript, ClojureCLR).

re-frame Subscriptions Got Even Better

Up until recently, to use re-frame subscriptions in Reagent views, you had to use a form-2 component.

A form-2 component is a function that returns another function, which does the actual rendering of the component to hiccup. In contrast, a form-1 component renders the hiccup directly.

;; form-1
(defn todo-item [todo]
  [:div.view
   [todo-checkbox (:id todo) (:completed todo)]
   [:label {:unselectable "on"} title]
   [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]])

;; form-2
(defn todo-item [todo]
  (fn [todo]
    [:div.view
     [todo-checkbox (:id todo) (:completed todo)]
     [:label {:unselectable "on"} title]
     [:button.destroy {:on-click #(dispatch [:todos/remove (:id todo)])}]]))

Game Development with Clojure/ClojureScript

This weekend it’s Ludum Dare again, the world’s longest running game jam. The idea is that, alone or with a team, you build a game in a weekend based on a certain theme.

We got a little team together here in Berlin, and so I’ve been reviewing what options there are for someone wanting to build a game in Clojure or Clojurescript.

The good news is there are plenty of options, as you’ll see from the list below. You can do desktop games, browser based games with canvas or webgl, and you can even create Unity 3D games, all from your comfortable Clojure parentheses.

Union Types with Clojure.Spec

Elm and other statically typed languages have a great feature called Union Types (also called Sum Types or Algebraic Data Types).

Here’s an example taken from Elm. Suppose your system used to represent users as integers, maybe just an auto-incrementing primary key, but then switched to UUIDs represented as strings.

To correctly model this situation, you need a way to create a type that can be either an integer or a string, that’s what union types give you.