Lambda Island

More and more sites offer the ability to log in with Facebook, Twitter, or Github, which can be quite convenient, as it means fewer passwords to keep track of. Adding this kind of capability to a Ring based app is easier than you might think, using the ring-oauth2 library. In this episode we’ll demonstrate how to implement a “Sign in with Google” button. The process is very similar for other providers.

Transcript

The most common way of authenticating on the web is using a username and password, but it is not the only way. Many sites will offer you the option to sign in with Facebook, Twitter, or Github. This is made possible by the OAuth protocol, an internet standard for delegating access to APIs. In this Episode, I will show you how you can achieve the same thing in your Ring-based web application using the ring-oauth2 library by James Reeves.

This episode will add a “Sign in with Google” button to the Booklog app, which you already encountered in episode 28 about Buddy and episode 34 about Acceptance Testing.

You can find the link to the Github repo in the show notes. Clone it, and check out the ep49-start branch.

git clone https://github.com/lambdaisland/booklog.git -b ep49-start

Here’s the project.clj for this project, which was created with Chestnut. In episode 28 we added Buddy for authentication, in episode 34 we used Sparkledriver to write integration tests. In this episode we’re adding ring-oauth2.

(defproject lambdaisland/booklog "0.1.0"
  :dependencies [,,,
                 [ring-oauth2 "0.1.4"]
                 ,,,]

  )

I initially ran into this issue where the REPL wouldn’t start, showing an error inside the Chestnut JSON library, caused by an error in com.fasterxml.jackson. Episode 29 explains in depth how to troubleshoot such dependency issues. Looking at the dependency tree I noticed that Cheshire was pulling in some fasterxml jackson libraries, and Transit was pulling in another one, at a different version. I added an exclusion on transit-clj, and the problem went away.

(defproject booklog "0.1.0-SNAPSHOT"
  :dependencies [,,,
                 [com.cognitect/transit-clj "0.8.313" :exclusions [com.fasterxml.jackson.core/jackson-core]]
                 ,,,
                 ])

If you checked out the right branch then all of this has been addressed already, so just open application.clj, and start up your REPL. While that’s booting up let’s talk about the why, what and how of OAuth.

Every time you register with a website, you are trusting that website with your credentials. But sometimes these credentials are not really stored properly and, if the website gets hacked, get exposed to the world. If you, by any chance, are using the same credentials somewhere else, you are in big trouble as hackers could impersonate you.

Unfortunately, data breaches are becoming more and more common. According to the website Have I been Pwned? there have been almost 400 data breaches and more than 8 billion credentials leaked, maybe yours included.

As a user, one way to solve this problem is by using a password manager like LastPass or 1Password.

These make it easy to have a different and unique password for every website. So that even if your credentials for one site are leaked, they can not be used anywhere else.

In an ideal world people would never reuse passwords, but this is not an ideal world. We can protect our users against themselves by not storing sensitive information in the first place, because what the application doesn’t know, it also can’t leak.

Take a look at the Gitlab login page. Below the username and password fields, you can see several of the buttons that I mentioned earlier. If you were to click on the button that says “Sign in with Google”, you are essentially saying to the website: “You can access Google on my behalf, and retrieve my identity details.”.

In this way, you just need to be logged in to Google and then you can log in to any other website, no extra password needed. This is at a high level, how OAuth works.

You should have a waiting REPL by now. Start the application using the (go) function defined in user.clj.

If you open the app at http://localhost:1234 you’ll already find a link that says ‘Log in with Google’, but it doesn’t do anything yet. Indeed, clicking the link brings you to a very bare 404 page.

user> (go)

Ring-oauth2 is implemented as a ring middleware, namely the wrap-oauth2 function in the ring.middleware.oauth2 namespace. Require the function, and then add it to the middleware stack. wrap-ouath2 relies on the parameter parsing provided by wrap-params, which is one of the many middlewares added by wrap-defaults, which is why I’m placing it before wrap-defaults.

If that doesn’t make sense then remember that middlewares are wrapped around the HTTP handler like layers of an onion. Requests arrive at the outside layer first, meaning the bottom middleware in this list, and are passed up to the top. Responses go the other way, outside in, or top to bottom.

If you’re a bit fuzzy about Ring and Ring middleware, then check out episode 13, 16 and 17 about Ring. This app also makes use of Component and System, which are covered in episode 26.

(ns booklog.application
  (:require [,,,
             [ring.middleware.oauth2 :refer [wrap-oauth2]]
             ,,,]))

(defn app-system [config]
  {,,,
   :middleware
   (new-middleware {:middleware [wrap-render-views
                                 [wrap-authentication (backends/session)]
                                 [wrap-oauth2 ,,,]  ;; <-------
                                 [wrap-defaults site-defaults]
                                 wrap-with-logger
                                 wrap-gzip
                                 prone/wrap-exceptions]})
   ,,,
   })

To configure ring-oauth2 you give it a map. Each map entry represents an OAuth provider. The key is just a name you choose to identify the provider, the value is a map with that provider’s configuration. Most of this episode will be spent explaining and implementing this configuration.

I already included a stub configuration in the oauth-config function above, so uncomment that, and then pass that to wrap-oauth2.

(ns booklog.application
  ,,,)

(defn oauth-config []
  {:google
   {:authorize-uri    "https://accounts.google.com/o/oauth2/v2/auth"
    :access-token-uri "https://www.googleapis.com/oauth2/v4/token"
    :client-id        <CLIENT-ID>
    :client-secret    <CLIENT-SECRET>
    :scopes           ["..."]
    :launch-uri       "..."
    :redirect-uri     "..."
    :landing-uri      "..."}})

(defn app-system [config]
  {,,,
   :middleware
   (new-middleware {:middleware [,,,
                                 [wrap-oauth2 (oauth-config)]
                                 ,,,]})
   ,,,
   })

Let’s have a better look at this configuration. The keys :authorize-uri and :access-token-uri are fixed due to the choice of Google as identity provider. These are values I found on their online documentation.

Next you need a :client-id and a :client-secret. To get these you will have to register your application with Google.

This will look a little different for other providers, but the principles are the same. Even Google’s interface might look different to you, depending on when you are watching this, but you should be able to find similar steps. In that case make sure to add a comment below for the learners who come after you!

Start by going to the Google Developer Console at console.developers.google.com, and logging in with your Google account. Make sure you are in the Developers Console, not the Google Cloud Console.

Click on the “Select a Project” dropdown and choose “New Project”. Now give it a project identifier, like booklog-oauth2, and hit “Create”.

This may take a moment, once it’s done you should see your project id at the top.

Next you need to configure the “OAuth Consent Screen”, which you find here on the left. This is what people will see before deciding to grant you access to their private data, so it’s important. Make sure the name is something people will recognize.

If you want to access additional APIs on behalf of the user then you need to add those to the “scopes”. We only want to know a user’s email address, so the default scopes are fine.

Before going live you should also put in links to your privacy policy and terms of service. For now just to try it out all you need is the application name.

Now you can generate a Client ID and Client Secret. Click on “Create credentials”, choose “OAuth Client ID”, and then select “Web application” as application type.

Start by giving the Client ID a name, we’ll just call it “booklog-dev”. We can create a second Client ID once we put this thing into production.

You also need to fill in the “origin” of your application. An “origin” in the context of the web security uniquely identifies an application. It is a combination of a protocol, either http or https, a domain name, and optionally a port number. So it looks like a URL without a path. Only requests coming from this origin will be accepted.

The booklog app runs on port 1234, so I’m using http://localhost:1234 as the origin. Note that you need to press enter to correctly register the URL in this form.

Add http://localhost:1234/oauth/google/callback to the Authorised Redirect URIs. We will see what this url does in just a moment. Also here, keep in mind to press Enter.

When you are all done, click on “Create”

What pops up are your Client ID and Client Secret, the exact things you needed for your configuration. Copy them somewhere as they are the keys to the whole process. These two strings are going to be used to authenticate your application with google, so you don’t want to leak these.

You also see this warning saying you only get 100 “sensitive scope logins”, after that you need to get your application approved. Luckily we’re not accessing any sensitive scopes, so that does not apply to us.

There’s also the option to download your client details as a JSON file, here you can see what’s in there. Here you actually find the majority of the things that need to be configured in your app. I’m going to configure these manually to explain what each thing is, but if you’re having trouble gettings things to work, then compare the values you have with what’s in this JSON. Maybe you made a simple typo.

{"web"
 {"auth_uri"                    "https://accounts.google.com/o/oauth2/auth"
  "token_uri"                   "https://oauth2.googleapis.com/token"
  "client_id"                   "712138220412-bj7ndil8hejeva95nmnhpa9qu734edvi.apps.googleusercontent.com"
  "client_secret"               "bDaCpi-7hE5AMg2Lea_OQraT"
  "project_id"                  "booklog-oauth2-251812"
  "auth_provider_x509_cert_url" "https //www.googleapis.com/oauth2/v1/certs"
  "redirect_uris"               ["http://localhost:1234/oauth/google/callback"]
  "javascript_origins"          ["http://localhost:1234"]}}

Now you can fill out the client-id and client-secret in your configuration. In a real application you wouldn’t want to hard-code them like this, instead getting them for instance from environment variables, or from a secure secret store like Hashicorp Vault, but for now this will do fine.

Scopes are the things we want permission for to access them. This needs to be a subset of the scopes you configured in the consent screen, as these are the only things a user will grant you access to. In this case all you care about is the "email".

(ns booklog.application
  ,,,)

(defn oauth-config []
  {:google
   {:authorize-uri    "https://accounts.google.com/o/oauth2/v2/auth"
    :access-token-uri "https://www.googleapis.com/oauth2/v4/token"
    :client-id        "12345.apps.googleusercontent.com"
    :client-secret    "xyz123"
    :scopes           ["email"]
    :launch-uri       "..."
    :redirect-uri     "..."
    :landing-uri      "..."}})

Finally there are these three URIs left, these are three endpoints our application provides, which all play a part in the OAuth 2 authorization process. Luckily you only need to implement one of them, the others will be handled by ring-oauth2.

It starts with the :launch-uri. This is where a user goes when they want to log in using Google. We already have a button which sends the user to /log-in-with-google, so you can fill that in here.

Save your changes and call the (reset) function to make sure they are picked up, and let’s try that out.

:launch-uri "/log-in-with-google"

user> (reset)

OAuth contains a lot of redirecting back and forth, to make it easier to debug things and to understand what’s happening it’s good to make those redirects visible.

Open your browser console, make sure it’s logging HTTP requests, and enable “Persist Logs”. You might have to search around a bit to see where your browser hides this setting.

Now click on the Login with Google link. You’ll get an error page from google, which is fine, we’re not done yet configuring everything, but let’s have a better look at the link that ring-oauth2 generated, in particular the parameters added to the URL.

This redirect went to the authorize-uri you configured, and includes a bunch of information that we pass on to google to kickstart the process.

OAuth supports a few different ways of authenticating, specified by the response_type parameter. We are interested in getting an access code.

The client_id makes sure google knows who we are.

Once permission is granted we ask to send the user back to us, namely to this :redirect-uri. We’ll have to configure that next.

We are requesting access to a single scope, namely we want permission to ask for the user’s email address.

state is just a random string. When the user returns that string will be passed back to us, so we can verify that we actually initiated this request. This protects against a type of vulnerability known as Cross Site Request Forgery or CSRF.

https://accounts.google.com/o/oauth2/v2/auth
- client_id=210057595309-qkju8agmnmo8dj2h2246fu4u0don6tgm.apps.googleusercontent.com
- scope=email
- redirect_uri=http://localhost:1234/...
- state=P-8Cv2qSKZBT
- response_type=code

The problem now is that this :redirect-uri does not match the ‘Authorised Redirect URIs’ I entered in the Consent Screen configuration, so let’s fix that. I don’t have to enter the full URL here, just the path is fine.

When configurting the consent screen it’s possible to enter multiple ‘Authorised Redirect URIs’. This can be useful when you have for instance a staging and a production server. ring-oauth2 will specify which one should be used when sending the user off to Google.

:redirect-uri "/oauth/google/callback"

Call (reset) again, and try again to “log in with google”. The first time you do this you will have to log into Google if you aren’t already, and then accept the consent screen.

Finally Google redirects the user back to the app, to the configured :redirect-uri. Things still aren’t quite working yet, instead ring-oauth2 is responding with a “400 Bad Request”. We’ll look into that next, let’s first look at what Google gave back to us.

state is the random strings you passed in the original request, that’s simply being passed back.

code is the access code, sweet! so far so good. You’ll still have to get this exchanged for an access token, we’ll look into what the difference is, but at least it seems google will let us in.

There’s also a list of scopes you get access to, and then a few extra parameters which ring-oauth2 just ignores, so I’ll do the same.

http://localhost:1234/oauth/google/callback
- state: AU3POjHwBhnJ
- code: 4/qgGb9MjvgxipvYtMJiA_8_Co1SuN8frrkNzv9DbYoEqodBPKf6gRpgoS5SGTS0llIjUbaOGbKk5vFH7wIzcJp9k
- scope: email+openid+https://www.googleapis.com/auth/userinfo.email
- prompt: none
- authuser: 0
- session_state: 3e24897a120c151f56edeb79e15f03ab7486c305..7b0f

So that actually looks pretty ok, but ring-oauth2 isn’t happy. It’s giving us a robot voice saying 400 Bad Request, that’s not too helpful.

The problem is the state parameter. ring-oauth2 saves this in the session, so it can check it later when the request comes back. Ring sessions rely on cookies, and by default it creates these cookies setting the SameSite attribute to strict. This tells the browser to only send the cookie back to the server when the request originated from the site which set the cookie. In other words, because we are being redirected from Google the browser is refusing to send us back our cookie, and so there is no way to access the session to verify the state parameter.

(def site-defaults
  {,,,
   :session   {:flash true
               :cookie-attrs {:http-only true, :same-site :strict}}
   })

Once you know what the problem is it’s easy to fix, by relaxing slightly this stance on cookies, changing site-defaults to set the :same-site cookie attribute to :lax

(defn app-system [config]
  (component/system-map
   ,,,
   :middleware (new-middleware {:middleware [wrap-render-views
                                             [wrap-authentication (backends/session)]
                                             [wrap-oauth2 (oauth-config)]
                                             [wrap-defaults
                                              (-> site-defaults
                                                  (assoc-in [:session :cookie-attrs :same-site] :lax))]
                                             wrap-with-logger
                                             wrap-gzip
                                             prone/wrap-exceptions]})
                                             ))

Let’s try that once more. Save your changes, reset your app, and try logging in. By now you’ve already logged in to google and given your consent, so this time the redirection back and forth happens immediately.

user=> (reset)

You can see from the browser console that ring-oauth2 is happy this time, it has handled the callback request, and has done one more redirect, to the :landing-uri.

Behind the scenes it has performed the final part of the handshake by exchaning the Access Code for an Access Token.

Now it’s on you to implement the last step of the process: using the access token to fetch the user’s email, and using that to authenticate them in the app.

So what’s the difference between the authorization code, and the access token? The authorization code is short lived, and is passed to us in a GET parameter as part of this redirect back from the OAuth provider. However you can’t use this code yet to access any APIs. The reason is that passing this as a GET parameter isn’t very secure. This was especially true before HTTPS became ubiquitous, but it’s still true today. Browser vulnerabilities and JavaScript injection attacks could quite easily find this code in the browser’s history.

So before you can do anything with the auth code you need to exchange it for a token. This happens behind the scenes, directly between ring-oauth2 and google. This includes the client secret, so Google knows for sure it’s us and not some man in the middle.

The result is an access token, also known as a “Bearer Token”, and optionally a refresh token. Now whenever you want to perform an API request on behalf of the user you can include this token. These tokens do have an expiration date, usually around 1 hour, after which you can use the refresh token to renew them.

I’ll leave a link to this blog post in the show notes, this talks in much more depth about how this process works exactly.

So long story short, once Google hands control over to ring-oauth2 it performs this token exchange in the background, and finally does one more redirect, to this :landing-uri, so let’s change that to something more sensible, like /oauth/google/done. You’ll have to (reset) once more after this, since the middleware needs to get reconfigured.

:landing-uri "/oauth/google/done"

The first two of these three routes have all been handled by ring-oauth2 automatically, but now that the process has reached its conclusion it’s up to you to implement the last step.

So open the booklog.auth.routes namespace to add a route handler for this “landing uri”.

I know that at this point ring-oauth2 has added the Access Token somewhere in the request map. I could look at the documentation to figure out where it is, but I find it even more helpful sometimes to just look at the data. So what I want to do in this route is to simply show a pretty printed version of the request map.

Simple ring responses like these are easy enough to write by hand. I’m responding with a “200 OK” HTTP status code, telling the browser that I’m just sending plain text back. The pprint function writes to standard out, so I’m using with-out-str to capture it and turn it into a string.

(ns booklog.auth.routes
  ,,,)

(defn auth-routes [{:keys [spicerack] :as endpoint}]
  (let [users (sr/open-hashmap (:db spicerack) "users")]
    (routes
     ,,,

     (GET "/oauth/google/done" req
       {:status 200
        :headers {"Content-Type" "text/plain"}
        :body (with-out-str (clojure.pprint/pprint req))})

        )))

Re-evaluate those routes, and then click again on “Login with Google” in the homepage of the Booklog app.

You should see a page with a lot of text, and somewhere in there you’ll find a key named :oauth2/access-tokens. Remember that you can configure ring-oauth2 to work with multiple providers. For each provider that the user has authenticated with you can find a map here with a token and expiration data.

{,,,
 :oauth2/access-tokens
 {:google
  {:token
   "ya29.GltVB5j7d89J7NKQ_AEUhw0z8L4xRlvxfYM6cTv-o837Qh-wZEFPkz-tLP5knNpgpdr6yaN812RjWQ8ggWkMnpHuKgsalOwEEzCy7PEnNAQi0PuvKONbnlNUKPqb",
   :expires
   #object[org.joda.time.DateTime 0x14b1cb3d "2019-07-30T16:17:15.079Z"],
   ,,,}}}

Now it should be clear how to pull out that token. With a get-in you first grab the access tokens map, then find the entry for Google, and finally get the token itself. Return the token from the handler, to make sure you got it right. Evaluate that, and refresh the browser.

(GET "/oauth/google/done" req
  (let [token (get-in req [:oauth2/access-tokens :google :token])]
    token))

Everything is looking great so far, and you’re almost done, time to finally fetch that email address. I’ll create a helper function for that called google-fetch-email, which queries Google’s userinfo API endpoint.

Require clj-http.client, aliased to http, and define a var named userinfo-url, just copy the URL over from the transcript.

(ns booklog.auth.routes
  (:require [,,,
             [clj-http.client :as http]
             ,,,]))

(def userinfo-url "https://www.googleapis.com/oauth2/v1/userinfo")

Now you can make the API call. Pass in the URL, and a map of options, containing the access token as a query parameter, and specifying that the output should be parsed as JSON.

This last part will differ from provider to provider. OAuth only goes as far as getting you an Access Token, and you’ll need to consult the API documentation to see which endpoint to call, and how to use the token to authenticate yourself. In Google’s case you can pass in the token as a query parameter.

The response of this call will be a map with :body, :headers, and :status, where the body is the parsed JSON response, so with get-in you can pull out the user’s email address.

(defn google-fetch-email [token]
  (-> (http/get userinfo-url {:query-params {:access_token token} :as :json})
      (get-in [:body :email])))

Let’s try that out in the handler! Call google-fetch-email, passing it the token, and assigning the result to a local var, which I’ll return again to check the result in the browser.

(GET "/oauth/google/done" req
  (let [token (get-in req [:oauth2/access-tokens :google :token])
        email (google-fetch-email token)]
    email))

Finally to hook this up with buddy-session you need to set the :identity in the session. This is all explained in much more detail in the Buddy episode. Pull out the session from the request, and add in the new identity. Then add some metadata to make sure this session gets a new session id, which is to prevent Session Fixation attacks.

Finally redirect back to the front page. Redirect returns a ring response map, so with assoc you can add in the new session map, and show the user a friendly welcome message after the redirect.

(GET "/oauth/google/done" req
  (let [token (get-in req [:oauth2/access-tokens :google :token])
        email (google-fetch-email token)
        next-session (-> (assoc (:session req) :identity email)
                         (with-meta {:recreate true}))]
    (-> (redirect "/")
        (assoc :session next-session)
        (flash-message (str "OAuth login successful, welcome back " email)))))

Ok, try this one more time…

And it works! Congratulations, you just implemented OAuth with Google on your website.

Before wrapping up I’ll share a few tips for when you decide to put this into production.

When you are ready for prime time, you need to remember that you need also to update details on Google Developers console. For example, if your application can be found at mybooklog.com, you need to add http://mybooklog.com and http://mybooklog.com/google/oauth/callback to the appropriate sections.

If you use HTTPS, and these days there is really no reason not to, then switch :redirect-uri and :landing-uri to absolute urls. This way you make sure the redirect coming back from the identity provider uses the right protocol.

(defn oauth-config []
  {:google
   {:authorize-uri    "https://accounts.google.com/o/oauth2/v2/auth"
    :access-token-uri "https://www.googleapis.com/oauth2/v4/token"
    :client-id        <CLIENT-ID>
    :client-secret    <CLIENT-SECRET>
    :scopes           ["email"]
    :launch-uri       "https://mybooklog.com/oauth2/google"
    :redirect-uri     "https://mybooklog.com/oauth2/google/callback"
    :landing-uri      "https://mybooklog.com/oauth2/google/success"}})

You should now have a good understanding of the basic mechanics of OAuth 2.0. As you can imagine, there’s a lot more to the standard and there’s definitely details I did not cover, in order to keep things simple.

As I have already mentioned, ring-oauth2 is also pretty self contained, so it doesn’t hurt to take a peek at the source code and see the the magic that happens behind the curtain.

In closing, keep in mind that OAuth is not for everyone. By using OAuth the third party provider now knows that the user is using your site, and vice versa. For this reason privacy conscious users might avoid this option altogether, keeping their online identies clearly separated. So while OAuth based logins add a nice convenience, I don’t recommend dropping password based logins just yet.

Show Notes

browse source code

Cheat sheet

(require '[ring.middleware.oauth2 :refer [wrap-oauth2]]
          [clj-http.client :as http])

(defn oauth-config []
  {:google
   {:authorize-uri    "https://accounts.google.com/o/oauth2/v2/auth"
    :access-token-uri "https://www.googleapis.com/oauth2/v4/token"
    :client-id        <CLIENT-ID>
    :client-secret    <CLIENT-SECRET>
    :scopes           ["email"]
    :launch-uri       "https://mybooklog.com/oauth2/google"
    :redirect-uri     "https://mybooklog.com/oauth2/google/callback"
    :landing-uri      "https://mybooklog.com/oauth2/google/success"}})

(def handler
  (-> ring-handler
      (wrap-oauth2 (auth-config))))

(defn google-fetch-email [token]
  (-> (http/get userinfo-url {:query-params {:access_token token} :as :json})
      (get-in [:body :email])))

(GET "/oauth/google/done" req
  (let [token (get-in req [:oauth2/access-tokens :google :token])
        email (google-fetch-email token)
        next-session (-> (assoc (:session req) :identity email)
                         (with-meta {:recreate true}))]
    (-> (redirect "/")
        (assoc :session next-session))))

Passwordless Authentication With ring-oauth2