rintaun's thoughts not cool enough for a tagline

RESTful Authentication and the Nature of Statelessness

I’m in the middle of a complete overhaul of the software that I use for my research, Co-Chu. Its exact use is beyond the scope of this article, but it’s basically a single-page AngularJS application built on a (mostly) RESTful API.

Anyway, I hope to eventually release it publicly, and to that end, I decided to undertake a massive cleanup of the codebase. Unfortunately, over the past three years, the aim and scope of the project have changed drastically, so that is going to involve rewriting big chunks of the code so that it is, once again, beautiful.

But I digress.

To bring this back to the topic of the post, as I’ve been working on this overhaul, I’ve been thinking about a lot of things, particularly documentation, application architecture, and API design.

While thinking about the necessary task flows and how to redesign the API endpoints to best suit them, I came across a conundrum: what is the proper way to perform authentication in a RESTfully?

RESTful Authentication

There are (probably unsurprisingly) an endless number of opinions on what REST really means and what that means for authentication. Some people have an almost dogmatic devotion to statelessness while seeming to have no real understanding as to what that means. There seems to be a lot of cognitive dissonance about what a proper authentication scheme looks like in a REST application.

I came to this subject by happenstance. I was young and innocent and now my thoughts are invaded constantly with just how little people (the ones who are talking, anyway) seem to understand REST and authentication.

Let’s break down the problem: when people talk about authentication, they are usually actually talking about two separate functions (even though they might all be bundled together): Authentication, and Authorization. In fact, they are often abbreviated, collectively, simply to “auth”. But they are in fact two separate functions and we need to talk about them separately if we’re going to maintain any reasonable degree of sanity in our discourse. So here goes:

Authentication

Authentication is the process of verification that an individual, entity or website is who it claims to be. Authentication in the context of web applications is commonly performed by submitting a user name or ID and one or more items of private information that only a given user should know.1

Authorization

Access Control, also known as Authorization — is mediating access to resources on the basis of identity and is generally policy-driven (although the policy may be implicit). It is the primary security service that concerns most software, with most of the other security services supporting it.2

As these definitions show, both Authentication and Authorization are essential in any application where user identity and privileges are at all relevant. And yet, many sources seem to pretend that Authorization does not exist when it comes to Authentication in a RESTful application.

Cognitive Dissonance

The issues surrounding the jumbled conversation about RESTful Authentication and Authorization were brought to my attention by a Stack Overflow question and its answers.3 The question was stated as follows:

What does RESTful Authentication mean and how does it work? I can’t find a good overview on Google. My only understanding is that you pass the session key (remeberal) in the URL, but this could be horribly wrong.

Oh, the innocent days of 2008, and the horror that the internet hath wrought in years since. I don’t know that this user had any idea what they were getting themselves into. There are so many different, conflicting ideas on this page that it’s hard to know where to begin. And the worst part is, people on both sides of the fence seem to hold, simultaneously, a variety of conflicting ideas.

At the core of the controversy are the ideas of “Session” and “Statelessness” – the former because it is how, traditionally, Authentication and Authorization are maintained in an application; and the latter because REST is supposed to be “stateless”. So let’s go ahead and define these terms as well, to get it out of the way:

Session

[…] a semi-permanent interactive information interchange […] between two or more communicating devices, or between a computer and user (see Login session). A session is set up or established at a certain point in time, and then torn down at some later point. An established communication session may involve more than one message in each direction. A session is typically, but not always, stateful, meaning that at least one of the communicating parts needs to save information about the session history in order to be able to communicate, as opposed to stateless communication, where the communication consists of independent requests with responses.4

Stateless

[…] no session state is allowed on the server component. Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server. Session state is kept entirely on the client.5

Of course, even defining these concepts, there seems to be a ton of confusion as to what the Stateless constraint of REST actually entails (for example, the following questions/answers: 1, 2, 3, 4, and 5 among oh so many others). Just Google around a bit. It’s saddening, really.

There is a common takeaway that people seem to have from REST’s Stateless constraint: Session is bad. Session is not REST. This idea is summed up quite well in various places, but an answer on Stack Overflow says it quite clearly:

In REST applications, there must not be session state stored on the server side. Instead, it must the handled entirely by the client.

So, if there’s session state on the server, it’s not REST.6

Up to this point, the user is totally in the clear. A-OK. However, they continue:

So, your server must not store any session state and, consequently, must not issue session identifiers.

At risk of spoiling the climax of this article: these people are fundamentally misunderstanding Statelessness.

The Nature of Statelessness

Returning to the definition of Statelessness:

Each request from client to server must contain all of the information necessary to understand the request, and cannot take advantage of any stored context on the server.

I would argue that one of two things is true:

  1. Applications which serve dynamic data (e.g. from a database or other data store) cannot be RESTful; or
  2. This does not mean what everyone seems to think it means.

For example, take the following flow:

sequenceDiagram
    Note over User: Authenticate
    Note over User,Server: POST /sessions
    User->>Server: Username credentials
    Note over Server: Create session token
    Server-->>User: Session token
    loop Subsequent requests
        Note over User: Request resources
        Note over User,Server: GET, POST, PUT, PATCH, etc. /...
        User->>Server: Data including session token
        Note over Server: Authorize with session token
        Server-->>User: Requested data
    end
    Note over User: Destroy authentication token
    Note over User,Server: DELETE /sessions/{tok}
    User->>Server: Anything
    Server-->>User: Success

Now, this flow is relatively common, and I saw some variation of this in a couple different places. This is, in fact, the flow that I use in my own web applications (and will continue to use). However, it is widely regarded as “stateful” and “not REST”. However, I strongly disagree with this assertion, to which one user responded quite aptly:

The server has no “authenticated” state. It receives information via hypermedia and has to work with it to return what was requested. Nothing less, nothing more. If the resource is protected and requires authentication and authorization, the provided hypermedia must include that information. I don’t know where the notion that authenticating a user before returning a resource means that the server is tracking state comes from. Providing a username and a password can very well be thought of as simply providing more filtering parameters.7

This user was referring specifically to HTTP Basic authentication, but the same is true of the “session token” described above. If you prefer, think of it as an “authentication token” or a “user identity token”. In this case, the “session” is not stateful. It is simply a resource referred to in the query. Now, I can hear the people in the back starting to get restless…

[…] and cannot take advantage of any stored context on the server.

“But what about this?!” they ask, pointing again to Fielding’s dissertation. Well, let’s talk about that.

It is entirely possible to interpret this in a way that precludes the above flow from being RESTful. However, I would argue that it not only precludes any type of Authorization at all, but might also preclude any REST system from existing at all, ever.

If the session token stored on the server (actually, not even, but I’ll come back to this) is state, then how is that any different from a username and password stored on the server? I would argue that it is not. Both are simply resources. Even using HTTP Basic authentication, where the username and password are sent with each request, the server would have to perform some kind of specialized operation with/on that data (i.e. Authorization) before serving the requested resources. In fact, the required operations are in nature almost identical to the session token model – while also being inferior in almost every single way. More on this later.

This seems to suggest that any Authentication in a REST system isn’t possible. Which would in and of itself render REST essentially useless, at least to my understanding.

Yet, there are quite a large number of people working to find the perfect, stateless, supposedly “RESTful” authentication scheme. So let’s talk about that, and how they fail.

Authentication and Authorization Pitfalls

There are several solutions to Authentication (notice that I did not mention Authorization) which are variously declared RESTful – or not, depending on the author. Because nobody really has any idea what they’re talking about. These are HTTP Basic Authentication, OAuth, OpenID, and JSON Web Tokens. Maybe there are other major solutions, but I’m not aware of any. But I’m going to go ahead and put the cart before the horse again here and say up front: none of these solutions fully address the issue or actually address another similarly important but entirely different problem. The issue, of course, being Authentication and Authorization.

One more thing to note before I get into this. Despite any claims about security made by various Authentication schemes, whenever any kind of Authenticating credential is exchanged, HTTPS (i.e. TLS) should be used.

HTTP Basic Authentication

HTTP Basic Authentication8 essentially sends the username and password in the HTTP headers with every request. In a RESTful application, it is functionally identical to the Session Token flow described above, minus the layer of abstraction that the Session Token provides. However, in an otherwise secure application, this has two major flaws: security and efficiency.

Security

Even if all requests are performed over HTTPS, there is the possibility (if remote) of information leaking. When you are sending your plaintext credentials with every request, that is problematic, because your identity could become compromised without you knowing.

Efficiency

I hope that in 2017, we are all storing our passwords properly, whether that is with bcrypt, scrypt, or something else. Even a single round of md5 hashing is (if only nominally) better than nothing. But therein lies a major pitfall of HTTP Basic authentication: in order to verify the user’s identity, password hashing, encryption/decryption, or some other similarly (purposefully) slow task must be performed on every request. This doesn’t scale, which defeats one major purpose of REST.

OAuth 2.0

I’m treating OAuth and OpenID separately here because they actually have different goals, even though OpenID is built on top of OAuth.

OAuth was not actually designed for Authentication. It is an Authorization protocol, with the purpose of giving Authorizing 3rd parties to use your restricted materials. To use OAuth for Authentication, one would simply stop the normal flow in the middle and (probably) never make use of the restricted materials. However, this is problematic; OAuth was not built for Authentication, and using it as such may open you up to a variety of possible problems that you would otherwise not have. Plus, Authorization to your own app always has to be performed locally (or by another service that you control) anyway.

OpenID

OpenID is actually an identity verification (i.e. Authentication) protocol, which is built on top of OAuth 2.0. It’s a great choice for Authentication, though it does have its own particular complexities. However, it’s primarily useful when using a 3rd party Authorization server (because it’s OAuth underneath) for Authentication. If you want (or need) to host credentials locally, rolling your own OpenID Authorization server implementation is… a bit much.

Who knows. Maybe you’re Google, dear reader, and I am just ignorant. But it’s certainly overkill for most web applications. In any case, if you’re fine okay with credentials that are not local to your application, go for it. Buuuuuuuut…. you’ll still need to have your own Authorization layer.

JSON Web Tokens

This one is actually a bit baffling to me. JSON Web Tokens are essentially the identity verification component of OpenID, but I’ve seen them described as, essentially, an end-all solution to Authentication and Authorization in a number of places, including JWT.io, which is devoted to JSON Web Tokens. Specifically, they claim:

Self-contained: The payload contains all the required information about the user, avoiding the need to query the database more than once.

And, further:

Authentication: This is the most common scenario for using JWT. Once the user is logged in, each subsequent request will include the JWT, allowing the user to access routes, services, and resources that are permitted with that token. Single Sign On is a feature that widely uses JWT nowadays, because of its small overhead and its ability to be easily used across different domains.

The emphasis in both cases is mine. I’m going to come right out and say that using JSON Web Tokens in the manner described above is extremely reckless and should never be done. I agree that JSON Web Tokens can be used for Authentication; their entire point is identity verification, so more power to you here. However, and this is essential to remember: do not ever use JSON Web Tokens for local Authorization unless you want to completely lose control of access to your application.

Indeed, a JSON Web Token can contain user information. Indeed, that user information can include the user’s privileges. Indeed, JSON Web Tokens have as one of their core claims an expiration date. So what is the problem?

The problem is that if you do not perform any Authorization verification and simply trust what is in the JSON Web Token, then until that token expires, that user’s privileges cannot change, nor can they be revoked. And if you add some mechanism to revoke tokens (or specific privileges within tokens) you may as well just be performing Authorization verification anyway.

…And so you Authorize

No major system offers a good solution to Authorization. They can’t. It’s your app. You decide who can access what. On top of that, it seems impossible to imagine a system which creates the kind of quasi-stateless Authorization that so many people out there seem to demand. But then, they’re wrong about what it means to be stateless anyway, so who cares, really?

The take away from all of this is that you’re going to have to do authorization either within your application, or you’re going to have to create a separate application to do it for that other application. It’s the same thing, just a bit different structurally.

And Finally, to the Conclusion

So what even is anything anymore? Let’s summarize.

REST is supposed to be stateless. That means that “each request from client to server must contain all of the information necessary to understand the request”. Storing session state in the server (i.e. application) does indeed add a lot of unnecessary complexity to the system, so I’ll agree that it is to be avoided – especially because it isn’t generally actually necessary. But it is the state, not the session, that is bad. A session, as we saw in the definition above, is just a conversation. A stateless session is one in which a participant can understand each exchange fully without any context.

Session tokens (or auth tokens, if you prefer) are simply one more resource that identifies the user. Whether you generate them in your app or defer that process to a 3rd party, it’s all the same. Plus, since I promised to come back to this, you’re probably not actually storing that on the application – it may be on the same physical machine, but it is almost certainly contained in a separate service, such as a database. So you’re basically relying on a 3rd party, at least from the perspective of your application, anyway. Sure, 6ou could abstract this even further into some kind of authentication/authorization microservice, but unless you have more than a few applications using the same underlying system, that probabl6 adds more complexity and problems than it removes.

So I’m going to go with what I have. And you should do what works for you, too. And if you tell me it isn’t REST, I hope you have a very good argument ready.

That’s it. That’s the end. Honestly, you should just do whatever works for your application. Striving for some ideal that either cannot exist or that nobody understands is misguided at best and very destructive (I’m talking to you, guy who wanted everyone to use JSON Web Tokens for Authorization) at worst.

Until next time!


References

  1. Authentication Cheat Sheet on OWASP. Last modified April 21, 2017. 

  2. Category:Access Control on OWASP. Last modified June 1, 2016. 

  3. Jim Keener (2008, November 26). RESTful Authentication on Stack Overflow

  4. Session (computer science) on Wikipedia. Last modified June 27, 2017. 

  5. Roy Thomas Fielding (2000). Architectural Styles and the Design of Network-based Software Architectures (doctoral dissertation), University of California, Irvine. 

  6. Cássio Mazzochi Molin (2015, December 7). Answer to How to understand “RESTful API is stateless”? on Stack Overflow

  7. mike (2014, August 27). Comment on RESTful Authentication on Stack Overflow

  8. J. Reschke (2015, September). The ‘Basic’ HTTP Authentication Scheme [RFC7617]. DOI 10.17487/RFC7617