:::: MENU ::::

Wednesday, August 19, 2015

Recently I was working on securing ASP.NET Web API HTTP service that will be consumed by a large number of terminal devices installed securely in different physical locations, the main requirement was to authenticate calls originating from those terminal devices to the HTTP service and not worry about the users who are using it. So first thing came to my mind is to use one of the OAuth 2.0 flows which is Resource Owner Password Credentials Flow, but this flow doesn’t fit nicely in my case because the bearer access tokens issued should have expiry time as well they are non revocable by default, so issuing an access token with very long expiry time (i.e one year) is not the right way to do it.
After searching for couple of hours I found out that the right and maybe little bit complex way to implement this is to use HMAC Authentication (Hash-based Message Authentication Code).

The source code for this tutorial is available on GitHub.

What is HMAC Authentication?

It is a mechanism for calculating a message authentication code using a hash function in combination with a shared secret key between the two parties involved in sending and receiving the data (Front-end client and Back-end HTTP service) . The main use for HMAC to verify the integrity, authenticity, and the identity of the message sender.
So in simpler words the server provides the client with a public APP Id and shared secret key (API Key – shared only between server and client), this process happens only the first time when the client registers with the server.
After the client and server agrees on the API Key, the client creates a unique HMAC (hash) representing the request originated from it to the server. It does this by combining the request data and usually it will contain (Public APP Id, request URI, request content, HTTP method, time stamp, and nonce) in order to produce a unique hash by using the API Key. Then the client sends that hash to the server, along with all information it was going to send already in the request.
Once the server revives the request along with the hash from the client, it tries to reconstruct the hash by using the received request data from the client along with the API Key, once the hash is generated on the server, the server will be responsible to compare the hash sent by the client with the regenerated one, if they match then the server consider this request authentic and process it.

Flow of using API Key – HMAC Authentication:

Note: First of all the server should provide the client with a public (APP Id) and shared private secret (API Key), the client responsibility is to store the API Key securely and never share it with other parties.

Flow on the client side:

  1. Client should build a string by combining all the data that will be sent, this string contains the following parameters (APP Id, HTTP method, request URI, request time stamp, nonce, and Base 64 string representation of the request pay load).
  2. Note: Request time stamp is calculated using UNIX time (number of seconds since Jan. 1st 1970) to overcome any issues related to a different timezone between client and server. Nonce: is an arbitrary number/string used only once. More about this later.
  3. Client will hash this large string built in the first step using a hash algorithm such as (SHA256) and the API Key assigned to it, the result for this hash is a unique signature for this request.
  4. The signature will be sent in the Authorization header using a custom scheme such as”amx”. The data in the Authorization header will contain the APP Id, request time stamp, and nonce separated by colon ‘:’. The format for the Authorization header will be like: [Authorization: amx APPId:Signature:Nonce:Timestamp].
  5. Client send the request as usual along with the data generated in step 3 in the Authorization header.

Flow on the server side:

  1. Server receives all the data included in the request along with the Authorization header.
  2. Server extracts the values (APP Id, Signature, Nonce and Request Time stamp) from the Authorization header.
  3. Servers looks for the APP Id in a certain secure repository (DB, Configuration file, etc…) to get the API Key for this client.
  4. Assuming the server was able to look up this APP Id from the repository, it will be responsible to validate if this request is a replay request and reject it, so it will prevent the API from any replay attacks. This is why we’ve used a request time stamp along with nonce generated at the client, and both values have been included into HMAC signature generation. The server will depend on the nonce to check if it was used before within certain acceptable bounds, i.e. 5 minutes. More about this later.
  5. Server will rebuild a string containing the same data received in the request by adhering to the same parameters orders and encoding followed in the client application, usually this agreement is done up front between the client application and the back-end service and shared using proper documentation.
  6. Server will hash the string generated in previous step using the same hashing algorithm used by the client (SHA256) and the same API Key obtained from the secure repository for this client.
  7. The result of this hash function (signature) generated at the server will be compared to the signature sent by the client, if they are equal then server will consider this call authentic and process the request, otherwise the server will reject the request and returns HTTP status code 401 unauthorized.
Important note:
  • Client and server should generate the hash (signature) using the same hashing algorithm as well adhere to the same parameters order, any slight change including case sensitivity when implementing the hashing will result in totally different signature and all requests from the client to the server will get rejected. So be consistent and agree on how to generate the signature up front and in clear way.
  • This mechanism of authentication can work without TLS (HTTPS), as long as the client is not transferring any confidential data or transmitting the API Key. It is recommended to consume it over TLS. But if you can’t use TLS for any other reason you will be fine if you transmit data over HTTP.
Sounds complicated? Right? Let’s jump to the implementation to make this clear.
I’ll start by showing how to generate APP Id and strong 265 bits key which will act as our API Key, this usually will be done on the server and provided to the client using a secure mechanism (Secure admin server portal). There is nice post here explains why generating APP Ids and API Keys is more secure than issuing username and password.
Then I’ll build a simple console application which will act as the client application, lastly I’ll build HTTP service using ASP.NET Web API and protected using HMAC Authentication using the right filter “IAuthenticationFilter”, so let’s get started!

The source code for this tutorial is available on GitHub.

Section 1: Generating the Shared Private Key (API Key) and APP Id

As I stated before this should be done on the server and provided to the client prior the actual use, we’ll use symmetric key cryptographic algorithm to issue 256 bit key, the code will be as the below:
And for the APP Id you can generate a GUID, so for this tutorial let’s assume that our APPId is:4d53bce03ec34c0a911182d4c228ee6c  and our APIKey generated is: A93reRTUJHsCuQSHR+L3GxqOJyDmQpCgps102ciuabc= and assume that our client application has received those 2 pieces of information using a secure channel.

Section 2: Building the Client Application

Step 1: Install Nuget Package
Add new empty solution named “WebApiHMACAuthentication” then add new console application named “HMACAuthentication.Client”, then install the below HTTPClient Nuget package which help us to issue HTTP requests.
Step 2: Add POCO Model
We’ll issue HTTP POST request in order to demonstrate how we can include the request body in the signature, so we’ll add simple model named “Order”, add new class named “Order” and paste the code below:
Step 3: Call the back-end API using HTTPClient
Now we’ll use the HTTPClient library installed earlier to issue HTTP POST request to the API we’ll build in the next section, so open file “Program.cs” and paste the code below:
What we implemented here is basic, we just issuing HTTP POST to the end point “/api/orders” including serialized order object, this end point is protected using HMAC Authentication (More about this later in post), and if the response status returned is 200 OK, then we are printing the response returned.
What worth nothing here that I’m using a custom delegation handler named “CustomDelegatingHandler”. This handler will help us to intercept the request before sending it so we can do the signing process and creating the signature there.
Step 4: Implement the HTTPClient Custom Handler
HTTPClient allows us to create custom message handler which get created and added to the request message handlers chain, the nice thing here that this handler will allow us to write out custom logic (logic needed to build the hash and set in the Authorization header before firing the request to the back-end API), so in the same file “Program.cs” add new class named “CustomDelegatingHandler” and paste the code below:
What we’ve implemented above is the following:
  • We’ve hard coded the APP Id and API Key values obtained earlier from the server, usually you need to store those values securely in app.config.
  • We’ve got the full request URI and safely Url Encoded it, so in case there is query strings sent with the request they will safely encoded, as well we’ve read the HTTP method used, in our case it will be POST.
  • We’ve calculated the time stamp for the request using UNIX timing (number of seconds since Jan. 1st 1970). This will help us to avoid any issues might happen if the client and the server resides in two different time zones.
  • We’ve generated a random nonce for this request, the client should adhere to this and should send a random string per method call.
  • We’ve checked if the request contains a body (it will contain a body if the request of type HTTP POST or PUT), if it contains a body; then we will md5 hash the body content then Base64 the array, we are doing this to insure the authenticity of the request and to make sure no one tampered with the request during the transmission (in case of transmitting it over HTTP).
  • We’ve built the signature raw data by concatenating the parameters (APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String) without any delimiters, this data will get hashed using HMACSHA256 algorithm.
  • Lastly we’ve applied the hashing algorithm using the API Key then base64 the result and combined the (APPId:requestSignatureBase64String:nonce:requestTimeStamp) using ‘:’ colon delimiter and set this combined string in the Authorization header for the request using a custom scheme named “amx”. Notice that the nonce and time stamp are included in creating the request signature as well they are sent as plain text values so they can be validated on the server to protect our API from replay attacks.
We are done of the client part, now let’s move to building the Web API which will be protected using HMAC Authentication.

Section 3: Building the back-end API

Step 1: Add the Web API Project
Add new Web application project named “HMACAuthentication.WebApi” to our existing solution “WebApiHMACAuthentication”, the template for the API will be as the image below (Web API core dependency checked) or you can use OWIN as we did in previous tutorials:
Step 2: Add Orders Controller
We’ll add simple controller named “Orders” controller with 2 simple HTTP methods, as well we’ll add the same model “Order” we already added in the client application, so add new class named “OrdersController” and paste the code below, nothing special here, just basic Web API controller which is not protected and allows anonymous calls (we’ll protect it later in the post).
Step 3: Build the HMAC Authentication Filter
We’ll add all our logic responsible for re-generating the signature on the Web API and comparing it with signature received by the client in an Authentication Filter. The authentication filter is available in Web API 2 and it should be used for any authentication purposes, in our case we will use this filter to write our custom logic which validates the authenticity of the signature received by the client. The nice thing about this filter that it run before any other filters especially the authorization filter, I’ll borrow the image below from a great article about ASP.NET Web API Security Filters by Badrinarayanan Lakshmiraghavan to give you better understanding on where the authentication filter resides.
Now add new folder named “Filters” then add new class named “HMACAuthenticationAttribute” which inherits from “Attribute” and implements interface “IAuthenticationFilter”, then paste the code below:
Basically what we’ve implemented is the following:
  • The class “HMACAuthenticationAttribute” derives from “Attribute” class so we can use it as filter attribute over our controllers or HTTP action methods.
  • The constructor for the class currently fill a dictionary named “allowedApps”, this is for the demo only, usually you will store the APP Id and API Key in a database along with other information about this client.
  • The method “AuthenticateAsync” is used to implement the core authentication logic of validating the incoming signature in the request
  • We make sure that the Authorization header is not empty and it contains scheme of type “amx”, then we read the Authorization header value and split its content based on the delimiter we’ve specified earlier in client ‘:’.
  • Lastly we are calling method “isValidRequest” where all the magic of reconstructing the signature and comparing it with the incoming signature happens. More about implementing this in step 5.
  • Incase the Authorization header is incorrect or the result of executing method “isValidRequest” returns false, we’ll consider the incoming request as unauthorized and we should return an authentication challenge to the response, this should be implemented in method “ChallengeAsync”, to do so lets implement the next step.
Step 4: Add authentication challenge to the response
To add authentication challenge to the unauthorized response copy and paste the code below in the same file “HMACAuthenticationAttribute.cs”, basically we’ll add “WWW-Authenticate” header to the response using our “amx” custom scheme . You can read more about the details of this implementation here.
Step 5: Implement the method “isValidRequest”.
The core implementation of reconstructing the request parameters and generating the signature on the server happens here, so let’s add the code then I’ll describe what this method is responsible for, open file “HMACAuthenticationAttribute.cs” again and paste the code below in class “HMACAuthenticationAttribute”:
What we’ve implemented here is the below:
  • We’ve validated that public APPId received is registered in our system, if it is not we’ll return false and will return unauthorized response.
  • We’ve checked if the request received is a replay request, this means that checking if the nonce received by the client is used before, currently I’m storing all the nonce received by the client in Cache Memory for 5 minutes only, so for example if the client generated a nonce “abc1234″ and send it with a request, the server will check if this nonce is used before, if not it will store the nonce for 5 minutes, so any request coming with same nonce during the 5 minutes window will consider a replay attack, if the same nonce “abc1234″ is used after 5 minutes then this is fine and the request is not considered a replay attack.
  • But there might be an evil person that might try to re-post the same request using the same nonce after the 5 minutes window, so the request time stamp becomes handy here, the implementation is comparing the current server UNIX time with the request UNIX time from the client, if the request age is older than 5 minutes too then it is rejected and the the evil person has no possibility to fake the request time stamp and send fresher one because we’ve already included the request time stamp in the signature raw data, so any change on it will result into new signature and it will not match the client incoming signature.
  • Note: If your API is published on different nodes on web farm, then you can store those nonce using Microsoft Azure Cache or Redis server, do not store them in DB because you need fast rad access.
  • Last step is we’ve implemented is to md5 hash the request body content if it is available (POST, PUT methods), then we’ve built the signature raw data by concatenating the parameters (APPId, requestHttpMethod, requestUri, requestTimeStamp, nonce, requestContentBase64String) without any delimiters. It is a MUST that both parties use the same data format to produce the same signature, the data eventually will get hashed using the same hashing algorithm and API Key used by the client. If the incoming client signature equals the signature generated on the server then we’ll consider this request authentic and will process it.
Step 5: Secure the API End Points:
Final thing to do here is to attribute the protected end points or controllers with this new authentication filter attribute, so open controller “Orders” and add the attribute “HMACAuthentication” as the code below:

 Conclusion:

  • In my opinion HMAC authentication is more complicated than OAuth 2.0 but in some situations you need to use it especially if you can’t use TLS, or when you are building HTTP service that will be consumed by terminals or devices which storing the API Key in it is fine.
  • I’ve read that OAuth 1.0a is very similar to this approach, I’m not an expert of this protocol and I’m not trying to reinvent the wheel, I want to build this without the use of any external library, so for anyone reading this and have experience with OAuth 1.0a please drop me a comment telling the differences/ similarities about this approach and OAuth 1.0a.
That’s all for now folks! Please drop me a comment if you have better way implementing this or you spotted something that could be done in a better way.