Record collections and RESTful API

RESTful API technologyHere is a typical situation encountered when developing server-side applications that expose a RESTful API: We have a resource that represents a collection of records of the same type and the API allows querying this resource. The result of this query is a list of matching records, normally represented as an array of JSON objects.

For example, we may have an application that deals with customer orders. The API may have an endpoint at URI “/orders”, which, when queried, returns a JSON response like this:

This is good. However, what if we are trying to display an admin screen that lists matching orders? What if, instead of seeing customerId and productId properties as numbers, we want to show the customer and product names?

One solution would be, after loading the orders, to parse the response and send separate requests to the API endpoints that allow loading product and customer records. This is bad for the following reasons:

  • Sending multiple requests is slow.
  • The order records need to be processed one by one to extract the ids of the referred product and customer records.
  • We lose real time “transactionality.” The screen is assumed to be showing a snapshot of the data at a given moment, but if the referred records change between sending our individual API requests, the displayed data may eventually be inconsistent.

So we want to load the orders and the referred products and customers in a single API call and receive all that data in a single response. But how to represent it?

If our server-side application internally works with some sort of ORM technology (e.g. Hibernate), the entity beans that represent the orders have references to the entity beans that represent products and customers. If simply serialized into JSON for our API response, the result will be something like the following:

There are two problems with this approach:

  • We lose clear record boundaries, which may create problems in the front-end logic.
  • If multiple orders refer to the same customers and products, the customer and product data is unnecessarily repeated.

And there is an additional problem too: what if we also want to load some information about the orders collection as a whole? For example, the total number of orders. There is no place for such data in the response format above.

The solution that is aimed at becoming the standard is Hypertext Application Language, or HAL. The RFC draft can be found here. It is an excellent generic solution, which is currenlty used by Spring Framework‘s REST module, giving it a certain level of credibility. However, this solution has certain shortcomings as well. Two can be identified right away:

  • The format includes a lot of boilerplate structure and complexity, which makes the API response larger than it could be.
  • For the same reason, the API response is rather hard to read, which compromises the beauty of JSON.

I suggest a simpler concept, which, by the way, is currently utilized by our X2 Framework. The X2 solution is characterized by the following:

  • Simlarly to HAL, the records collection response is wrapped in an object. As its top-level properties, the response object includes the array of matched records, the collection properties (such as the total), and a bag of referred records.
  • The references to other records (products and customers in our example) are represented by a special string that includes two parts: the resource type name (no need for a URI here, as in HAL, since the application will always know where to find its resources) and the record id.

Using X2, our API response can look like this:

Compact, easy to read, and machine-process – no unnecessary repetition.