Over the last few months, the engineering team at BitGo has been quietly transitioning away from our old API documentation, and migrating to new docs that are based on an OpenAPI 3.0 schema. We’re happy to announce that the new API docs are now live. The new documentation is a significant improvement and provides benefits to both third party developers and BitGo engineering.
Our original API documentation was written in Markdown and processed with Slate. Slate is an open source Markdown renderer with built-in support for API documentation. Slate processes Markdown text and generates a 3-pane document with a table of contents on the left, text in the middle, and code samples on the right.
Here’s a snippet of the Markdown source for the endpoint that lists wallets:
Below are the generated HTML docs:
While Slate generates docs that look nice and are conveniently laid out, it is only a templating engine. Our old API docs were written manually and were completely decoupled from the code base. As time went on, we found more and more places where documentation was missing, or drifted out of sync with server behavior.
OpenAPI (formerly known as Swagger) is an API description format that’s specifically designed for REST APIs. It features built-in support for declaring REST API endpoints, query and URL parameters, supported HTTP methods and status codes, and the structures of request and response bodies. Here’s a snippet from the YAML declaration of the same wallet listing endpoint, using OpenAPI 3:
This endpoint declaration defines the request URL, parameters that are passed via the URL or query string, and the structure of the response that is sent to the client. When we run this OpenAPI spec through ReDoc, we get API documentation that looks just as nice as the Slate equivalent:
Lest you write me off as another server engineer who says: “Look, it does the same thing, but it’s better!”, there is also a significant technical benefit: request validation. When a client request comes in, we use ajv to validate the structure of the request body against the JSON Schema that is declared for the endpoint. Here’s an example of the client experience when it sends an invalid request with httpie:
The great thing about request validation is that it requires zero application logic. If the request is missing a required field, uses the wrong data type, has an unexpected enum value, or anything else that doesn’t conform to the schema, our validation middleware returns an HTTP 400 response with an intuitive error message. This makes the request handler logic much shorter, and allows us to focus on writing application logic instead of boilerplate.
Be aware that API schema validation handles many, but not all cases. Validation that requires a database lookup, such as returning 404 when an object doesn’t exist, still needs to be done in code. But overall, API schema yields a lot of value. It significantly reduces the number of lines of server code, while providing accurate documentation and a contract of server behavior to client developers.
One thing to watch out for with OpenAPI is that endpoint declarations can get lengthy. A comprehensive endpoint declaration specifies the JSON schema for every URL parameter, query parameter, request body, successful response, and error response. This can result in endpoint specs that are hundreds of lines long.
To keep things manageable and consistent, we make liberal use of the $ref feature. $ref allows us to reference a JSON Schema that is stored either in the same file or an external file, and use it in multiple places. For example, we use the coin path parameter in many of our URLs. Instead of declaring it over and over, we declare it once:
We reference these seven lines with a single $ref at the endpoint level:
Schemas for endpoint responses can get even longer, so we store them in external files:
This keeps the endpoint declaration concise and allows us to reuse the same object structure in multiple endpoints for consistency.
We wrote a significant amount of infrastructure to leverage OpenAPI. When we began working with OpenAPI in 2018, this type of tooling either didn’t exist for Node, wasn’t ready for prime time, or didn’t support version 3 of OpenAPI. To get around these limitations, we
It’s been great to see over the last year that there’s been more interest and development in this area. We’re starting to see more open source projects that leverage API schema, like Connexion, a web service framework with built-in support for OpenAPI 3.
At BitGo engineering, we continue to be enthusiastic about OpenAPI and look forward to future improvements that take the grunt work out of web services, and help developers be productive while creating a great experience for the consumers of their APIs.