Building Your API using Spec Driven Development

Looking Back

"It's all done!"  At a previous job the last line of code had finally been written and our API was ready for release into the wild!  We couldn't have been more proud; it was the final product of six grueling months of intercompany meetings, careful coding, and a lot of patience from QA.  What I didn't realize during this beaming moment was that in just a few months the entire project would collapse right before our eyes, forcing it to be shut down.

No, there wasn't a change in executive thought, or a "strategic shift" in our focus, but rather no one wanted to use our API.  Despite many long meetings, careful coding, and brutal testing – we forgot one little detail.  We forgot to design our API before trying ot build it, and to include our actual users in the design cycle.

Sadly, in the rush to build external facing APIs and get them into the customer's hands, companies have forgotten the golden rule of "selling" any product or service: understanding your customers needs.
 

Understand WHY You Are Building Your API

When I talk to other companies the most common response I get when I ask why they are building their API is that "they want to expose their application/ data to developers."  But when I ask "why," the answer seems to fall short, not because they aren't providing a great service, but because they have not taken the time to understand or think of their developers' use cases.

Before building an API it is important to understand WHY you are building the API.  It is important to understand WHO you are building for – is it for internal use, your customers, or developers who will be extending your services/ complimenting their own?  Is it for data migration or service utilization?  What types of actions are your API users going to be expecting?

This means going a step further than saying, "they're going to use our data" to actually drawing out a resource map that shows the exact services developers will be utilizing, and how they will utilize them.  It's easy to confuse this step with the implementation of CRUD – trying to connect these actions to GET, POST, PUT, PATCH, and DELETE.  But this process should be much more simple, as at this stage you are simply creating user stories, not technical definitions.  As such, instead of focusing on CRUD, focus on what the client will be able to do, such as your API being able to create a new user, update a user, reset a forgotten password, etc.  Or if you have a message system, being able to send a message, create a draft, send a draft, view messages, move messages to folders, view contacts, etc.

During this stage be sure to INVOLVE your users.  Ask them what they'd like to do, what's important to them.  Because it doesn't matter how carefully thought out your API design is, or how carefully you code it- if it isn't what they want, it's not going to do you any good.

Once you have your API user requirements in place, now it's time to design the API.  Surprisingly many companies skip this step, instead electing to go straight to development.  The problem is that your API is a contract, an agreement between you and your users.  It is your company's word to them, and they are depending on your API not just to make their lives easier, but also for their livelihood.  When you break backwards compatibility, when you change things, you are taking away their ability to focus on developing new features and forcing them to focus on fixing problems created by YOUR API.

For that reason it's important to ensure that your API design provides a solid foundation not just for the initial release of your API, but also for future development.  It's important that your API design is focused on what will happen in the long-term.  Don't just focus on the current product road-map, but think about where your product might be in 2-3 years- and what your API will need to be able to do then.
 

Take Advantage of Spec Driven Development

One of the easiest ways to do this is to take advantage of spec driven development, or by building your API in two different stages: design and development.

The first stage, the design stage means working with your development team and potential API users to craft a spec, or a blueprint of how your API should work.  There are several specs out there to help you do this including RAML, Swagger, and API Blueprint.  Personally, my favorite is RAML, as it provides a single format to manage your API with (YAML) and allows for code reuse while encouraging pattern-driven design.  RAML also has some great opensource projects including an API Designer that visually displays your API as you define it, interactive documentation, and an API Notebook that let's users interact and explore your API using JavaScript.

With MuleSoft's freely hosted API Designer you can also prototype or mock your API, letting you solicit immediate user feedback without having to write or change any code.  Prototyping/ mocking your API is crucial in the design process as it helps you identify potential design flaws and inconsistencies while also letting your future API developers provide you with invaluable feedback and use cases you may not have thought of.

The second stage of Spec Driven Development is development, or writing the code for the API's different layers.  By having a strong blueprint for your developers to use they can quickly and fearlessly build out your API, not having to worry about whether or not their resources will work with another developer's, or if the schemas will match.

The idea is that in the design phase you will have eliminated 99% of the design flaws/ potential design bugs.  For that reason it's important to make sure that you stick to the spec during the development cycle.  It's easy to want to deviate or change things, but doing so puts you back at square one - creating an API on the fly without putting careful thought into it before hand.  Remember, we are great at short-term design, but horrendous at long-term design, and APIs are intended to last long-term.

In the event that you do find an issue with the design, that's ok.  It just means you need to go back to the design phase, implement a solution, and then test it out to make sure that solution works as your users would expect and doesn't break anything else/ cause any inconsistencies.  Once you have done so, you can immediately pick back up in the development stage.
 

A Hard Lesson Learned

They say some of the best life lessons are the ones you learn the hard way.  While I couldn't agree more, I still wish we would have had known about Spec Driven Development and had technologies like RAML, Swagger, and API Blueprint available back when we were working on that project.  It's hard to imagine what might have happened if we would have been able to identify the problems with our API before writing a single line of code.  Where the API might be today, and just how much time and money we would have saved.

Spec Driven Development doesn't guarantee that your API will be perfect, or that it will even adhere to the best API Design Practices.  But what it does do is ensure that it makes sense to your users, and provides a solid, thought-out foundation that you can build your API on- increasing the chances that it will be flexible, extensible, and long-lived.