While lots of nascent companies know the importance of allowing third-party access to their data, they don’t always make it easy for developers to connect: confusingly named endpoints, deprecated chunks of code, and out of date docs are detritus which accumulates quickly and slows down the people working on your platform.
In enterprise software, having a simple, advanced API is not just an exercise in neat coding: It’s a business necessity, and not an easy one to master. Putting into practice the latest standards and philosophy, Box recently revamped their API to make it as easy as possible for third-party app developers to build their own applications on top of Box’s file storage system. That, in turn, helps Box increase its footprint, creating a cottage industry around the platform.
When Peter Rexer came on board, the first step was to take a look at the hard questions: How was Box scaling? How much header was in API calls? How much load do those calls put on the database? The answers prompted Rexer to recommend a re-architecture of a big chunk of Box’s backend.
As a result, the Box team took big steps like building a common abstraction layer, renaming all parameters, and automating documentation so that it was never out of date. By creating a totally new code path to accessing objects, Rexer’s team accomplished something that every platform should aspire to: an elegant way to get programmatic access. Here’s their advice for doing the same for your platform.
The glue that lets different applications talk to each other, APIs–or Application Programming Interfaces–are ubiquitous and essential to the modern cloud-based technology landscape. They’re what make it possible to have platforms where third-party developers get behind-the-scenes access to data stored in Box’s servers, which they can use to build customer-facing applications of their own.
Box thinks of the Platform team as its own operating group. It has its own Engineering, Product Management, Marketing, and Business side that specifically works on developer outreach. Most companies don’t put this kind of energy into building a separate Platform team–but perhaps they should.
What exactly do you do at Box?
I basically work with a group of engineers that are separated from the rest of our product team. And we’re focused on building out tools and APIs and frameworks, such that partners, customers, and anyone who wants to build on top of Box, can come and work with us. So we’re really focused on making the next level of the virtuous cycle where you just get viral growth out of doing this kind of stuff.
What was the catalyst for revising the Box API?
When the version 1 API was rolled out, it was state of the art for seven years ago. But the state of the art has really changed since then. If you look at how people built APIs seven years ago, PHP and Python and Ruby weren’t big languages; Java and C++ and C# were the big languages. People built with verb-noun names. So you can think of the API operations like “get me some pizza” or, in our case, “get a file” or “get a folder.” What you saw with the emergence of social networking and social media is the move towards dynamic languages and more consumerization not just of the websites themselves, but of how we build them. So the consumerization of how you build APIs also started to happen there too, and this whole standard called REST and RESTful APIs was born. And REST is really moving away from this verb-noun nomenclature for things, and moving to creating a set of nouns that then you have operations on. They really leverage HTTP standards as much as we can, to do what would classically be in database parlance CRUD operations on nouns. So basically, these would be mapped as create is POST, read is a GET, update is a PUT, and delete is DELETE.
Were there APIs you looked at for inspiration?
Philosophically, our approach was “How do we make this thing the best API out there for content collaboration? And more than that, one of the best APIs on the Internet?” We’ve looked some of the best APIs out there and challenged ourselves–Can we one-up them? Can we do it one better? Certainly guys like Stripe and Twilio are some of the ones that we really respect. The Twilio guys have built a business out of being an API provider. Essentially their whole product is API. Not only are they providing a really cool, free API, but they’re also providing great documentation, great support. So one thing we took from that is we’ve sponsored a tag on Stack Overflow where people can ask questions about the API. And what was cool to see is the community has started answering questions before we get around to it. So we go in there every morning to answer questions from the previous night. And the community’s been answering their own questions now, which is kind of a critical mass turning point. We realized that there’s a big enough developer community building on Box and that we’ve hit something that really resonates with the community.
It seems like the new API is more extensive. How much of the solution was just adding more stuff?
Our philosophy is that everything but the kitchen sink is there-–if you want to get at it. We’ve actually exposed more in our v2 API than we had in our v1 API. But by default, we don’t throw the kitchen sink at you. By default, we give you a nice, standardized set of fields. We try to give an easy on-ramp with both good defaults and good documentation. In our space, the “Hello World!” example really is a folder browsing mechanism. You’re not required to send in fancy IF-MATCH headers if you don’t want to. You’re not required to ask for a custom payload. We’ll just give you the default one. And then there are nice elegant ways for developers to get a little bit more, and a little bit more, and a little bit more as they need it. Our documentation is another area that’s getting a lot getting a lot of kudos. We’re actually auto-generating parts of our documentation so that they’re as accurate as they can possibly be.
Is it really worth the time to build documentation that writes itself?
Well, if you think about all the practices that have come into play in the last 10 years, having self-documenting code has become really popular. And on top of that the whole don’t-repeat-yourself philosophy has really permeated development spaces. So we figured we can use a combination of documentation auto-generated out of our code to pull together our docs. We wrote a doc generator that basically goes through the nightly builds, pulls together all the code that’s associated with the APIs, pulls out all the endpoint parameters, and generates those parameterized lists that go into the documentation. Then we’re actually using WordPress APIs to push those things into our documentation, which is just a WordPress site. It’s pretty cool stuff. And it’s all built by just stacking things and gluing them together.
What was your benchmark for simplicity?
What that comes down to is–my daughter’s a freshman in high school. One weekend we were driving in the car and she heard an NPR piece about Douglas Rushkoff‘s book Program or Be Programmed. We chatted a little bit about it because she knows that I work on software. And the next week, I’m in her room and I see written on her desk, there on her goals list, “learn how to code.” So we talk more and she asks me what’s the best way to learn how to code. So I pointed her to Codecademy. Within my team that got us thinking along these lines, like “Hey, teenagers wanna learn how to write code.” They know it’s cool. They know they want write a cool iPhone app or whatever it is they want to play with. And wouldn’t it be great if we could make it so that high school kids can learn how to hit our API? ‘Cause that’s really the magic of being able to build a really cool app, when you have all these demanding services out there. You know, if you can hit Google Maps and you can hit Stripe and you can hit Twilio and you can hit Box, you could mash up some really cool applications you dream up as a kid.
What’s the biggest obstacle to a simple API?
Reducing the cognitive complexity of what people need to understand makes our API much easier to get started with. And while this looks rather obvious in hindsight, elegant minimalist design is hard. A specific example of what was difficult to deal with in the v1 API were the inconsistencies in naming of parameters. Each API endpoint defined its own list of inputs and outputs, so there were no consistent naming standards across APIs. We had file_name, new_name, name, and item_name for the names of files, depending on which API method you called. In V2 we created a framework that parses the input and generates JSON responses out of a set of object-models. The object models are all centrally located, and naming standards are enforced with unit tests. It has resulted in making it easier for us to write new API endpoints, and made it easier to ramp up new engineers on how to build or work on our v2 API.
How has the new API been received by the developer community?
Platform engagement has had pretty steady growth. We released the beta version of the v2 API back in April. Since then the total number of developers using our API has skyrocketed from 9,000 to 21,000. Third-party app users have increased 66%. We’re now totaling over 3 billion API calls per month and that rate has been growing 10% month-on-month. The v2 API became generally available in December and as of mid-February, two months after GA, more than half of all our API calls are on the v2 API. We’ve also had quite a few developers giving us some pretty incredible anecdotal feedback. I mean, I’ve had developers come on our forums saying, “Hey, this is the easiest API I’ve ever worked with.” I have guys who have built C# SDKs on top of our API. And when I chatted with them about building an SDK, they said they’d already built one on v1, and it took them about a fifth to a third of the amount of time to build it on top of v2, just because the API was so much easier to work with.
The core of Box’s file storage system is of course folders. The version 2 API completely overhauled endpoints that touched folders, making it much more intuitive to work with. In the v1 API, every endpoint that did some operation on folders took a different set of parameters. To move and rename a folder, you had to use the “destination_id” and “new_name” parameters which were specific to the move_and_rename API. In v2, all inputs and outputs share parameters. There are fields such as “name” and “parent” with the subfield “id” which are used in all endpoints.
A comparison of v1 operations and v2 operations looks like this:
get_account_tree –> GET ./folders/ID
create_folder –> POST ./folders
move –> PUT ./folders/ID
copy –> POST ./folders/ID/copy
rename –> PUT ./folders/ID
copy_and_rename –> POST ./folders/ID/copy
move_and_rename –> PUT ./folders/ID
delete –> DELETE ./folders/ID
set_description –> PUT ./folders/ID
toggle folder email. –> PUT ./folders/ID
[Image: Flickr user Rabih]