Airbnb's Move from Monolith

Like most startups, Airbnb started with a monolithic application built using Ruby on Rails. Internally, it was called the monorail because everything went through this single application.

This monolith was responsible for both client and server-side functionality. The model, view, and controller layers were packed into a single repository.

A simple diagram for this architecture would appear as follows:

You can check out the diagram on Eraser.io

There were several advantages to this monolithic approach:

• Easy to get started initially.

• Good for agile development

• Manageable complexity levels

Those were simpler times and such times usually don’t last. Airbnb grew fast, and its engineering team doubled and tripled. Suddenly, you had more and more developers adding new code and features to the monorail.

Just like too many cooks spoil the dish, the code base started to get tightly coupled, and data ownership became muddled. Any developer could make changes to any part of the application, making it difficult to track and coordinate the changes.

Airbnb decided to migrate from the monorail to a Service-Oriented Architecture. You could also see it as a move towards loosely coupled microservices.

The Rules of the Game

Of course, Airbnb was wise and came up with some key design principles for building services. Some of the rules are mentioned below:

• A service should own both the reads and writes to its data. This is similar to the database-per-service pattern where a particular piece of data is owned by one and only one service.

• A service should address a specific concern

• Services should avoid duplicating functionality

• Data changes should happen via standard events to encourage loose coupling.

• Each service must have proper alerting and observability settings

The SOA Design

After a few years of migration stuff, Airbnb ended up with the below architecture:

You can play around with the diagram on Eraser.io

Let’s unpack it a bit.

• The bottom layer is composed of data services. These services are the entry point for all interactions with the database and aren’t dependent on any other service.

• The derived services read data from other services and apply basic business logic. They can also have specialized data storage to store any derived data.

• The middle-tier services contain the key business logic that doesn’t fit at the data service level or the derived data service level.

• At the very top are the presentation services. These services aggregate data from all other services and apply frontend-specific business logic before returning the data to the client.

The Migration

The move from the monorail to shiny new services wasn’t an overnight process. Also, migrating reads and writes had their problems.

Let’s look at both scenarios and how Airbnb handled the migration.

Migrating the Reads

Airbnb used dual reads and response comparison for the old read path (through the monorail) and the new read path (the service).

See the diagram below:

You can play around with the diagram on Eraser.io

Every read request went through both paths. The responses were sent to a comparison framework. Once the engineers were sure that things were as expected, the traffic to the service path was ramped up.

Migrating the Writes

Writes were trickier because one cannot dual-write to the same database. Therefore, the engineers used a shadow database.

See the diagram below:

You can play around with the diagram on Eraser.io

The new service initially writes to the shadow database. After that, read requests are sent to the prod and shadow database, and the responses are sent to the comparison framework.

Only after the comparison came out clean, the writes were completely switched to the new service.

Lessons

Airbnb had several important lessons from this whole migration process. A few important ones for us are as follows

• Invest in common infrastructure pieces early on. Things like shadow databases, comparison frameworks, etc. can help during the migration.

• Simplify the service dependencies

• Moving to services is not just technical but also a cultural change for the organization.

• The migration isn’t a fixed destination but a journey.

So - what do you think about Airbnb’s migration journey? Would you have done something differently?