Data is the new oil. And the oil well of the tech industry is a database. No matter what application or platform you are trying to build, you will definitely require a database.
There are tons of databases out there. Every week we hear about a new database to hit the open source world. In all this madness, ever wondered which database would dominate the industry? I have. And if you’re as excited as I am, you must be wondering what’s gonna come out on top. But before we get into that, let’s understand the current database landscape.
The Trend
The growing trend, it seems, is specialisation. Today we have databases which solve niche problems like working with time series data or for search. You are supposed to glue these pieces together, pretty much yourself. For tiny startups like ours, you probably do not have the resources to adopt such diverse architectures.
There are another breed of multi-model database coming up like ArangoDB and Cosmos DB to solve this problem. Their idea is simple. Have a single database which support different data modelling techniques (like document, graph, etc.). This gives an architect, unprecedented flexibility to model the application. Since the database is multi-model, it can fit in a variety of use cases.
This might sound like the answer to all our prayers, but there is a dark side to it. Such database architectures are extremely opinionated. That is, it might not be specialised for any particular use case. The focus is more on being general purpose.
Such a trade-off is usually acceptable for most of us. But in my experience, at some point, we all need to specialise. Scale isn’t a generic process. It’s highly customized. The tech that works for one app wouldn’t necessarily work for another app of the same genre. Here’s a great read on how to build scalable apps.
The need for specialised databases arises due to varied types of workloads. Most IoT use cases, for example, are write intensive. The writes must have low latency. The reads are usually an aggregation with respect to time. It makes sense to store data points in a chronological order to boost query performance. If we built a database for precisely this need, the result would be a time series database.
So basically, going forward with a multi-model database would eventually lead to migration in the longer run. Migration to a specialised database helps saves cost and increases performance.
Now that’s a pickle. What’s the right path forward? Who shall finally get the throne in the kingdom of databases?
The Return of the Key Value Store
“Wait. Why are we talking about a key value database?" - A question one may ask.
Most of the database out there are basically modelled on top of a key value database. That’s the degree of flexibility a key value store has to offer. Be it a relational database like MySQL or a graph database like Dgraph.
You can easily make a document database like mongodb on top of a key value database. Each unique field of a document would be mapped to a key. To know how this works, you can check out this data modeling guide.
So where does that bring us?
You can customise a key value database to suit your modeling needs. Since its only set and get operations under the hood, wide range of performance tuning is possible. Also, we all know that key value databases are some of the most scalable databases in the world.
A single database can pretty much solve all your specialised database needs. But, where’s the catch? Why aren’t people doing it already? Let’s try and answer that.
In order for any of this to work we would need a distributed database which can store keys in an ordered fashion. Also we would need transactional capabilities to make sure our objects (in case of a document database) are consistent.
FoundationDB: The Rising Star
FoundationDB is an lexicographically ordered, transactional key value database. It’s fully ACID compliant which means that our database (data and indexes) will always be in a consistent state.
This gives us unbelievable power. You can implement various high level constructs (like indexes) the way which is best suited for your need. You can build special data structures which would speed up graph traversals.
“That sounds great! But wouldn’t I have to build everything myself then?" - Asked the impatient architect.
Sure, not everyone has the time or the will to model a database. And that’s where FoundationDB comes with a revolutionary concept - FoundationDB Layers.
“FoundationDB decouples its data storage technology from its data model. FoundationDB’s core ordered key-value storage technology can be efficiently adapted and remapped to a broad array of rich data models. Using indexing as an example, FoundationDB’s core provides no indexing and never will. Instead, a layer provides indexing by storing two kinds of key-values, one for the data and one for the index." - extract from the FoundationDB Docs.
The “layer” mentioned here is like a script running beside the database which essentially implements a data model on top of it. For example, you might have a layer implementing the MongoDB interface, while another layer could expose a MySQL layer.
Moreover, since FoundationDB is fully open source, the community is bound to come up with all sorts of Layers for various applications.
Summing Up
FoundationDB takes the database race to a whole new level. I can’t say it’s the perfect database since it’s underlying architecture isn’t well known yet. But it definitely is in the right direction.
The same database can now act as multiple different kinds of databases specialised for different use cases. This gives the user’s super high flexibility and performance while maintaining operational simplicity.
And with the way tech is evolving, you don’t even need to learn how each database works in order to use it. Tools like Space Cloud, provide a neat abstraction to fully automate your backend. So essentially you can extract every ounce of performance while keeping the learning curve flat.
FoundationDB Layers offer a good starting point for those starting afresh. It’s easy to customise for the performance hungry people. That one database will suffice throughout the evolution of your product. It’s truly your lifetime companion!
Did this article help you? What database do you use? What do you think is the king of databases? Share your thoughts below. And if you want help on architecting your apps, you can always reach out to Noorain on LinkedIn.
