A Story of How Multimodel Databases Can Reduce Data Sprawl (Told in the Pixar Style)

Pixar uses a consistent structure to develop all of its stories. It basically follows the pattern of:
Once upon a time, there was…

• Every day,… 
• One day,… 
• Because of that,… 
• Because of that,… 
• Until finally,…”  

I thought I would borrow that style to tell you a story of multimodel databases. 

Once upon a time applications were built almost exclusively in a monolithic style, where the application was built as a single and indivisible unit. This approach was created a long time ago when data capacity was limited, databases were designed for a single box, and no one had mobile devices. 

Every day development teams worked using a linear and sequential waterfall approach. Given the demands of the applications, databases, and infrastructure of the day, that method worked well to develop some very successful applications. That said, it also had its own drawbacks like applications growing too large and complicated over time for a team to understand what is going on and how to manage them. Making changes was harder in such a large and complex application with highly tight coupling. Scaling components independently was not an option and code changes could affect the whole system, so each change needed to be thoroughly coordinated. Often, this made the development process much longer overall. And finally, it could be very difficult to apply a new technology because then the entire application had to be rewritten.

Then one day the Internet was born and this changed everything. (OK, it wasn’t one day but stick with me.) More and more people were getting online using web browsers and eventually mobile phones and they were doing it with more frequency and from more places. What came next was an increased demand from these users to have a richer experience with their applications. Companies also wanted to deliver better experiences both digitally and in real life, enhanced by technology. More applications were built to engage with users and customers. At the same time, storage and processing power became much more affordable. This led to a data explosion.

Because of that, new approaches to application development began popping up and growing in popularity. Terms like agile, scrum, kanban, minimal viable product, and lots more entered our lingo as these things became more popular. This led to a big macro trend of what we now call microservices development, which breaks application needs down into a collection of smaller independent units. These units carry out every application process as a separate service. So all the services have their own specific function, logic, and database.

Because of that, development teams are now able to build applications much more quickly. As all the services can be deployed and updated independently, that provides developers with more flexibility. Secondly, a bug discovered in one microservice has an impact only on a particular service and does not influence the entire application. It is also much easier to add new features to a microservice application than a monolithic one. By separating an application into smaller and simpler components, microservices are easier to understand and manage. Additionally, the microservices approach provides the advantage that each element can be scaled independently. So often the process is more cost-effective and time-effective than scaling the entire monolith application.

Microservices are not perfect though, and bring their own challenges. Teams need to choose and set up the connections between all the modules and databases so all the connections have to be handled carefully, there are cross-cutting concerns like logging, metrics, health checks, and more. And testing/troubleshooting can be difficult across the services. And most importantly this type of architecture can lead to big challenges with data sprawl.

Database sprawl brings issues with duplicate data, data movement, security, data integration, information inconsistency, latency, and increased cost. Teams need to have domain knowledge and multilingual programming skills, and then license, and support more types of databases, which causes technical and operations challenges that result in slowing down development. 

Until finally, some database companies decided to consolidate multiple data access methods and other integrated services into their databases to reduce the negative effects of data sprawl. 

Couchbase flexibly stores data within JSON documents, allows for speedy, in-memory Key/Value data access, utilizes SQL for query, has built-in full text search, server-side eventing, and real-time analytics. And it can relay and synchronize data to, from and among mobile devices and other connected things. This means developers can leverage these services against a single database avoiding data sprawl and the issues that come with it. That means developers and organizations can see faster time to value for new applications and new requirements. There are no cross-service latency issues, as they are using the same dataset. Real-time analytics can be run on operational data and need no ETL processing and can be done in a way that doesn’t affect operational processes. Database administration is simplified, as it is all within one platform. Additionally, with Couchbase, services can be scaled independently – like microservices. Customers may want compute-optimized hardware for one service and memory-optimized hardware for another, which helps them performance match application needs to the database and infrastructure. Our in-memory key-value managed cache delivers millisecond performance, without needing a separate caching product to learn. The end result: A multimodel database that reduces time spent before, during, and after application development, while driving down total cost of ownership.

Happily Ever After

Are multimodel capabilities needed in every application case? No, of course not, but they are applicable in many cases, and having them in place helps future proof an application. So now organizations can get the best of the monolithic and microservice approaches supported by a single database. 

And applications can live happily ever after.

Learn more about Couchbase’s multimodel database

Try Couchbase for yourself today with our free trial.