Migrate from MongoDB Mongoose to Couchbase with Ottoman

When talking to Node.js developers, it is common to hear about NoSQL as the database of choice for development. JavaScript and JSON come hand in hand because after all JSON stands for JavaScript Object Notation. This is a format most common in document oriented databases which Node.js developers tend to use.

A very popular stack of development technologies is the MongoDB, Express Framework, Angular, and Node.js (MEAN) stack, but similarly there is also the Couchbase, Express Framework, Angular, and Node.js (CEAN) stack. Now don’t get me wrong, every technology I listed is great, but when your applications need to scale and maintain their performance, you might have better luck with Couchbase because of how it functions by design.

So what if you’re already using MongoDB in your Node.js application?

Chances are you’re using Mongoose which is an Object Document Model (ODM) for interacting with the database. Couchbase also has an ODM and it is called Ottoman. The great thing about these two ODM technologies is that they share pretty much the same set of APIs, making any transition incredibly easy.

We’re going to see how to take advantage of MongoDB’s REST API to transition from a Node.js application using Mongoose and migrate it to Couchbase, featuring Ottoman.

The Requirements

This tutorial is going to be a little different because of all the technologies involved. We’re going to be building everything from scratch for simplicity, so the following are requirements and recommendations:

• Couchbase Server 4.5+
• MongoDB 3.4+
• Node.js 6.0+

We’re going to start by building a MongoDB REST API in Node.js using Mongoose, hence the Node.js and MongoDB requirement. Then we’re going to take this application and migrate it to Couchbase.

For purposes of this example, we won’t be seeing how to configure Node.js, MongoDB, or Couchbase Server.

Understanding our NoSQL Data Model

Both MongoDB and Couchbase are document databases. One stores BSON data and the other stores JSON, however from the developer perspective they are incredibly similar. That said, let’s design a few models based around students attending courses at a school. The first model we create might be for actual courses, where a single course might look like the following:

In the above, notice that the course has a unique id, and we’ve defined it as being a course. The course has naming information as well as a list of students that are enrolled.

Now let’s say we want to define our model for student documents:

Notice that the above model has a similar format to that of the courses. What we’re saying here is that both documents are related, but still semi-structured. We’re saying that each course keeps track of its students and each student keeps track of their courses. This is useful when we try to query the data.

There are unlimited possibilities when it comes to modeling your NoSQL data. In fact there are probably more than one hundred ways to define a “courses and students” model, versus what I had decided. It is totally up to you, and that is the flexibility that NoSQL brings. More information on data modeling can be found here.

With a data model in mind, we can create a simple set of API endpoints using each MongoDB with Mongoose and Couchbase with Ottoman.

Developing an API with Express Framework and MongoDB

Because we are, in theory, migrating away from MongoDB to Couchbase, it would make sense to figure out what we want in a MongoDB application first.

Create a new directory somewhere on your computer to represent the first part of our project. Within this directory, execute the following:

The above commands will create a file called package.json that will keep track of each of the four project dependencies. The express dependency is for Express framework and the body-parser dependency allows for request bodies to exist in POST, PUT, and DELETE requests, all of which are common for altering data. Then mongodb and mongoose are required for working with the database.

The project we build will have the following structure:

Go ahead and create those directories and files if they don’t already exist. The app.js file will be the application driver where as the routes will contain our API endpoints and the models will contain the database definitions for our application.

Defining the Mongoose Schemas

So let’s work backwards, starting with the Mongoose model that will communicate with MongoDB. Open the project’s models/models.js file and include the following:

In the above we’re creating MongoDB document schemas and then creating models out of them. Notice how similar the schemas are to the JSON models that we had defined previously outside of the application. We’re not declaring an id and type because the ODM handles this for us. In each of the arrays we use a reference to another schema. What we’ll see at save is a document id, but we can leverage the querying technologies to load that id into actual data.

So how do we use those models?

Creating the REST API Routes

Now we want to create routing information, or in other words, API endpoints. For example, let’s create all the CRUD endpoints for course information. In the project’s routes/courses.js file, add the following:

In the above example we have three endpoints. We can view all available courses, view courses by id, and create new courses. Each endpoint is powered by Mongoose.

When creating a document, the request POST data is added to a new model instantiation. Once save is called, it gets saved to MongoDB. Similar things happen when reading data from the database.

In the case of the above, the find function is called and parameters are passed in. When there are no parameters, then all documents are returned from the Course collection, otherwise data is queried by the properties passed. The populate function allows the document references to be loaded so instead of returning id values back, the actual documents are returned.

Now let’s take a look at the other route.

The second route is responsible for creating student data, but there is an exception here. We’re also going to be managing the document relationships here. Open the project’s routes/students.js file and include the following source code:

The first three API endpoints should look familiar. The new endpoint student/course is responsible for adding students to a course and courses to a student.

The first thing that happens is a course is found based on a request id. Next, a student is found based on a different request id. If both documents are found then the ids are added to each of the appropriate arrays and the documents are saved once again.

The final step here is to create our application driver. This will connect to the database and serve the application to be consumed by clients.

Connecting to MongoDB and Serving the Application

Open the project’s app.js file and add the following code:

In the above code we are importing each of the dependencies that we previously installed. Then we are initializing Express and telling it to accept JSON bodies in requests.

The routes that were previously created need to be linked to Express, so we’re importing them and passing the Express instance. Finally, a connection to MongoDB is made with Mongoose and the application starts serving.

Not particularly difficult right?

Developing an API with Express Framework and Couchbase

So we saw how to create an API with Node.js, Mongoose, and MongoDB, so now we need to accomplish the same thing with Node.js, Ottoman, and Couchbase. Again this is to show how easy it is to transition from MongoDB to Couchbase and get all the benefits of an Enterprise ready, powerful NoSQL database.

Create a new directory somewhere on your computer and within it, execute the following to create a new project:

The above commands are similar to what we saw previously, with the exception that now we’re using Couchbase and Ottoman. The project we build will have exactly the same structure, and as a refresher, it looks like the following:

All Ottoman models will exist in the models directory, all API endpoints and Ottoman logic will exist in the routes directory and all driver logic will exist in the app.js file.

Defining the Ottoman Models

We’re going to work in the same direction that we did for the MongoDB application to show the ease of transition. This means starting with the Ottoman models that will represent our data in Couchbase Server.

Open the project’s models/models.js file and include the following:

The above should look familiar, yet you have to realize that these are two very different ODMs. Instead of designing MongoDB schemas through Mongoose we can go straight to designing JSON models for Couchbase with Ottoman. Remember there are no schemas in Couchbase Buckets.

Each Ottoman model has a set of properties and an array referencing other documents. While the syntax is slightly different, it accomplishes the same thing.

This brings us to the API endpoints that use these models.

Creating the REST API Endpoints

The first set of endpoints that we want to create are in relation to managing courses. Open the project’s routes/courses.js file and include the following JavaScript code:

In the above code we have three endpoints structured in a near identical way to what we saw with MongoDB and Mongoose. However, there are some minor differences. For example, instead of using promises we’re using callbacks.

One of the more visible differences is how querying is done. Not only do we have access to a find function like we saw in Mongoose, but we also have access to a getById function. In both scenarios we can pass information on how we expect a query to happen. Instead of using a populate function we can use load and provide which reference documents we wish to load. The concepts between Mongoose and Ottoman are very much the same.

This brings us to our second set of routes. Open the project’s routes/students.js file and include the following JavaScript code:

We already know the first three endpoints are going to be of the same format. We want to pay attention to the last endpoint which manages our relationships.

With this endpoint we are obtaining a course by its id value and a student based on its id value. As long as both return a document, we can add a reference of each to each of their arrays and re-save the document. The same thing and nearly the same code was found in the Mongoose version.

Now we can look at the logic to start serving the application after connecting to the database.

Connecting to Couchbase and Serving the Application

Open the project’s app.js file and include the following JavaScript:

Does the above look familiar? It should! We are just swapping out the Mongoose connection information with the Couchbase connection information. After connecting to the database we can start serving the application.

Conclusion

Migration from Mongoose and MongoDB is easier than you might think. In fact, you just saw how to build a REST API with Node.js, Mongoose, and MongoDB, then bring it to Couchbase in a very seamless fashion. This was meant to prove that the migration process is nothing to be scared about, especially if you’re using Node.js as your backend technology.

With Couchbase you have a high-performance, distributed NoSQL database that works at any scale. The need to use caching in front of your database is eliminated because it is built into Couchbase. For more information on using Ottoman, you can check out a previous blog post I wrote. More information on using Couchbase with Node.js can be found in the Couchbase Developer Portal.