Creating REST Server with Express and Mongo
What is REST
REST stands for Representational state transfer. It is a communication protocol used to request data and recieve data. It is based on the HTTP protocol to request and recieve information. Specifically it uses the web request and response for data transmit. Meaning the server gets the request and he understands from the request what data he needs to send back or what action he needs to perform on the data. After finishing that action he sends the needed data or confirmation back to the client via the response. REST is stateless meaning every request is dealt in the server regardless of previous requests sent. This means that the REST protocol is easily tested, and in fact when controlling the testing frameworks properly it becomes easier to write test for every api then to perform the actual request.
REST example
Lets say we want to create a todo application which lists the tasks we need to do. Our app will probably have a database to save our data, and in our database we will have a table called Tasks that holds the todo tasks. Our rest server will probably have the following api for: Create, Read, Update, Delete (CRUD).
- Get all tasks - We will need to send a get request to the url: /task and it will get all the tasks we have in the tasks table
- Get a single task - We will need to send a get request to the url: /task/<pk> and it will get the task with the primary key in the URL
- Create - To add a task to the task table we will need to send a POST request to the url: /task In the request body we will send the details for creating a new task. A new task has a title, description and date.
- Update - to update a specific task we will need to send a request of type PUT to the url: /task/<pk> with a body in the request with the updated values of the field.
- Delete - to delete a task we will have to send a request of type DELETE to the url: /task/<pk>
Our goal
In this tutorial our mission is to create a REST server for a todo application with an API as described above. Our server will save the data in a Mongo database, and we will use Express as the framework for creating our server.
Bootstrapping Express
Create a new folder for your application named: todo-rest cd in that directory, init npm, and then install express:
> cd todo-rest
> npm init --yes
> npm install express --save
In the root folder of the project create a file called app.js with the following code:
const express = require('express');
const app = express();
app.use(function(req, res) {
res.send('hello world');
});
app.listen(3001, function() {
console.log('now listening on port 3001');
});
The following express app is just printing an hello world message on the screen. So to run the current app you can simply:
> node app.js
open your browser at the address: http://localhost:3001 and you should the hello world message on the screen
Express routers
Express routers are mountable route handlers. For our task we can define a route for the url: /task and let a route handler to deal with the routing of our api. It makes our app more modular, plus it will be easier if we will want someday to create a new version for our api, for example lets say we want to create new api at url /v2/task modular approach for our api will make this kind of transition and backward compatability much easier to maintain. Create a new directory in the root project called: routes in that directory create a new file called task.js with the following code:
const express = require('express');
const taskRouter = express.Router();
taskRouter.get('', function(req, res) {
res.send('get all tasks')
});
taskRouter.get('/:pk', function(req, res) {
res.send(`get single task with pk: ` + req.params.pk);
});
taskRouter.post('', function(req, res) {
res.send('create a task');
});
taskRouter.put('/:pk', function(req, res) {
res.send('update task')
});
taskRouter.delete('/:pk', function(req, res) {
res.send('delete a single task');
});
module.exports = taskRouter;
For now our router just defines the routes and we are not doing much in every route defined. For now this router is just a stub and we will fill it with more logic later on. Few things to note here:
- We didn't write the common prefix of the path which is /task We will write it when we connect the router to the express app.
- When defining our routes, we can specify url params like we are doing with the primary key. We can later access those params with req.params
Lets connect the router to the main application. Modify the app.js to this:
const express = require('express');
const taskRouter = require('./routes/task')
const app = express();
app.use('/task', taskRouter);
app.listen(3001, function() {
console.log('now listening on port 3001');
});
you can run the app.js file and browse to the url: /task or /task/33 to see the result so far
Mongo
Before we start connecting our app to our mongo database, lets talk a bit about MongoDB. Mongo is an open source document database. Think of a document as representing a row in a relational database table. document structure is similar to JSON with key and value. The document is stored as BSON which is binary representation of json documents which means each document can store more than just what JSON brings us it can store more types, array and even other documents or array of documents. The documents are stored in collections which is similar to tables in relational databases. we can query collections for data, define schemas for our documents in the collection, and define validation for the documents in the collection. Lets create a collection called task and add a few documents in our collection. We will use the mongodb console for that, but first we need to install mongodb. Mongodb can be easily installed with Homebrew
> brew install mongodb
To activate the mongodb database use the following command
> /usr/local/Cellar/mongodb/3.2.10/bin/mongod
Make sure to match to your mongodb version. Open another terminal and activate the mongo shell to connect to the activated database.
> /usr/local/Cellar/mongodb/3.2.10/bin/mongo
In the mongo console, create our todo database with the following command:
> use todo
Mongo will create the database when we insert a new document. When inserting a new document, the field _id represents the primary key of the document and has to be unique . If not specified, Mongo will create that field value for us. When creating the _id field value, Mongo will automatically create a 12-byte ObjectId which is a 24 character string of numbers and letters. If you would like to create the _id as an autoincrementing integer and not ObjectId you would have to create a collection that holds the name of the collection and the latest number used for pk.
> db.counters.insert({_id:"task",sequence_value:0})
a collection is created when inserting the first document so we created a counters collection with a document holding the name of the collection in the _id field, as well as the latest pk used.
We can also create a function in mongo that will return the next pk and will also update the value of the document.
function getNextSequenceValue(sequenceName){
var sequenceDocument = db.counters.findAndModify({
query:{_id: sequenceName },
update: {$inc:{sequence_value:1}},
new:true
});
return sequenceDocument.sequence_value;
}
add this function to your mongo shell, adding functions in mongo is extremly easy and we don't have to deal with SQL language functions which are extremly hard to write and we can just write our functions using JavaScript. Our function gets the collection name as argument, and looks in the counters collection for a document with _id as the name of the table, we grab the document with the modification with the new option. we then return the sequance value.
Lets add a few documents to the task table with the insertMany function
db.task.insertMany([
{_id: getNextSequanceValue('task'), title: 'todo1', when: new Date()},
{_id: getNextSequanceValue('task'), title: 'todo2', when: new Date()},
{_id: getNextSequanceValue('task'), title: 'todo3', when: new Date()},
]);
to view all the documents we just inserted we can use the find command
> db.task.find()
This is just a small taste of Mongo but the dynamic structure of a document, and how easy it is to manipulate the database using javascript is a big advantage of Mongo. Time for us to connect the database we created to our express app.
Mongoose
Mongoose helps us create an abstraction layer over our mongo database. With Mongoose we can connect to our database, we can define schemas that will match to our database collection fields. With the Schema defined we can define models according to the schema. Models correspond to the documents in our collection, using the models we can create new document in collection, query the collection, update record(s), delete record(s). The schemas we create can also include validation to verify the documents we create. Lets install mongoose with the following command on your terminal:
> npm install mongoose --save
Mongoose connection
First we will have to connect mongoose to our database. In your app.js modify to look like this:
const express = require('express');
const taskRouter = require('./routes/task')
const mongoose = require('mongoose');
mongoose.connect('mongodb://localhost:27017/todo').then(
() => {
console.log('we are now connected to the database');
},
err => {
console.log('error connecting to db');
}
);
const app = express();
app.use('/task', taskRouter);
app.listen(3001, function() {
console.log('now listening on port 3001');
});
We pass mongoose.connect the url to our database, time to define our schemas and models.
Mongoose Schemas
Schema describes the structure of our collection. Our collection contains the following fields:
- _id - which is an incrementing number
- title - string
- when - date
lets define the schema, from the schema create a model which represents a document. In our schema we will also auto calculate the _id field like we did earlier with the mongo function. In the root directory of the project, create a directory called: models. In the models directory create a file called: task.js with the following code:
const mongoose = require('mongoose');
const Schema = mongoose.Schema;
// define the schema
const taskSchema = new Schema({
_id: {type: Number, unique: true},
title: {type: String, required: true, maxlength: 200},
when: {type: Date, default: new Date()}
}, {
collection: 'task'
});
const Task = mongoose.model('Task', taskSchema, 'task');
// make the schema calculate the id
taskSchema.pre('save', function(next) {
Task.findOne().sort({_id: -1}).limit(1).then((item) => {
this.set('_id', item._id + 1);
next();
});
});
// export the model
module.exports = Task
In the first section we define the schema with mongoose.Schema When creating the schema the first argument are the fields in the collection. for each field we can define the type, validation, default value. We are then using a middleware to connect to pre save event so before saving we are dynamically calculating the _id field based on the largest _id currently in the collection. From the schema we are creating a model and exporting it out.
Implementing our REST
We will now connect the router we created earlier, with the database and the mongoose model we created. The following modifications will be to the file routes/task.js In this file change the following section:
taskRouter.get('', function(req, res) {
res.send('get all tasks')
});
to this
const Task = require('../models/task');
taskRouter.get('', function(req, res) {
Task.find().then((tasks) => {
res.json(tasks);
}, (err) => {
res.send(err.message);
})
});
so for the route /task we are grabbing the model we created, and we use the find method to grab all the tasks from the task collection. You can try and launch the app now and go to that url and verify that we are gettings all the tasks from the collection. The find can get as first argument a dictionary with query with similar syntax as query for mongo. you can pass a callback to the second argument or just use the promise version and attach a then function. Lets add logic for the grabbing of a single element from the collection. In the same file routes/task.js change this section:
taskRouter.get('/:pk', function(req, res) {
res.send(`get single task with pk: ` + req.params.pk);
});
to this:
taskRouter.get('/:pk', function(req, res) {
Task.findById(req.params.pk).then((task) => {
res.json(task);
}, (err) => {
res.send(err.message);
})
});
The mongoose model has a method findById to get a single item by the id, and we pass it what we get in the param. For the post and put to create and modify a task, we will need to pass data via the request body. In order to grab the body from the request, we will need to connect an express middleware. In the file app.js add the following middleware after creating the express app.
app.use(express.json());
app.use(express.urlencoded());
after doing that we can now access the body of the request by using req.body.<param_name> Now lets create the post create new task. Back to the routes/task.js change this part:
taskRouter.post('', function(req, res) {
res.send('create a task');
});
to this:
taskRouter.post('', function(req, res) {
const newTask = new Task(req.body);
newTask.save(function(err, item) {
res.json(item);
})
});
What we are doing here is creating a new instance of our model from the body we get. We then call the save and return a json response of the item we saved. Time to deal with updating an element. In the routes/task.js change the following section:
taskRouter.put('/:pk', function(req, res) {
res.send('update task')
});
To this:
taskRouter.put('/:pk', function(req, res) {
Task.findOneAndUpdate({ _id: req.params.pk}, req.body).then((task) => {
res.json(task);
})
});
We are using the model findOneAndUpdate which gets as first argument the item to find, and the second argument is the fields we are updating. To finalize we will do the delete so in the same file change this:
taskRouter.delete('/:pk', function(req, res) {
res.send('delete a single task');
});
to this:
taskRouter.delete('/:pk', function(req, res) {
Task.deleteOne({_id: req.param.pk}).then(() => {
res.json({success: true});
})
});
We are using the deleteOne method on the model which gets the query to delete an element by the primary key. You can now try and run our app and try every route to see if it works. You can use Postman to create the more complicated requests.
Summary
In this tutorial we learned how we can create a rest server using express mongo and mongoose. We learned what is a rest server. We learned basic mongo database and how we can connect the mongo db to our express app using mongoose. Creating a rest server with express and mongo is quite simple and having a dynamic db like mongo really makes the job easier.
Nerdeez is a software compnay specializing in delivering quality software, educating software teams by giving advanced courses about software technologies, and giving consulting services to software development.