NodeJS - Common problems with async calls
Lesson Plan
In this lesson we will learn best practices when dealing with async non blocking functions. We will cover common bad practices and proper way to solve them and we will learn good convention when dealing with async code. As developers async code present us with some chalanges, its bug prone code and when there is bug in an async code its harder to find and solve, so knowing all the tools we have for dealing with async code is essential.
Bad Practice
First lets go over some bad practices that might happen when dealing with async code. In the previous lesson we talked about node built in modules and specifically we talked also about the file system module (fs). To read a file we placed the following code:
const fs = require('fs');
fs.readFile('stam.txt', function(err, result) {
console.log(result.toString());
});
This function will read the file stam.txt and will call the callback when its done. The callback first argument is an error if something went wrong, or null if everything is ok. The second argument is the result of reading the file. If we are getting an error, one option is to deal with the error inside the function, yet sometime we want to deal with errors outside the callback function. In this case we might be tempted to do something like this:
const fs = require('fs');
try {
fs.readFile('stam.txt', function(err, result) {
if (err) {
throw err;
}
console.log(result.toString());
});
} catch(err) {
// THIS WON'T CATCH THE ERROR IN THE CALLBACK
}
This code place the readFile in a try and catch block the only problem with this code is that when the callback will be called async, our try catch block will already exit and be irrelevant. This is why throwing the error in the callback won't reach the wrapping catch.
Lets go over another bad practice that can happen. Lets say we have a file we want to read. That text file contains url of a server we want to query with a get request. After getting the response from the server we want to print this on the screen. These actions we want to perform on an interval every 10 seconds. Our code might look something like this:
const fs = require('fs');
const https = require('https');
setInterval(function() {
fs.readFile('stam.txt', function(err, result) {
https.get(result.toString(), function(res) {
res.on('data', function(text) {
console.log(text.toString());
});
});
});
}, 10000)
This code will actually work and query the server every 10 seconds and print the result. The only problem with this code is that its hard to read and understand, it will also be hard to debug and in case we want to deal with errors and know exactly the point of error in our async code it will be complicated to achieve. Nesting function inside a function like this code is considered bad practice. In JavaScript programming its called "Callback Hell" and as a rule of thumb you should never write this kind of code.
To avoid the problems we mentioned we first have to familiarize ourselves with the tools we have in JavaScript to deal with these issues. Lets go over the first one: Promise
Promise
A promise is a class that wrapps our async code. In the promise constructor we are passing a function with an async code inside. We call resolve or reject after the async code return with success or error. A promise can be in one of three states: - Pending - the async code is not finished - Resolved - the async code resolved successfully - Reject - the async code resolved with an error. Lets try and create a promise with a timer that will be resolved after 2 seconds.
const timerPromise = new Promise((resolve, reject) => {
setTimeout(() => {
resolve('hello world');
}, 2000)
});
timerPromise.then((msg) => {
console.log(`this runs when the promise resolves argument passed from the resolve: ${msg}`);
}, () => {
console.log('this will run if the promise rejects');
});
when the async callback returns we can call resolve and pass success arguments, or reject and pass error arguments (usually instance of Error). To attach a listener that will be called when the promise returns we use the then method. Promise can only be resolved or rejected once. Promise constructor function will run synchronously, regardless if there is a listener attached. Lets go over some more useful promise properties.
Promise chaining
when attaching a listener with the then method, the then method will return a promise. The promise returned contains data of what the success or error callbacks returned. The success or error callbacks can also return a promise, in that case the chained then will be resolved with the data of the promise resolved. Lets modify the code above and create another promise which reads a file and resolves when the file is read. After which lets try and chain those promises together. Change the code above to this:
const fs = require('fs');
/**
* returns a promise that will be resolved with hello world after 2 seconds
* @returns {Promise<string>}
*/
function getTimerPromise() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve('hello world');
}, 2000)
});
}
/**
* returns a promise with a string read from a file
* @returns {Promise<string>}
*/
function getFileReadPromise() {
return new Promise((resolve, reject) => {
fs.readFile('stam.txt', function(err, result) {
if (err) reject(err);
resolve(result.toString());
})
});
}
// promise chaining
getTimerPromise()
.then(() => {
return getFileReadPromise();
})
.then((fileContent) => {
console.log(fileContent);
});
We wrapped the timer promise we made before in a function that returns that promise (getTimerPromise). We also made another function which returns a promise which will be resolved after reading a file (getFileReadPromise). We are getting the timer promise, attaching a then, whatever returned from the then is transformed to another promise so if we are returning the readFile then we are transforming the promise to a promise containing the content of the file. So what happens is that after 2 seconds the read file is running and our final then gets the content of the file. Notice that our code is less nested then doing everything in the then of the timer promise.
Lets try and modify our code a bit, this time the timer promise will reject after 2 seconds. In our last then we will attach a fail function.
const fs = require('fs');
/**
* returns a promise that will be resolved with hello world after 2 seconds
* @returns {Promise<string>}
*/
function getTimerPromise() {
return new Promise((resolve, reject) => {
setTimeout(() => {
reject('hello world');
}, 2000)
});
}
/**
* returns a promise with a string read from a file
* @returns {Promise<string>}
*/
function getFileReadPromise() {
return new Promise((resolve, reject) => {
fs.readFile('stam.txt', function(err, result) {
if (err) reject(err);
resolve(result.toString());
})
});
}
// promise chaining
getTimerPromise()
.then(() => {
return getFileReadPromise();
})
.then((fileContent) => {
console.log(fileContent);
}, (msg) => {
console.log(msg);
});
Notice that in the promise chaining, if one error happens in one of the promises in the chain it will jump to the nearest fail function, in our case it jumped to the error function of the last promise and did not call the read file. Note that the fail function in the then will also return a promise with what returned from the fail function. This kind of error we will call critical error, an error that blocks the execution of the rest of the promises. But what if we want a non critical error, meaning if someone if our timer rejects we still want to read the file. We can do something like this:
// promise chaining
getTimerPromise()
.then(() => {
return getFileReadPromise();
}, (msg) => {
console.log(msg);
return getFileReadPromise();
})
.then((fileContent) => {
console.log(fileContent);
});
In this case we attached a fail function to the timer promise then, and the fail function and success both returns the same promise, which means we can attach another then even if the first failed and the file will still be read.
catch
instead of attaching then with a fail function we can also use the short catch function on a promise. The previous example of non critical error in promise has a similar result to this code.
// promise chaining
getTimerPromise()
.then(() => {
return getFileReadPromise();
})
.catch((msg) => {
console.log(msg);
return getFileReadPromise();
})
.then((fileContent) => {
console.log(fileContent);
});
The catch section will only enter if our first timer rejects and then will return a promise as well with what is returned from the function.
Waterfall or Parallel
In the example above we ran the async code non parallel. This means only when the timer promise is finished, we launched the read file promise. When running different block of async code, we need to decide if we want to run each block after the previous has finished or maybe run them all in parallel. Lets convert the previous example to run the two promises parallel.
Promise.all([getTimerPromise(), getFileReadPromise()])
.then(([timerMessage, fileContent]) => {
console.log('resolved both');
console.log(timerMessage);
console.log(fileContent);
})
using Promise.all will run the array of promises in parallel and return a promise. Will return to the then only when all promises are resolved. The success function will contain an array of all the values from the promises, in the same order they are entered. A reject function will be called if one of the promises fail with what is sent in that promise reject. If we do want to run non parallel and a promise should wait for the previous one, we will refer to this technique as waterfall where the content of one promise flows to another.
async await
with the usage of promises, we now have another weapon at our disposal that we can use, that is the async await function. we can declare a function as async. An async function returns a promise. Inside an async function we can place an await before a promise. the function will exit and return when the promise is resolved. You can use try and catch blocks in the async functions and the reject will go to the catch block. you can assign the await promise to a variable and that variable will get what the promise resolved. First lets try to achieve the waterfall example where we wait for the timer promise and after it's finished we exectue the read file promise. We will try to do this with an async method. Change the code to this:
async function timerThanReadFile() {
try {
const timerMessage = await getTimerPromise();
const fileContent = await getFileReadPromise();
console.log(fileContent);
}
catch(err) {
console.log(err);
}
};
timerThanReadFile();
Our code seems much more understandable now that we write it like an async code. The file content is read only after the timer promise is resolved. If the timer promise will reject the read file won't enter and will jump to the catch block (critical error) In case we want a non critical error we would have to create to try catch block one for the timer and one for the file content. Or just place the file content promise outside the try catch block. The async function returns a promise which will be resolved when the function finished running (after the last await returns). The promise will contain data of what the function returns or you can throw an error to return a rejected promise. If there is an await with no try catch and it rejects it will reject the entire promise returned from the function.
What if we want to use the async await function and run the promises in parallel. In that case our code will look like this:
async function timerThanReadFile() {
const [timerText, fileContent] = await Promise.all([getTimerPromise(), getFileReadPromise()]);
console.log(fileContent);
};
timerThanReadFile();
So we can just use the Promise.all to run things in parallel and wait for the parallel function to return. Now that
Solving the bad practices
So the first bad practice is the try and catch on async code. This example can be solved by creating a promise for the read file and wrapping the code in async function. In that case as we saw we can still use the try and catch. The second problem of the callback hell is a bit more intresting so lets show it can be solved by code. As a reminder we tried reading a file, from the file to read a url and query that server, and this action should repeat every 10 seconds. Our code looked like this:
const fs = require('fs');
const https = require('https');
setInterval(function() {
fs.readFile('stam.txt', function(err, result) {
https.get(result.toString(), function(res) {
res.on('data', function(text) {
console.log(text.toString());
});
});
});
}, 10000)
Our goal is to turn this code to be a bit less nested. Lets create a promise for every async call - a promise that reads a file - a promise that query a server
/**
* returns a promise with a string read from a file
* @returns {Promise<string>}
*/
function getFileReadPromise() {
return new Promise((resolve, reject) => {
console.log('readfile');
fs.readFile('stam.txt', function(err, result) {
if (err) reject(err);
resolve(result.toString());
})
});
}
function queryServerPromise(url) {
return new Promise((resolve, reject) => {
https.get(url, function(res) {
let data = '';
res.on('data', function(d) {
data += d.toString();
});
res.on('end', function() {
resolve(data);
});
res.on('error', function(err) {
reject(err);
})
});
})
}
setInterval(async function() {
const fileContent = await getFileReadPromise();
const response = await queryServerPromise(fileContent);
console.log(response);
}, 5000)
Now our interval can run periodically with an async function. Which means all the async callbacks in the interval can be prefixed with an await. So we managed to remove the callback hell and deal properly with errors in our code.
Summary
Now that we are dealling with a lot of async code, we have to make sure we are doing things properly. Always remember, if you have a nested code, it's bad practice and if you turn things to promises it can be easily avoided.
Nerdeez is a software compnay specializing in delivering quality software, educating software teams by giving advanced courses about software technologies, and giving consulting services to software development.