Node - Async
What is Async
Async is a utility module that provides us functions for working with asynchronous JavaScript. To understand async first we need to understand what is node style async function.
Node style async function
Node style async function, is a function that gets any number of arguments and the last argument is a callback function. The callback function has a signature where the first argument specify an error and will be given a null if there is no error, the other arguments can contain the results passed from the async function. We can see this style of callbacks on many async functions in node. Lets take the fs module (file system module). Lets say we have a file in the current directory that we want to read asynchronously, using fs it would probably look something similar to this:
const fs = require('fs');
const path = require('path');
fs.readFile(path.resolve(__dirname, 'stam.txt'), function(err, data) {
console.log(data.toString());
});
Notice that the file is read async and then calls the function in the second argument. The point of this example is that when using async utility library we have to understand certain players that we will use often throughout the usage of async module. The async function player (in the example above fs.readFile) and the callback player (in the example above the function we sent as the second argument of the fs.readFile)
Installing async
So lets try and experiment with the async library. Open a new folder called: async-tutorial cd to that folder, init npm and install async.
> mkdir async-tutorial
> cd async-tutorial
> npm init --yes
> npm install async --save
Popular async functions
Lets go over the most used async functions. the async utilities functions are divided to categories
Collections
When we have a collection of async functions and we want to do a manipulation on that collection, then the utility function will be in this category. The async module functions in this category have a similar signature. The first argument is a collection of items (can be an array or an object). The second argument is an iteration async function that will be called for each item in the collections. The third is a callback with the result depending on the async function we chose. Lets go over a few main ones.
Filter
The use case for this function is this: We have a multiple async functions We want to run those functions in parallel. We want only the result of the async functions that called the callback with truthy value. Lets give a small example of array of files we are trying to read, and we are intrested only on those files we successfully managed to read
const fs = require('fs');
const path = require('path');
const filter = require('async/filter');
filter(
['stam.txt', 'nofile1', 'nofile2'],
function(fileName, callback) {
fs.readFile(path.resolve(__dirname, fileName), function(err, data) {
if (err) return callback(null, false);
return callback(null, true);
});
},
function(err, results) {
console.log(results);
}
)
With filter we are calling the callback with a boolean value, after all the async functions resolve we will get in the final function only the results of the array with callback true.
Map
The map function is an extremly useful and popular function we are going to use a lot from the async module. The use case of this is if we have the following conditions: We have multiple async functions we want to run in parallel. We want to transform the data we are getting to a different data. The last function will be called with the array of the changed results in the order they are send. If one of the async fails then the last function will be called with an error. Lets give the following example with code. Lets say we have multiple files we want to read. We have the filenames but we are intrested more in the data in the files. We can use map with fs.fileRead to get the text result of all the files.
const fs = require('fs');
const path = require('path');
const map = require('async/map');
map(
['stam.txt', 'stam2.txt', 'stam3.txt'],
function(fileName, callback) {
fs.readFile(path.resolve(__dirname, fileName), function(err, data) {
if (err) return callback(err);
return callback(null, data.toString());
});
},
function(err, results) {
console.log(results);
}
)
for each of the file names in the array we are running a read file and calling the callback with the string result. The last function will be called with all the text of the files.
Control Flow
The next category of utility functions we have in async is called Control Flow We will look for methods in this category if we want to control the order of multiple async functions. The signature of the methods in this category are pretty similar. The first argument will be a collection of async functions, The second argument will be an aggregation of the above functions depending on the function we chose from the async module. Lets examine a few async functions from this category.
Waterfall
We are passing this function an array of async functions. They will not run in parallel, meaning the second function will run when the first is finished. Each function will be called with the arguments passed from the previous function. The final function will get the arguments from the last function in the array. If one of the functions in the array send and error, then the final function will run right away with the error. Lets give an example of this function we a set of timers.
const waterfall = require('async/waterfall');
waterfall([
function(callback) {
setTimeout(() => {
callback(null, 1)
}, 1000)
},
function(numberIsOne, callback) {
console.log('this will run after one second with argument: ' + numberIsOne)
setTimeout(() => {
callback(null, 2);
}, 2000);
},
function(numberIsTwo, callback) {
console.log('this will run after three second with argument: ' + numberIsTwo);
setTimeout(() => {
callback(null, 3);
}, 3000)
}
], function(err, result) {
console.log('this will run after six second with argument: ' + result);
});
What will happen in the following code is this: First the first function in the array will run and will call the callback after 1 second passing the argument 1. After 1 second that the callback is called the second function will run and will get the argument from the previous one which is the number 1. After 2 seconds the second argument will call the callback passing the value 2. The 3 function will start and after 3 seconds will call the callback with the value 3. The final function will run with the result passed from the 3rd function which is 3.
retry
Retry a few times if an async function fails. Example: Lets say we want to send a request to a remote api. The server we are sending the request sometimes fail during high traffic. We want to send the request and if the request is failing we want to send it again up to 5 times. Lets see how this kind of example will look like in code.
const https = require('https');
const retry = require('async/retry');
function sendRequestToApi(callback) {
let data = '';
const req = https.request({
hostname: 'nztodo.herokuapp.com',
port: 443,
path: '/api/task/?format=json',
method: 'GET'
}, function(res) {
res.on('data', (d) => {
data+=d;
});
res.on('end', () => {
callback(null, data);
})
});
req.on('error', (e) => {
callback(e);
})
req.end();
}
retry(3, sendRequestToApi, function(err, response) {
console.log(response);
})
Covering the above example. We are wrapping a function with a callback. The function use the https library to send a request to our todo server. When the response is finished we call the callback with the result. If there is an error we will call the callback with an error. This function in case of error will run 3 times.
Utils
Utils for the most of them decorate an async function, meaning it will give additional power to an async function.
timeout
A nice util which is popular to use is the timeout util. This function will wrap an async function and if the async does not call the callback within the number of miliseconds provided then an error will be throws. The previous example of query a remote api is a perfect example of a common use case for this function. We might want to say that if the server does not return a result withing 3 seconds then the request will fail with a timeout. So the previous example might look like this:
const https = require('https');
const retry = require('async/retry');
const timeout = require('async/timeout');
function sendRequestToApi(callback) {
let data = '';
const req = https.request({
hostname: 'nztodo.herokuapp.com',
port: 443,
path: '/api/task/?format=json',
method: 'GET'
}, function(res) {
res.on('data', (d) => {
data+=d;
});
res.on('end', () => {
callback(null, data);
})
});
req.on('error', (e) => {
callback(e);
})
req.end();
}
retry(3, timeout(sendRequestToApi, 3000), function(err, response) {
console.log(response);
})
exact same example as before but this time we wrapped the function with the timeout and we will retry 3 times where a request must be fulfilled in less then 3 seconds.
Summary
There is 70 function in the async module so every common use case for async functions in your application, you will probably find a suitable function in the async module that will help you deal with your problems. code running asynchronously is one of the major feature of javascript and we use it a lot, but async code is prune to errors and hard to deal with. Having an arsenal of weapons to tackle our async problem is really useful and getting to learn the async module properly will definetly worth your time.
Nerdeez is a software compnay specializing in delivering quality software, educating software teams by giving advanced courses about software technologies, and giving consulting services to software development.