Power up your node.js REST API

Building micro services can be very exciting. The ease and speed of creating them using node.js has changed the way devs think. From requirement analysis to continuous integration, micro services can be developed within weeks and operate quite impressively.
Using node.js you can create very robust apps with asynchronous calls by design. Still though there are some times where you may add up a little more in that and make your application even stronger and harder to collapse on tough situations like an unpredictable request that destroys the process or too much concurrent requests or maybe a dos attack from a hacker.
With this post I want to present some node modules that will help you overcome problems like these.

Module 1) cluster. This module comes in handy when you want to overcome memory limitations and processing power when using a single process app. Apart from that, will give you the ability to respawn dead node processes (like pm2 does).
Useful links:
http://www.sitepoint.com/how-to-create-a-node-js-cluster-for-speeding-up-your-apps/
https://nodejs.org/api/cluster.html

Module 2) express.  It is one of the best modules for creating HTTP endpoints. Some people prefer other modules like restify – and this is quite ok. Just use the one you think it fits more to you. Just keep in mind that in terms of speed express might be the best one.

Module 3) ddos. This module will give you some basic dos protection against malicious users. Probably you will never need it but again why risking it? Note though that ddos module will only protect you against simple dos attacks. If you also want protection against ddos attacks you will need something more. Personally, I believe that this is the work of the load balancer. Those that using Haproxy read this: http://blog.haproxy.com/2012/02/27/use-a-load-balancer-as-a-first-row-of-defense-against-ddos/ .
Useful links:
https://github.com/rook2pawn/node-ddos

Module 4) toobusy-js. This is one of the many implementations of the original toobusy module. What it does? When receiving an http request instead of trying to execute it, it first checks the load of the process. If the process is overburdened it returns a 503 response telling that the server is busy at the moment. So instead of having too many failures with timeouts you will return on some requests service unavailable fast and serve the rest normally.
Useful links:
https://www.npmjs.com/package/toobusy-js
https://hacks.mozilla.org/2013/01/building-a-node-js-server-that-wont-melt-a-node-js-holiday-season-part-5/

Module 5) bunyan. Yes it is a logger but who can live without a logger. Just be very cautious with production environments to only log what is necessary. Also bunyan’s log rotation using cluster is buggy so you should better avoid it. You should anyway let the operating system do the job with logrotate.

For your convenience I have created a github repo here with a template project which uses all the previous modules. Feel free to clone it and use it as your as a skeleton for your new project
Have fun!

Useful links:

https://github.com/mdagis/api-template

Install Node.js on EC2 Linux AMI

Recently I wanted to install  node.js on ec2 linux machine and to my surprise this task turned to be a little more tedious than I thought it would be. The reason for that is mainly because there are many ways to do it each of them with their pros and cons. The only alternative that I found easier to follow was to run the command: "sudo yum install nodejs npm --enablerepo=epel" . Unfortunately the epel repository has a very old version of node so I had to stick with a better option.

In this topic I am going to cover the option to download the binaries and just add the node directory to path.

The steps:

1. Go to https://nodejs.org/dist/ and select the version of node.js that you want. Normally you will need the latest. When I did the installation i used https://nodejs.org/dist/v4.2.1/node-v4.2.1-linux-x64.tar.gz .

2. After selecting the url that you want go to your machine and create a folder to have your node installation (be sure that the owner of the folder is ec2-user, if not change it with chown). In my case I created /opt/ . Then download your package with the command: wget https://nodejs.org/dist/v4.2.1/node-v4.2.1-linux-x64.tar.gz

3. Uncompress the file with command: tar -xvzf node-v4.2.1-linux-x64.tar.gz  and then navigate to created folder. In my case it was node-v4.2.1-linux-x64

4. The last step now is to add the node bin directory to path. To do so just create a new file in /etc/profile.d/node.sh . Edit the file and add the line: pathmunge /opt/node-v4.2.1-linux-x64/bin/ . Save the file and then run: source /etc/profile to refresh profile and you are ready.


This should normally be enough to have your node installation up and running.

How to survive from Node.js and JavaScript callbacks

Many people coming from Java ecosystem to Node.js (like I did) are facing problems to adjust with the new platform philosophy since the basic principles of JavaScript itself are way different from the other well-known Object Oriented Languages.

Personally I think the biggest problem that I had was to understand the nature of functions in JavaScript and subsequently the way callbacks behave.

The truth is the first time I read about them I thought that I understood them. Unfortunately understanding them was not enough because all those years of experience turned into a barrier for me.

I will try to present the conceptual problem that I had in the beginning and also what I did in order to overcome it.


THE PROBLEM


Let’s say that you want to make a simple program that does 3 steps.

Step1. It reads a file.

Step 2. It accepts a prompt from console with username and password.

Step 3. It sends an email connecting to the mail server using the given credentials and having as a body the file contents read from step 1.


Normally with a conventional OO language you would create 3 functions each one doing one step and call them all three from the body of a third with the order 1, 2, 3.

Most programming languages are doing things synchronously by default which means that step 2 will be executed after 1 and 3 after 2.

This is a sample of something we could do in Java. For the sake of the example please ignore the static and the lack of OO architecture.

public static void main(String[] args) {
        readfile();
        getCredentials();
        sendMail();
}

    public static void readfile(){}

    public static void getCredentials(){}

    public static void sendMail(){}


Unfortunately (and fortunately) the following work flow would not work in Node.js. When your code reaches the step 2 the result of step 1 will not be ready and on step 3 the result of step 2 will not be ready either. The reason is that most modules of node.js work asynchronously and that requires a more delicate approach.


THE SOLUTION


As a matter of fact there are 3 solutions.

A) Using callbacks properly

B) Using nested callbacks

C) Using Promises.

I will skip solution B and C because B is not a decent solution since the readability of the code would be terrible and C because it is a different topic.

I will try to focus on solution (A).


As we saw on the previous code we had 3 functions doing the work and one more calling those 3 with a sequence.

In Node.js you will only need the fourth function to trigger the Step 1 and passing as a parameter the function that knows what to do after Step 1 ends.

Note that the fourth function - let’s call it trigger function – will not do anything more than calling step 1. The callback function 1 will be responsible to acquire the result of step 1 and trigger step 2 passing a new callback to step 2. Call back function 2 will be called by step 2 after finishing and that will call step 3.


Here is a diagram showing the procedure:

Here is some ample code:


function trigger(){
    readfile(callBack1);
}

function callBack1(){
    getCredentials(callBack2);
}

function callBack2(){
    sendMail();
}

// Step 1

function readfile(nextStep) {
    fs.readfile(path, function (error, text) {
        nextStep();
    });
}

// Step 2

function getCredentials(nextStep) {
    prompt.getCredentials(value, function (error, result) {
        nextStep();
    });
}

// Step 3

function sendMail() {
    transporter.sendMail(mailOptions, function (error, info) {
        //done
    });
}

Implement Simple Websocket with Node.js

It's been almost two months since I wrote anything in here and before moving forward in topics I will try to add something more with the web socket technology.

As I have written in the last topic here I 'd like to present a very simple way to create Websocket server. This was implemented with Java EE7.

This time I wanted to try to use Node.js and to my surprise things got much easier.

The only thing I had to do (apart from installing Node.js) was to install the module ws with the command:

npm install ws

That's it! Now the following code is a sample of doing the whole thing:

... and in case you haven't read the previous post here is the html5 client to test it:

Mastering MEAN: Introducing the MEAN stack

MEAN stack is today's favorite development technology stack (previously was LAMP) for web developers. Here is and entry point tutorial for those that are interested.

More resources:

1. http://mean.io/
2. Rails to MEAN