Write simple, maintainable Nodejs

@fantasyni from Pomelo Netease

About me

• github: @fantasyni
• yixin, wechat: fantasyni
• core member from pomelo at Netease
• node.js hacker, over 3 years node experiences
• not just a code farmer, but a code poet

Category

• Overview
• Framework
• Practice
• Conclusion

Overview -- What is Node.js

• In a nutshell: server-side javaScript
• Used to build fast, scalable network applications
• Built on Google's v8 javaScript engine
• Event-driven, non-blocking(asynchronous) I/O

Overview -- What is Node.js

• Small core, vibrant community
• Extreme modularity
• Reimplement everything in Javascript

Overview -- What is Node.js

Overview -- Node.js in the industry

PayPal measured a 2x increase in developer productivity and high performance (2x)

Overview -- Node.js in the industry

Pomelo industry

Overview -- Node.js in the industry

• Everything seems to be fine, but can be done better
• What is the problem and pain in current node.js development ?

Goal: Simple and Maintainable

Overview -- Node.js development

require exports is not good enough

• you have to know where the file is, and what it is really exported and then do a lot everytime after require

require a constructor function

var B = require('b');
var b = new B();
b.run();

require a function

var B = require('b');
B();

require an object

var B = require('b');
B.run();

it seems to be nothing serious... however when you require a lot...

Overview -- Node.js development

require exports is not good enough

With simple require and exports, caller and callee are tightly coupled, developers have to deal with it everytime

Overview -- Node.js development

Consistent development

Node is quite flexible, developers can write several styles of code, like callback, promise, generator with koa, function programming, OOP programming etc...

app.get('/', function(req, res){
  res.send('Hello World');
});

Q.fcall(promisedStep1)
.then(promisedStep2)
.done();

var koa = require('koa');
var app = koa();

app.use(function *(){
  this.body = 'Hello World';
});

app.listen(3000);

Inconsistent code stypes, makes it hard to maintain especially in large team work projects

Overview -- Node.js development

Consistent configuration

var serverConfig = require('../../config/server');
var redis = require("redis");
var client = redis.createClient(serverConfig['redisPort'], serverConfig['redisHost']);

client.on("error", function(err) {
  console.error("redis error " + err);
});

client.on("ready", function() {
  console.log("redis is ready");
});

module.exports = client;

you require a config file, and set the redis connection

however, when you switch environments which means serverConfig may be changed, you can not do well with it

Overview -- Node.js development

How to avoid these ?

How to make node.js development simple and maintainable ?

Solution: framework

Framework -- Bearcat

Bearcat is a POJOs based application framework which provides a lightweight container for writing simple, maintainable node.js, it is designed to solve all of these painful things.

Simple POJOs + Configuration metadatas = Elastic, maintainable system

Framework -- Bearcat

the main concept of Bearcat is:

• POJO
• IoC
• AOP
• Consistent configuration

Framework -- Bearcat -- POJO

What is POJO ?

• POJO === Plain Old Java Object in java Platform
• POJO === Plain Old JavaScript Object in node.js Platform

var POJO = function() {
    this.props = null;
}

POJO.prototype.method = function() {

}

module.exports = POJO;

Why should we use POJO ?

• POJO is simple, everyone can write it
• POJO makes development consistently, all your codes is POJOs
• POJO makes it friendly to document

Framework -- Bearcat -- IoC

• Inversion of Control (IoC) is a design pattern that addresses a component's dependency resolution, configuration and lifecycle.
• IoC is best understood through the Hollywood Principle: "Dont's call us, we'll call you".
• Dependency injection (DI), components do not look up, they provide plain simple configuration metadata enabling the container to resolve dependencies.

Framework -- Bearcat -- IoC

Without IoC

var Engine = require('./engine');
var Wheel = require('./wheel');

var Car = function() {
  this.engine = new Engine();
  this.wheel = new Wheel();
}

Car.prototype.run = function() {
  this.engine.run();
  var res = this.wheel.run();
  console.log('run car...');
  return 'car ' + res;
}

module.exports = Car;

car have to require engine and wheel, and then instantiate with constructor functions, it is tightly coupled

Framework -- Bearcat -- IoC

With IoC

var Car = function($engine) {
  this.$id = "car";
  this.$engine = $engine;
  this.$wheel = null;
}

Car.prototype.run = function() {
  this.$engine.run();
  var res = this.$wheel.run();
  console.log('run car...');
  return 'car ' + res;
}

module.exports = Car;

car do not have to know where engine, wheel are from, and how they are being instantiated

Framework -- Bearcat -- IoC

How to make IoC work ?

just add a simple configuration metadata file context.json

{
  "name": "simple_inject",
  "scan": ""
}

Framework -- Bearcat -- IoC

Bearcat Startup

var Bearcat = require('bearcat');
var contextPath = require.resolve('./context.json');

var bearcat = Bearcat.createApp([contextPath]);
bearcat.start(function(){
   var car = bearcat.getBean('car'); // get bean
   car.run(); // call the method
});

[2014-05-04 18:50:41.996] [INFO] bearcat - [app] Bearcat startup in 6 ms
run engine...
run wheel...
run car...

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Scopes

By default, scope is singleton

var Car = function($engine) {
  this.$id = "car";
  this.$scope = "singleton";
  this.$engine = $engine;
  this.$wheel = null;
}

Car.prototype.run = function() {
  this.$engine.run();
  var res = this.$wheel.run();
  console.log('run car...');
  return 'car ' + res;
}

module.exports = Car;

var car1 = bearcat.getBean('car');
var car2 = bearcat.getBean('car');
// car2 is exactly the same instance as car1

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Scopes

you can set scope to prototype

var Car = function($engine) {
  this.$id = "car";
  this.$scope = "prototype";
  this.$engine = $engine;
  this.$wheel = null;
}

Car.prototype.run = function() {
  this.$engine.run();
  var res = this.$wheel.run();
  console.log('run car...');
  return 'car ' + res;
}

module.exports = Car;

var car1 = bearcat.getBean('car');
var car2 = bearcat.getBean('car');
// car2 is not the same instance as car1

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Lifecycle callbacks

Initialization method

var Car = function() {
    this.$id = "car";
    this.$init = "init";
    this.num = 0;
    this.$engine = null;
}

Car.prototype.init = function() {
    console.log('init car...');
    this.num = 1;
    return 'init car';
}

Car.prototype.run = function() {
    this.$engine.run();
    console.log('run car...');
    return 'car ' + this.num;
}

module.exports = Car;

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Lifecycle callbacks

Destruction method

var Car = function() {
  this.$id = "car";
  this.$destroy = "destroy";
}

Car.prototype.destroy = function() {
    console.log('destroy car...');
    return 'destroy car';
}

Car.prototype.run = function() {
    console.log('run car...');
    return 'car';
}

module.exports = Car;

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Lifecycle callbacks

Async Initialization method

var Car = function() {
    this.$id = "car";
    this.$init = "init";
    this.$order = 2;
    this.num = 0;
}

Car.prototype.init = function() {
    console.log('init car...');
    this.num = 1;
    return 'init car';
}

Car.prototype.run = function() {
    console.log('run car...');
    return 'car ' + this.num;
}

module.exports = Car;

Framework -- Bearcat -- IoC

More magic in Bearcat IoC

• Lifecycle callbacks

Async Initialization method

var Engine = function() {
  this.$id = "engine";
  this.$init = "init";
  this.$async = true;
  this.$order = 1;
}

Engine.prototype.init = function(cb) {
    console.log('init engine...');
    setTimeout(function() {
        console.log('asyncInit setTimeout');
        cb();
    }, 1000);
}

Engine.prototype.run = function() {
    console.log('run engine...');
    return 'wheel';
}

module.exports = Engine;

Framework -- Bearcat -- IoC

More details abount Bearcat IoC

• Visit Bearcat wiki IoC-Container
• Chinese version is IoC-Container-Chinese

Framework -- Bearcat -- AOP

Aspect-Oriented Programming (AOP) -- key unit -- aspect.

Object-Oriented Programming (OOP) -- key unit -- class.

Aspects enable the modularization of concerns such as transaction management that cut across multiple types and objects.

var Car = function() {
}

Car.prototype.run = function() {
  // pointcut
  // advice code ...
  console.log('Car run...');
  // pointcut
  // advice code ...
}

Framework -- Bearcat -- AOP

aspect -- log -- without AOP

var Car = function(engine) {
  this.engine = engine;
  this.wheel = null;
  this.log = null;
}

Car.prototype.run = function() {
  this.log.info('log...');
  this.engine.run();
  var res = this.wheel.run();
  console.log('run car...');
  return 'car ' + res;
}

module.exports = Car;

Framework -- Bearcat -- AOP

aspect -- log -- without AOP

var Engine = function() {
    this.log = null;
}

Engine.prototype.run = function() {
  this.log.info('log...');
  console.log('run engine...');
  return 'engine';
}

module.exports = Engine;

Framework -- Bearcat -- AOP

aspect -- log -- without AOP

var Wheel = function() {
    this.log = null;
}

Wheel.prototype.run = function() {
  this.log.info('log...');
  console.log('run wheel...');
  return 'wheel';
}

module.exports = Wheel;

Framework -- Bearcat -- AOP

aspect -- log -- with AOP

• Declare an aspect

var Aspect = function() {
  this.$id = "aspect";
  this.$aop = true;
}

Aspect.prototype.doBefore = function(next) {
  var $pointcut = "before:.*?run";
  console.log('log...');
  next();
}

module.exports = Aspect;

Framework -- Bearcat -- AOP

aspect -- log -- with AOP

[2014-05-04 18:50:41.996] [INFO] bearcat - [app] Bearcat startup in 6 ms
log...
run engine...
log...
run wheel...
log...
run car...

Framework -- Bearcat -- AOP

aspect -- transaction -- with AOP

Transaction management is a good example of a AOP crosscutting concern in enterprise Node.js applications

node-mysql provides simple transaction support at the connection level:

connection.beginTransaction(function(err) {
  connection.query('INSERT INTO posts SET title=?', title, function(err, result) {
    if (err) { 
      return connection.rollback(function() {});
    }
    var log = 'Post ' + result.insertId + ' added';
    connection.query('INSERT INTO log SET data=?', log, function(err, result) {
      if (err) { 
        return connection.rollback(function() {});
      }  
      connection.commit(function(err) {
        if (err) { 
          return connection.rollback(function() {});
        }
        console.log('success!');
      });
    });
  });
});

Framework -- Bearcat -- AOP

aspect -- transaction -- with AOP

with Bearcat AOP, transaction will be simpler

SimpleService.prototype.testMethodTransaction = function(cb, txStatus) {
    var self = this;
    this.simpleDao.transaction(txStatus).addPerson(['aaa'], function(err, results) {
        if (err) {
            return cb(err); // if err occur, rollback will be emited
        }
        self.simpleDao.transaction(txStatus).getList([1, 2], function(err, results) {
            if (err) { 
                return cb(err); // if err occur, rollback will be emited
            }
            cb(null, results); // commit the operations
        });
    });
}

more details can be refered to bearcat-dao transaction

Framework -- Bearcat -- Consistent configuration

Configurations are actually properties in POJO, therefore, you can use DI to solve it

var Car = function() {
  this.$id = "car";
  this.$Vnum = "${car.num}";
}

Car.prototype.run = function() {
  console.log('run car' + this.$Vnum);
  return 'car' + this.$Vnum;
}

module.exports = Car;

Framework -- Bearcat -- Consistent configuration

• Using placeHolders

placeHolder to be replaced by the specific envrioment value

${car.num}

then in config.json file you can define car.num with the specific value

{
    "car.num": 100
}

Framework -- Bearcat -- Consistent configuration

• Environment configuration

Different environment configurations:

├─┬ placeholderSample/
│ ├─┬ config/
│ │ └─┬ dev/
│ │ │ └── car.json
│ │ └─┬ prod/
│ │   └── car.json
│ └── car.js
└── context.json

• dev sub-directory -- development envrioment configurations
• prod sub-directory -- production envrioment configurations

Framework -- Bearcat -- Consistent configuration

• Switching environment

Setup env at the startup command, by default the env is dev

Run with env or --env args

node app.js env=prod

Run with NODE_ENV

NODE_ENV=prod node app.js

Framework -- Bearcat -- Practice

• Simple web todo

the whole repository is bearcat-todo, forked from fengmk2-todo

Framework -- Bearcat -- Practice

• Simple web todo

Project code structure -- MVC driven

├─┬ app/
│ ├─┬ controller/
│ │ └── todoController.js
│ ├─┬ dao/
│ │ └── todoDao.js
│ ├─┬ domain/
│ │ └── todoDomain.js
│ ├─┬ service/
│ │ └── todoService.js
├─┬ config/
│ ├─┬ dev/
│ │ └── mysql.json
│ ├─┬ prod/
│ │ └── mysql.json
├── views/
├── public/
├── server.js
├── package.json
├── context.json
└── RERAME.md

Framework -- Bearcat -- Practice

• Simple web todo

todoController.js -- simple POJO

var TodoController = function() {
  this.$id = "todoController";
  this.$todoService = null;
}

TodoController.prototype.index = function(req, res, next) {
    res.render('index.html', {});
}

module.exports = TodoController;

Framework -- Bearcat -- Practice

• Simple web todo

Setup Bearcat and add route

var contextPath = require.resolve('./context.json');

var bearcat = Bearcat.createApp([contextPath]);

bearcat.start(function() {
  /**
   * Routing
   */
  var router = urlrouter(function(app) {
    app.get('/', bearcat.getRoute("todoController", "index"));
  });
  app.use(router);

  // start app
  app.listen(config.port);
  console.log('Server start on ' + config.port);
});

Framework -- Bearcat -- Practice

• Simple web todo

Setup Bearcat and add route

context.json setup scan path to enable auto-scan POJOs

{
  "name": "bearcat-todo",
  "scan": "app"
}

Framework -- Bearcat -- Practice

• Simple web todo

Add todoService and todoDao

todoService.js

var TodoService = function() {
  this.$id = "todoService";    
  this.$todoDao = null;
}

TodoService.prototype.getList = function(params, cb) {
  return this.$todoDao.getList(params, cb);
}

module.exports = TodoService;

Framework -- Bearcat -- Practice

• Simple web todo

Add todoService and todoDao

todoDao.js

var TodoDomain = require('../domain/todoDomain');

var TodoDao = function() {
  this.$id = "todoDao";
  this.$init = "init";
  this.$domainDaoSupport = null;
}

TodoDao.prototype.init = function() {
  this.$domainDaoSupport.initConfig(TodoDomain);
}

TodoDao.prototype.getList = function(params, cb) {
  var sql = ' 1=1 order by finished asc, id asc limit ?,?';
  return this.$domainDaoSupport.getListByWhere(sql, params, null, cb);
}

module.exports = TodoDao;

Framework -- Bearcat -- Practice

• Simple web todo

Update todoController

todoController.js

TodoController.prototype.index = function(req, res, next) {
  this.$todoService.getList([0, 50], function(err, results) {
    if (err) {
      console.log(err);
      return;
    }
    res.render('index.html', {
      todos: results
    });
  });
}

Framework -- Bearcat -- Practice

• Simple web todo

Just run it

node server.js

Framework -- Bearcat -- Practice

• Simple web todo

How to set up mysql configuration ?

magic is also in context.json

{
  "name": "bearcat-todo",
  "dependencies": {
    "bearcat-dao": "*"
  },
  "scan": "app",
  "beans": [{
    "id": "mysqlConnectionManager",
    "func": "node_modules.bearcat-dao.lib.connection.sql.mysqlConnectionManager",
    "props": [{
      "name": "port",
      "value": "${mysql.port}"
    }, {
      "name": "host",
      "value": "${mysql.host}"
    }, {
      "name": "user",
      "value": "${mysql.user}"
    }, {
      "name": "password",
      "value": "${mysql.password}"
    }, {
      "name": "database",
      "value": "${mysql.database}"
    }]
  }]
}

Framework -- Bearcat -- Practice

• Simple web todo

Set up mysql configuration

mysql.json in dev

{
  "mysql.port": 3306,
  "mysql.host": "localhost",
  "mysql.user": "root",
  "mysql.password": "test",
  "mysql.database": "bearcat_test"
}

Framework -- Bearcat -- Practice

• Simple web todo

Swith to production env ?

mysql.json in prod

{
  "mysql.port": 3306,
  "mysql.host": "10.123.22.3",
  "mysql.user": "todo_online",
  "mysql.password": "todo_online_aqz",
  "mysql.database": "bearcat_todo_online"
}

Framework -- Bearcat -- Practice

• Simple web todo

Just run it

node server.js

node server.js env=prod

Framework -- Bearcat -- Conclusion

Bearcat -- lightweight container

• Dependency injection implementated Inversion of Control (IoC) to manage POJOs
• AOP based declarative middleware services
• Consistent development and configuration

Framework -- Bearcat -- Conclusion

Brings to node

• Simple meta configuration and POJOs
• Reuseable: POJOs can be used to any development environment
• Invasive: there's no dependence on a container API
• Modular: you just use only those parts that you need
• One stop shop: no need for a dedicated Singleton or factory for every object etc

Framework -- Bearcat -- Conclusion

Simple POJOs + Configuration metadatas = Elastic, maintainable system

• Homepage: http://bearcatnode.github.io/bearcat/
• Mailing list: https://groups.google.com/group/bearcatnode/
• Wiki: https://github.com/bearcatnode/bearcat/wiki/
• Issues: https://github.com/bearcatnode/bearcat/issues
• Tags: nodejs, POJO, IoC, AOP, consistent configuration

QA