JavaScript

and stuff

Node.js & real-time video encoding

So I’m doing some slides for my upcoming kind of a talk about Node.js, here in Lviv, at the office. We had one already about Node’s internal stuff and event loop on a really low level, like CPU caches and I/O. I’m going to talk about actual code, API, use cases and popular libs.

Besides just a bunch of words I wanted to show some code and the best way is a working demo application. Even better would be to have an app with all major Node’s use cases presented. They are: fast prototyping, streaming data, JSON API development, real-time and single page applications. It seems for me like video sharing service app would be great for that.

There are AngularJS and Bootstrap on front-end, and Express, MongoDB, Passport and Socket.io on server-side. Every single part is relatively easy to start playing with, if you are ok with JavaScript and back-end stuff. Express is serving a single page where Angular is running the app, the main use of it is REST API. It’s pretty simple and self-descriptive as it should be. Passport manages users auth using API token strategy. And all data is stored in MongoDB, which is much more great with Mongoose, it brings you a set of useful features like schemas and virtual methods.

I’ve been looking across the web for more info about real-time video encoding from Node perspective, but haven’t found much. Before we dive into server-side code, there’s a thing worth to know about Angular and XHR file uploading: content-type header should be multipart/form-data including boundary parameter which is presented in FormData object as well. This is handled by browser, I’ve seen some code for that, not sure if it works. So you can’t just set a header, the best way is to let the browser to figure out the content of the data and so it can set appropriate header. This can be done by setting content-type to false in Angular, at least this is what I’ve seen people are doing on the web. As it turned out, it no longer works (Angular 1.2.10), use undefined instead.

Node.js streams are great, if you haven’t used them before or just can’t figure out how to use it, read an article about “Node.js Stream Playground” and play with streams online. In Node.js, streams is what you need when talking about real-time, it allows you to consume and process data on the fly, which means you don’t need to wait until the end of the upload process. With streams you are working with chunks of incoming data. Node’s streams can be both writable and readable as well as piped one into another, which is a UNIX way of doing things. Both stdin and stdout are streams too, so Node can pipe some data into a separate process and get it back.

For video encoding I’m using avconv, which is a new ffmpeg on Ubuntu and multiparty form data parser instead of Express built-in formidable, it kind of same, but simpler and I like its API. Btw, if you are going to use separate form parser, you should not use express.bodyParser(), just replace it with express.json() and express.urlencoded(), both are included into bodyParser along with formidable. Data parsing, encoding and writing to file system is simple as it sounds thanks to streams:

var multiparty = require('multiparty'),
    spawn = require('child_process').spawn,
    fs = require('fs');

var form = new multiparty.Form(), // Init form data parser
    args = ['-i', 'pipe:0', '-f', 'webm', 'pipe:1'], // Set args, define i/o streams
    avconv = spawn('avconv', args), // Spawn avconv process
    output = fs.createWriteStream('./output.webm'); // Write to file system

form.on('part', function (part) { // Listen on parsed `part`
      if (part.filename) { // Only file has a 'filename' property
        part.pipe(avconv.stdin); // Write chunks of file into avconv standard input
      }
});

avconv.stdout.pipe(output); // Write encoded chucks to the file system

form.parse(req, function (err, fields) { // Run parsing process
      if (err) return console.log(err);
});

That’s it, it works, unless you are not on really high-end machine, which is able to process data faster than it can be uploaded. That’s what I ran into when did some testing. Video encoding is pretty heavy task you know, and when incoming data can’t be handled fast enough — all real-time things will not work at all. Both readable and writable streams in Node.js has a Buffer object which holds a chunk of incoming data and will pass it further on a pipeline when the next stream is ready to consume it (when its buffer becomes empty). So when you run this code on a low-end hardware, and especially on localhost, every single video file you pass will be fully uploaded as fast as avconv will done with encoding. This makes no sense. To make things run truly in real-time you need a really fast machine or to limit bandwidth, which is applicable for testing purpose only. Otherwise: upload first, then encode. Not real-time, but still doing its job.

The application I’m working on is available on GitHub, maybe you can contribute some optimizations?