Generating Server-side tile maps with Node.js (Part 2 - Using Mapnik)

Part 1 - Programatically with Canvas
Part 2 - Using Mapnik

On my previous post I've shown how to use node.js to generate server-tiles, painting them "manually" using the canvas API. This is perfect for those scenarios where one wants to overlay dynamic or custom data on top of an existing map. The following images (although done on client-side with canvas) are good examples of this:

Now, suppose we don't want to generate data on top of a map but the map itself. Meaning, a tile more or less similar to this:

This is very complex as it requires loading spatial data, painting the countries, roads, rivers, labels, always taking into account the zoom level, etc. Not trivial at all.


Fortunately there are some frameworks/toolkits that provide some assistance in achieving this. The most notorious one is probably mapnik.

Mapnik provides a descriptive language in XML supporting an incredibly rich feature-set. It can read data from multiple sources, apply various transformations and has lots of cosmetic options. The problem is that this configuration becomes really hard to manage. The easiest way to create this XML is, IMHO, using TileMill (which I've already discussed on an older post).

I'm going to create an incredibly basic map using TileMill, export the Mapnik XML file, create a tile-server using node and Mapnik and show the resulting map on Bing Maps.

Step 1: Open TileMill

Step 2: Create a new project, calling it "Simple"

The default map already contains a world map layer which should be enough for the sake of this post.

I've changed the colour of the countries using the style editor on the right. Basically I'm just throwing around some colours to make it less bland.

Map { background-color: #5ac2d7; }

#countries 
{
  [MAP_COLOR=0] { polygon-fill: gray } 
  [MAP_COLOR=1] { polygon-fill: purple } 
  [MAP_COLOR=2] { polygon-fill: green } 
  [MAP_COLOR=3] { polygon-fill: yellow } 
  [MAP_COLOR=4] { polygon-fill: navy } 
  [MAP_COLOR=5] { polygon-fill: blue } 
  [MAP_COLOR=6] { polygon-fill: darkblue } 
  [MAP_COLOR=7] { polygon-fill: black } 
  [MAP_COLOR=8] { polygon-fill: pink } 
  [MAP_COLOR=9] { polygon-fill: brown } 
  [MAP_COLOR=10] { polygon-fill: darkgreen } 
  [MAP_COLOR=11] { polygon-fill: darkgray } 
  [MAP_COLOR=12] { polygon-fill: gainsboro } 
  [MAP_COLOR=13] { polygon-fill: orange }
}

Yep, an awful looking map :)

By the way, if you want a great tutorial on styling a map on TileMill check this crash-course created by Mapbox.

Now, to create the corresponding Mapnik file just press the "Export > Mapnik XML" button.

I'm saving it on a "Data" folder that will be a sibling to my node app.

Step 3: Install the required components

First things first. We need (besides Node obviously):

• Mapnik
• Node Mapnik module

I'm actually writing this post in a MacOSX computer and my instructions will be targeted at this OS.

• Installing Mapnik:
• Either using Homebrew or using a binary installer the result is pretty much the same. Both are simple and viable options.

• Updating the PYTHONPATH environment variable

Edit the .bash_profile file (I'm using nano on the terminal window but any editor will do):

pico ~/.bash_profile

Add the following line to the file:

export PYTHONPATH="/usr/local/lib/python2.7/site-packages"

For reference, this is what my .bash_profile file looks like

• Now, to make sure that Mapnik is in fact working, open a Python window and just run the "import mapnik" command. If no error appears it should be properly installed.

• Install the Node Mapnik module

npm install mapnik

• Install the Express module

npm install express

Step 4: Create the node code

I'm going to create a really basic server. It parses the x,y,z parameters of the URL, creates a mapnik map based on the XML file created in step 2 and sets the map boundaries to the corresponding coordinates of the map tile.

It will be built using the previous post as a starting-point, were I had an express server set up.

The full code is:

var express = require('express');
var app = express();
app.use(express.compress());

var mapnik = require('mapnik');
mapnik.register_datasources("node_modules/mapnik/lib/binding/mapnik/input");

var mercator = require('./sphericalmercator');

var stylesheet = './Data/Simple.xml';

app.get('/', function(req,res) {
    res.sendfile('./Public/index.html');
});

app.get('/:z/:x/:y', function(req, res) {

    res.setHeader("Cache-Control", "max-age=31556926");

    var z = req.params.z,
        x = req.params.x,
        y = req.params.y;

    var map = new mapnik.Map(256, 256);

    map.load(stylesheet,
        function(err,map) {
            if (err) {
                res.end(err.message);
            }

            var bbox = mercator.xyz_to_envelope(x, y, z, false);
            map.extent = bbox;

            var im = new mapnik.Image(256, 256);
            map.render(im, function(err,im) {
                if (err) {
                    res.end(err.message);
                } else {
                    im.encode('png', function(err,buffer) {
                        if (err) {
                            res.end(err.message);
                        } else {
                            res.writeHead(200, {'Content-Type': 'image/png'});
                            res.end(buffer);
                        }
                    });
                }
            });
        }
    );
    res.type("png");
});

app.listen(process.env.PORT || 8001);

console.log('server running');

Running the server locally:

node App/server.js

Opening a browser window to validate the result:

Some important bits:

• For some reason the datasources couldn't be loaded on mapnik, hence the specific "register_datasources" command
• I'm also serving the static Index.html file, which loads the tiles. The complete code for the client is mostly the same of the one from my last post:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
    <title>Bing Maps Mapnik Layer - Simple Demo</title>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
</head>
<body onload="loadMap();">

<div id='mapDiv' style="width:100%; height: 100%;"></div>

<script type="text/javascript" 
        src="http://ecn.dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=7.0">
</script>
<script type="text/javascript">

    function loadMap()
    {
        var MM = Microsoft.Maps;
        var map = new MM.Map(document.getElementById("mapDiv"), {
            center: new MM.Location(39.5, -8),
            backgroundColor: MM.Color.fromHex('#5ac2d7'),
            mapTypeId: MM.MapTypeId.mercator,
            zoom: 3,
            credentials:"YOUR CREDENTIALS"});

        var tileSource = new MM.TileSource({ uriConstructor:  function (tile) {

            var x = tile.x;
            var z = tile.levelOfDetail;
            var y = tile.y;

            return "/" + z + "/" + x + "/" + y;
        }});

        var tileLayer = new MM.TileLayer({ mercator: tileSource});
        map.entities.push(tileLayer);
    }
</script>
</body>
</html>

You can check the end-result here.

Extra:

As I did on my previous post, I'm going to show the required steps to deploy this to a Linux Virtual Machine on Azure.

I'll just pick up where I left, which already included setting up an FTP server, the proper accesses and NodeJS.

I'll start by opening an additional port. I've already used 8000 for my previous demo so I'll use 8001 for this one.

Now for the missing pieces:

• Install Mapnik

sudo apt-get install -y python-software-properties

sudo add-apt-repository ppa:mapnik/v2.2.0
sudo apt-get update
sudo apt-get install libmapnik libmapnik-dev mapnik-utils python-mapnik

node-mapnik requires protocol buffers to be installed, which in turn requires the g++ compiler. The complete set of required commands is:

• Install g++

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install build-essential
gcc -v
make -v

• Install protobuf

Download source file from here. Extract to some folder and run the following commands inside it.

sudo ./configure
sudo make
sudo make check
sudo make install
sudo ldconfig
protoc --version

• Install node-mapnik (inside the folder for the server)

npm install mapnik

• Set locale-gen

export LC_ALL="en_US.UTF-8"

Now, assuming everything is copied over to Azure (including the shapefile, the mapnik xml, the node server-code) running the server in background will be as simple as:

node App/server.js &

And that's it.