Playing with Mapbox Vector Tiles (Part 1 - End-to-end experiment)

Part 1. End-to-End experiment 
Part 2. Generation custom tiles
Part 3. Using Mabox GL

So, what are vector tiles?

For those that are not familiar with this approach, it provides an alternative to raster tiles where the main difference is that vector tiles provide data instead of a rendered image, although still using a similar xyz tiling structure.

This provides some benefits, such as the ability to have meta-data associated with the tile and being able to generate different presentations for the same information.

One of the problems is that there isn't an official standard for vector tiles. Regardless, with time, a particular vector tile format seems to have gained more traction than all others: the Mapbox Vector Tiles.

Mapbox has provided a consistent ecosystem around it and various providers have started to support it and, on this post, I'm going to play around with some of the existing technology around Mapbox Vector Tiles.

First, showing an actual example from Mapbox: https://www.mapbox.com/mapbox-gl-js/examples/

This page uses:

• Vector tiles hosted by Mapbox
• Mapbox GL: a browser plugin to render Mapbox Vector Tiles using WebGL

The interesting part is that everything is being rendered in client-side, thus allowing for a really smooth transition between zoom levels and lots of flexibility, particularly as the styles are also applied in client-side.

So, how can we use this technology for our own maps?

1. Using Mapbox Studio

The obvious choice is using Mapbox Studio, an online service where you can load data, style and publish the maps without much fuss. Mapbox provides various price ranges, with a free tier that allows 50.000 map views per month, which seems pretty reasonable to test it out.

2. Serving the Mapbox Vector Tiles (mvt) ourselves

Although Mapbox provides a paid service they don't force you to use it. The mvt spec is open and other libs/apps are able to read/write that format. Mapbox mantains a list of such projects here: https://github.com/mapbox/awesome-vector-tiles

So, to setup a local server there are various choices but I found the easiest one to be tileserver-gl. Setting it up was a breeze using npm:

npm install -g tileserver-gl-light

and to run it:

tileserver-gl-light <map>

Note: There's a "tileserver-gl" and a "tileserver-gl-light". The full version also supports a model were it renders the vector tiles in server-side and serves them as rasters to the client. I'm using the light version for simplicity.

So now we need the actual map data to display. https://openmaptiles.org/downloads/ provides MVT files for the whole world, ready to use. I've downloaded the Portuguese map using the following command:

curl -o portugal.mbtiles https://openmaptiles.os.zhdk.cloud.switch.ch/v3.3/extracts/portugal.mbtiles

Then I can just run the server as:

tileserver-gl-light portugal.mbtiles

It should be running properly on http://localhost:8080

http://localhost:8080/styles/osm-bright/?vector#5.91/40.357/-5.857

Zooming in we can see that the tile info is being fetched from the server and rendered accordingly.

http://localhost:8080/styles/osm-bright/?vector#15.91/38.7109/-9.3284

Also, to prove that the rendering is actually being done in client-side, one can actually use a different style for the same data, which yields totally different results. Tile-server gl actually includes two different styles by default: osm-bright and klokantech-basic. Changing the "osm-bright" part of the url to "klokantech-basic":

http://localhost:8080/styles/klokantech-basic/?vector#15.91/38.7109/-9.3284

3. Changing the style

Now I'm going to create a new style for the map. Creating one from scratch is very hard-work so I'll simply copy an existing one and change it slightly.

Some steps required to do this in tileserver-gl:

• Run the server with a --verbose argument to know the default config

It will show the default configuration file, including a section with the root path under /options/paths/root

Automatically creating config file for portugal.mbtiles
{
  "options": {
    "paths": {
      "root": "/Users/pedrosousa/.nvm/versions/node/v6.10.0/lib/node_modules/tileserver-gl-light/node_modules/tileserver-gl-styles",
      "fonts": "fonts",
      "styles": "styles",
      "mbtiles": "/Users/pedrosousa/Projects/vectorTiles"
    }
  },

• On that root folder there is a "styles" subolder with a folder for each style, particulary "klokantech-basic" and "osm-bright". I'm going to copy "klokantech-basic" as it's simpler to a new folder called "custom"

• Inside I'm just going to change a couple of properties:
• Name: "Custom"
• Background-Color to white: "rgba(255, 255, 255, 1)"   
• Water Fill-Color to be a dark blue: "rgba(12, 55, 84, 1)"

• Last, I need to add this new style to the tileserver-gl configuration file. As I've been using the default one I actually need to create a new file. The easiest way is to copy the output obtained on the first step and just add the new style copying one of the others. For reference this is my file:

{
  "options": {
    "paths": {
      "root": "/Users/pedrosousa/.nvm/versions/node/v6.10.0/lib/node_modules/tileserver-gl-light/node_modules/tileserver-gl-styles",
      "fonts": "fonts",
      "styles": "styles",
      "mbtiles": "/Users/pedrosousa/Projects/vectorTiles"
    }
  },
  "styles": {
    "klokantech-basic": {
      "style": "klokantech-basic/style.json",
      "tilejson": {
        "bounds": [
          -31.6575302,
          29.7288021,
          -6.0891591,
          42.2543112
        ]
      }
    },
    "custom": {
      "style": "custom/style.json",
      "tilejson": {
        "bounds": [
          -31.6575302,
          29.7288021,
          -6.0891591,
          42.2543112
        ]
      }
    },
    "osm-bright": {
      "style": "osm-bright/style.json",
      "tilejson": {
        "bounds": [
          -31.6575302,
          29.7288021,
          -6.0891591,
          42.2543112
        ]
      }
    }
  },
  "data": {
    "v3": {
      "mbtiles": "portugal.mbtiles"
    }
  }
}

• Now you need to launch the server with a "-c" property to specify the new configuration file that was just created. Assuming it's called "config.json":

tileserver-gl-light portugal.mbtiles -c config.json

• Voilá. Now opening the browser with the corresponding "/custom" style will show the updated map style

4. Serving the vector tiles directly without a tile-server

Although I would recommend having a tile-server (particularly with a nginx front-facing it) sometimes one might want to serve the individual tiles directly.

The easiest way is probably to use another tool from mapbox called mb-util to extract the tiles from the .mbtiles file.

I've used the steps that I've found on this page: https://github.com/klokantech/vector-tiles-sample

1. After installing mbutil you can run the following command on the folder where you have your mbtiles file

mb-util --image_format=pbf portugal.mbtiles portugal

2. We still need to extract them as they're usually gzipped.

gzip -d -r -S .pbf *

3. After extracting the pbf extension is lost. The following command iterates the various files and puts a .pbf extension on them again:

find . -type f -exec mv '{}' '{}'.pbf \;

Ok, now that we have our tiles we're ready to serve them. We just need a very simple index.html file:

<!DOCTYPE html>
<html>

<head>
    <meta charset='utf-8' />
    <title>Vector Map with Mapbox GL JS</title>
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
    <style>
    body {
        margin: 0;
        padding: 0;
    }
    #map {
        position: absolute;
        top: 0;
        bottom: 0;
        width: 100%;
    }
    </style>
</head>

<body>
    <script src='https://api.mapbox.com/mapbox-gl-js/v0.32.1/mapbox-gl.js'></script>
    <div id='map'></div>
    <script type="text/javascript">

    var map = new mapboxgl.Map({
        container: 'map',
        center: [12, 1],
        zoom: 1,
        style: 'style.json'
    });
    </script>
</body>

</html>

Most of the relevant information comes from the style.json file that I'm referencing on the map definition.

I'm not going to put the entire style file here as it's quite big and is mostly a copy from the klokantech-basic I've mentioned above. The relevant bits are:

{
  "sources": {
    "openmaptiles": {
      "type": "vector",
      "tiles": [
        "http://localhost:8000/portugal/{z}/{x}/{y}.pbf"
      ]
    }
  }
}

Basically setting up the source of the tiles to the folder that we've extracted.

Now running a simple web-server on the folder (ex: using Python Simple Server)

python -m SimpleHTTPServer

Opening http://localhost:8000 should show a map similar to the one we had before using tileserver-gl, although with a slightly different url pattern. The tiles are being served directly from the web-server without any translation layer in the middle

Ok, that's it for now. Next post: generating my own vector tiles