Generating Server-side tile maps with Node.js (Part 2 - Using Mapnik)

Part 1 - Programatically with Canvas
Part 2 - Using Mapnik

On my previous post I've shown how to use node.js to generate server-tiles, painting them "manually" using the canvas API. This is perfect for those scenarios where one wants to overlay dynamic or custom data on top of an existing map. The following images (although done on client-side with canvas) are good examples of this:

Now, suppose we don't want to generate data on top of a map but the map itself. Meaning, a tile more or less similar to this:

This is very complex as it requires loading spatial data, painting the countries, roads, rivers, labels, always taking into account the zoom level, etc. Not trivial at all.

Generating server-side tile-maps with Node.JS (Part 1 - Programatically with Canvas)

Part 1 - Programatically with Canvas
Part 2 - Using Mapnik

This will be a series of posts on leveraging Node.js to generate tile-images on server-side that will be displayed on top of Bing Maps (although applicable to any other mapping API).

For those not familiar with it, Node has gained lots of momentum on the web. It provides a simple event oriented, non-blocking architecture on which one can develop fast and scalable applications using Javascript.

It lends itself really well for generating tile-images and is in-fact being used by some large players like Mapbox.

On this first post I'm going to generate very simple tiles. I won't fetch data from the database nor do anything too fancy. Each tile will simply consist on a 256x256 png with a border and a label identifying the tile's z/x/y. Conceptually something like this:

I'm going to use an amazing Node module called Canvas which provides the whole HTML5 Canvas API on server-side. My main reason is that it's an API that I know particularly well and I can reuse lots of my existing code.

I'm going to develop on Mac OS X but most of this would be exactly the same on Windows or Linux.

Afterwards, as an extra, I'm also going to provide instructions on how to setup a Linux Virtual Machine in Azure running the resulting node application.

UTFGrid in Bing Maps (Part 2 - Bing Maps Module)

Part 1 - Introduction
Part 2 -Bing Maps Module

On this post I'm adapting my previous UTFGrid code and packaging it into a very easy to use Bing Maps Module.

I'm going to show a couple of simple demos on how to use it.

Demo #1 - Basic (try it here)

Very simple demo. Click a country and an alert message will display its name. All this without any round-trip to the server.

Note: in all these demos there are only 4 zoom levels of UTFGrid tiles. Thus, if zooming paste the satellite images, nothing should happen when clicking/hovering the map.

UTFGrid in Bing Maps (Part 1 - Introduction)

Part 1 - Introduction
Part 2 -Bing Maps Module

UTFGrid is a technology that provides the ability to have client-side interaction over raster map-tiles. Everything is very well explained in this page (which includes a working example), but the idea is simply to have an accompanying JSON file for each map-tile containing meta-information about that image. This info is also very fast to handle in client-side.

For instance, assuming this raster tile which contains Great-Britain:

The accompanying UTFGrid file will have something that looks like an ASCII-Art representation of the image (albeit a little bit stretched):

                                 !!!! !! !!!!!!!!!!!!!!!!!!!          
                                 !!!  !!! !!!!!!!!!!!!!!!!!!!         
                                  !  !!!!! !!!!!!!!!!!!!!!!!!!        
                                     !! !! !!!!!!!!!!!!!!!!!!!        
                            ##       !!    !!!!!!!!!!!!!!!!!!!        
                          #####  !!! !    !!!!!!!!!!!!!!!!!!!!        
                       ########!!!!!      !!!!!!!!!!!!!!!!!!!!        
                      #######!!!!!!!!    !!!!!!!!!!!!!!!!!!!!!!       
                     #######!!!!!!!!!    !!!!!!!!!!!!!!!!!!!!!!       
                      ######!!!!!!!!!!   !!!!!!!!! !!!!!!!!!!!!       
                    ########!!!!!!!!!!   !! !!!!  !!!!!!!!!!!!!!      
                    ######!!!!!!!!!!!!!   !  !    !!!!!!!!!!!!!!      
                      ####!!!!!!!!!!!!!!          !!!!!!!!!!!!!!!!!   
                      ##!!!!!!!!!!!!!!!!          !!!!!!!!!!!!!!!!!!  
              #  #   ####!!!!!#!!!!!!!!!    $$    !!!!!!!!!!!!!!!!!!  
            ##############!!!###!!!!!!!    $$$     !!!!!!!!!!!!!!!!!! 
            ###############!!####!!!!      $$$      !!!!!!!!!!!!!!!!!!
            ########################!      $$       !!!!!!!!!!!!!!!!!!
            ###################### #                 !!!!!!!!!!!!!!!!!
              #####################                  !!!!!!!!!!!!!!!!!
              #####################                  !!!!!!!!!!!!!!!!!
             #######################                 !!!!!!!!!!!!!!!!!
            ########################                !!!!!!!!!!!!!!!!!!
            ########################        !!      !!!!!!!!!!!!!!!!!!
              ######################       !!!!!!!!!!!!!!!!!!!!!!!!!!!
               ### #################        !!!!!!!!!!!!!!!!!!!!!!!!!!
              ## ###################         !!!!!!!!!!!!!!!!!!!!!!!!!
                #####################        !!!!!!!!!!!!!!!!!!!!!!!!!
                ####################       !!!!!!!!!!!!!!!!!!!!!!!!!!!
                ####################       !! !!!!!!!!!!!!!!!!!!!!!!!!
               #####################           !!!!!!!!!!!!!!!!!!!!!!!
             ######################            !!!!!!!!!!!!!!!!!!!!!!!
              #####################            !!!!!!!!!!!!!!!!!!!!!!!
              #####################            !!!!!!!!!!!!!!!!!!!!!!!
           ########################           !!!!!!!!!!!!!!!!!!!!!!!!
          ######################           !!!!!!!!!!!!!!!!!!!!!!!!!!!
            ################             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
           ################             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
           ##############                !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            ###########                 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            ##########                   !!   !!!!!!!!!!!!!!!!!!!!!!!!
              #####                              !!!! !!!!!!!!!!!!!!!!
                                                  !!!!!!!!!!!!!!!!!!!!
                                                !!   !!!!!!!!!!!!!!!!!
                                           !  !!!!!!!!!!!!!!!!!!!!!!!!
                                            ! !!!!!!!!!!!!!!!!!!!!!!!!
                                            !!!!!!!!!!!!!!!!!!!!!!!!!!
                                            !!!!!!!!!!!!!!!!!!!!!!!!!!
                                           !!!!!!!!!!!!!!!!!!!!!!!    
                                          !!!!!!!!!!   !!!!!  !!      
                                         !!!!!!!!!!     !             
                                         !!!!!!!!!!                   
                                       !!!!!   !!!                    
                                      !!!!!                           
                                      ! !!                            
                                  !     !                             
                                                                      
                                                         % && &&      
                                                            &&&&      
                                                       %%  &&&&&      
                                                            &&&&&&&&& 
                                                         ''  &&&&&&&&&
                                                          '  &&&&&&&&&
                                                             &&&&&&&&&

The file also contains a dictionary which maps these characters to the items they represent. Thus, in this particular case the following mappings exist:

• ! maps to United Kingdom
• # to the Republic of Ireland.
• $ to Isle of Man
• % to Guernsey (I actually have a colleague that lives there)
• ' to Jersey
• & to France

Last, but not least, the file also includes a dictionary with some meta-info for those "objects". For instance, as we're talking about countries it could include stuff like "name", "population", "area", "average earnings", etc.

Unfortunately there's no support for UTFGrid in Bing Maps, hence the reason for me writing this particular blog post.

Dynamically creating hexagons from real-world data (Part 3 - Data-Services)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

In this iteration of my experiment I've add a services layer that is able to fetch real world-info in GeoJSON format, instead of having it hard-coded as on the previous post.

Also, I've added a small degree of interaction on the hexagon board creation, as the user is now able to choose where the board will be created. Nothing too fancy though.

As a finishing touch I've also improved the overall appearance of the hexagons. The following image shows the end-result.

Dynamically creating hexagons from real-word data (Part 2 - Railroads and Forests)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

In this post I'm going to add two new concepts to the hexagon representation: railroads and forests. Also, instead of having the data hardcoded, I'm going to load it from GeoJSON files. Eventually I'll change this behavior to have the data being served by a service, but for now static files will do.

Let me use another image of the wargame Memoir'44 as an example of what I'm trying to achieve, which includes railroads and forests.

Obviously I'm not targeting (for now) this degree of presentation. I just want to detect the underlying objects and represent them differently.

Dynamically creating hexagons from real-word data (Part 1 - Roads)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

As I mentioned in one of my previous posts I have some cool ideas that I would really like to try out. One of them involves creating hex-based battlegrounds dynamically around the world.

So, one would pick a region and that particular area would be overlaid with hexagons that represent real world-data like rivers, roads, forests, mountains, etc. All of this on top of Bing Maps.

It's not very easy to explain my goal so let me use the boardgame Memoir'44 as an example. Here's an actual map from that game.

Now suppose all of this was generated dynamically based on a real map, which would contain a road and some forests.

This is not an easy task and I'll break this into pieces (pun intended). On this first post I'll just address road creation.

Bing Maps Module: InfoboxAutoPan

I've just created a very, very simple Bing Maps module. I'll jump ahead to the demo before any explanation.

Demo:  http://psousa.net/demos/bingmaps/infoBoxAutoPanModule/simpleExample.html

Basically there are two maps, both with a pushpin near the edge and a click handler to display an infobox.

After clicking the pushpins this is the end-result:

Converting from Leaflet to Bing Maps

A few months ago I developed a map-based game for a contest. It was a simple multiplayer wargame on which 4 players could try to conquer Europe using a bunch of different units.

For the mapping API I had to decide between Bing Maps and Leaflet. At the time, as I expected to have to use Canvas to draw the hexagons (which Leaflet supports with canvas tile-layers) I chose Leaflet.

I've now decided to pick-up this game and evolve it into something more "tangible". Thus, and although Leaflet has nothing wrong by itself, it's "just" a map api. I need something more complete and so I've decided to port the game to Bing Maps. Afterwards I'll improve it with some really cool ideas that I have. This blog post should be more or less a catalog of the changes I had to make to convert this game from Leaflet to Bing Maps.

Creating a Bing Maps Canvas TileLayer (Part 2 - Bing Maps Module)

Part 1 - Introduction
Part 2 - Bing Maps Module

In my previous post I've developed a working Canvas TileLayer for Bing Maps. Since then I've enhanced that code and packaged it inside a working Bing Maps Module, adding some new features in the process.
This post is going to be very "demo driven", and I'll show how to use the module with all of its variants. All the demos in this post are included with the module package.

Note: If you want to read a little bit more about Bing Maps Modules you should check http://bingmapsv7modules.codeplex.com, which also includes a catalog for several modules out there.

Update 20/04/2013
This module has been added to the bingmapsv7modules project: https://bingmapsv7modules.codeplex.com/wikipage?title=Canvas Tile Layer Module

Creating a Bing Maps Canvas TileLayer (Part 1 - Introduction)

Part 1 - Introduction
Part 2 - Bing Maps Module

I've already done some experiments with Canvas on Bing Maps. You can check the blog post here and the end-result here.

The approach that I've shown in that post uses a single canvas on top of the map and updates it every time the map view changes. This allows for some really funky results, but it's a tad intensive on the CPU and sometimes that just goes to waste if there's no change on the data or its representation.

What I would like is the ability to do something like I've done previously with Leaflet: creating a client-side canvas tile layer. Thus, the tiles would be created dynamically and Bing Maps would treat them as regular images, leveraging all its goodness regarding panning, zooming and such.

Cocos2d-html5 (Part 1 - Introduction)

Hi there,

In this series of posts I'm going to show how to use the html5 version of the popular cocos2d framework, called "cocos2d-html5".

I'll use version 2.1.1, which has now feature parity between most variants, namely: cocos2d, cocos2d-x and cocos2d-html5. In this particular post I'm just going to setup everything in Windows 8 and prepare the placeholder for future posts.

Maps and Boardgames (Part 4 - Creating a simple game)

Part 1 - Introduction
Part 2 - Using server tiles
Part 3 - Client side drawing
Part 4 - Creating a simple game

After several months since part 3 I've decided to pick up my experiments and make a simple game out of them. "Why?" you may ask. Well, mostly because a friend of mine challenged me to, as his company, IBT, is promoting a contest for their core technology: xRTML.  Also, I thought it could be a fun experiment.

xRTML is a technology which allows developers to easily add real-time features to their websites and applications. It works flawlessly and is freaking fast. Hence should be a perfect match for any multiplayer web-based game in real-time, like the one I'm showing here.

So, what's the idea?

I wanted to do something very simple. Basic rules, player opens the browser and immediately starts to play.



URL: http://psousa.net/games/xrtml

Displaying WebGL data on Google Maps

Off-topic:
I haven't really payed too much attention to my blog lately. A mix of several projects have kept me occupied and without much time to write. Anyway, I've got some posts on the "pipeline" and I'll try to make up for the lost time :)


On-topic:
Recently I watched a video from the Google Maps Team (don't remember the link, sorry) were I was blown away by a demo using WebGL on top of a map. Basically it displayed thousands and thousands of points on a map without any performance hit whatsoever. I was very interested in exploring the demo and fortunately the Google lads were nice enough to create a simplified version of it inside the Google Maps Utility Library.

You can view this demo directly from the library repository here.

So, I'll begin by editing this example to show the same information set that I used in a previous blog-post, particularly one about using Canvas to display thousands of points in Bing Maps (link).

Basically it contains all the localities of mainland Portugal (about 140.000 points) and should be a nice test for this technology.

The canvas version fared itself quite nicely, but I expect the webgl to perform much better.

Zoomable image with Leaflet

Last week the European Southern Observatory (ESO) published a 9 giga-pixel image cataloging 84 million stars in the Central parts of the Milky Way. They even provided this image in a zoomable format at http://www.eso.org/public/images/eso1242a/zoomable/ .

It's really cool, but I'm not particularly fond of the Flash Plugin that they've used. It's a little bit slugish, and IMHO something like Google/Bing/Leaflet could provide a much better user experience.

That said, here's what I'll do: I'll get a smaller image from ESO, split it in tile images and display it on Leaflet.

To split the image I'm going to use an utility called GDAL2Tiles. I'll show two variants of this:

• using the command-line tool
• using a Graphical Interface called MapTiler

If you just want to keep things simple jump right ahead to the MapTiler version. I'm also showing the command-line version because it's more productive and suitable to batching scenarios.

NHibernate Spatial (Part 5 - NHibernate 3.3.1+)

Part 1 - Introduction
Part 2 - First Demo
Part 3 - Improving
Part 4 - Querying 
Part 5 - NHibernate 3.3.1+

NHibernate Spatial is sort-of moribund, which is rather unfortunate because it's a really great project with lots of potential. Anyway, not conformed with it I decided to put the sucker onto Github to create an unofficial working version of it.

Fortunately some lads had the same idea avoiding me the extra-work. Unfortunately their projects don't have that much activity and most don't seem to really work. Anyway, in the spirit of GitHub and collaboration I decided to fork one of these to try my luck. Also, if time permits, I'll try to maintain it for future NHibernate releases.

I've forked the following project: https://github.com/suryapratap/Nhibernate.Spatial. The author already set it up nicely, making the following changes to the broken nhcontrib version:

• Fixed compilation errors
• Used NuGet for some references
• Upgraded the projects from VS2008 to VS2010

So, props for suryapratap for providing a much nicer starting point. My own fork is at: https://github.com/pmcxs/Nhibernate.Spatial where I'm going to update this library.

Going the micro-ORM way with Dapper

If you're a developer you've most certainly used stackoverflow (or any other of the stackexchange sites). It's the reference Q&A site out there and has got tons of helpful content and a really enthusiastic community.

One of my favorite things about Stackoverflow is that it's damn fast. The interface, although minimalistic, shows lots of information, refreshes everything on-the-fly and just feels snappy. It's public knowledge that Stackoverflow developers are obsessed with performance, and fortunately they've been more than happy to share their secrets with the community through some blog posts and videos.

To achieve the results you see today they've used many clever tricks and tweaks, but have also developed some in-house tools to overcome the performance limitations of existing technologies. In this post I'm going to look at their own ORM: Dapper, which is widely used in Stackoverflow and has been open-sourced on Github (link).

OpenStreetMaps

I haven't blogged yet specifically about OpenStreetMaps (OSM). On this post I'm going to talk a little bit about OSM and the easiness and importance of adding information.

For those that haven't heard about OSM the simplest explanation is: it's the Wikipedia for maps. A joint collaborative effort of thousands of people to create a free world map.

Recently OSM has gained a lot of traction, mostly because of the transition of some key players from other mapping technologies/data to OSM. Notable examples:

• Foursquare
• Apple
• Wikipedia
• Craigs List

I'm really glad that OSM is finally getting the exposure that it deserves, but I still feel that it's a niche thing. Everyone that likes maps obviously knows it, but it isn't as entrenched on our web culture as something like Wikipedia, and most people don't realize that there's a free mapping wiki where everyone can contribute with unique and valuable information.

Freehand drawing with Cocos2d-x and Box2d

After going on holidays and recharging my batteries I'm ready for some more blogging action.

This will be a fun one. Basically I want to:

• Allow the user to draw a freehand shape on the screen
• After releasing the click/gesture the shape should materialize itself into a dynamic physics body.
• The generated bodies should be affected by gravity and will collide with each other

I'll be using cocos2d-x 2.0.1 with Box2D for the physics.

(Update: 24 March 2013: As some people were having trouble making it work I've replaced the current download with a complete XCode project using the latest Cocos2d-x version (cocos2d-x 2.1.2))

Anyway, a video is worth more than 10.000 words (heck, if a picture is worth 1000 words a video is certainly more). Here's the final effect:

C++ revisited

During my academic and professional life I've been in touch with lots of different programming languages, for example:
Pascal, C, C++, Prolog, Java, Lisp, Javascript, C#, Smalltalk, and many others.

With time I lost contact with most of them, except C#, Java and Javascript.

Recently, as explained in my blog, I began to delve onto mobile game development using C++, mostly because it's a supported language in most mobile operating (iOS, Android and Windows Phone 8), and the "El Dorado" of being able to develop once and run everywhere (sort of).