Game-development Log (10. Real-time with SignalR)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance Checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net/ 

On this iteration I'm adding real-time notifications to the unit movement using the (awesome) SignalR library.

The idea for this will be simple: if a player moves a unit on his browser window all the others players will see that unit move in realtime on their browsers.

I've created an ultra-simple video demoing it. I've opened three separate browser windows: one with Chrome, Firefox and Safari. The movement of the units should be synchronised in real-time across the various browsers, regardless of the one that triggered the movement.


So, how was this setup? Easy as pie.

1. Add SignalR NuGet package

install-package Microsoft.AspNet.SignalR

2. Initialise SignalR on the Startup routine of ASP.NET MVC 5 (on previous versions it's slightly different)

public partial class Startup
{
    public void Configuration(IAppBuilder app)
    {
        app.MapSignalR();
    }
}

3. Create a Hub
My hub will be ultra simple. It will simply receive a movement notification and share it with all the other SignalR clients as is.

public class UnitHub : Hub
{
    public void SetPosition(dynamic unitdata)
    {
        Clients.Others.unitUpdated(unitdata);
    }
}

4. Create Javascript code both to send a SignalR message when the player moves a unit or when a notification of another player moving a unit has been received.

//code to receive notification from server
unitUpdated = function (unit) {
 // (...)
};

// Code to submit to server
server.setPosition(unit);

5. (Optional) If you're using Azure, you should enable WebSockets on your website.

And that's it.

Please note that I haven't yet added persistence nor support for different users. Thus, everyone will be moving the same units. Anyway, I don't have that many page views on my blog for it to be a problem :)

So, what's next?
- Adding a database to persist the units position
- Adding users, each with his own units.
- Authentication with Facebook, Twitter, Google, Windows, Email

Game-development Log (9. "Bootstraping")

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attack and Explosions
8. Performance Checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

This will be a short one. I'm basically picking up my map page and adding a bootstrap template around it.

I'm not tweaking it too much, just adding a couple of navigation buttons, some modals and some cosmetic details (like a subtle transparency on the navigation bar).

Without Bootstrap:

With Bootstrap:

Some buttons on the navbar already open some modals but it's mostly a placeholder for the functionality, like the Sign in one.

Also, Boostrap provides, out-of-the-box, responsive design. Thus the mobile-like display looks like:

So, nothing too fancy but I really love the "cleaness" that Bootstrap provides. I still need to tweak the UI a little bit but I'm not expecting it to change that much.

Game-development Log (8. Performance Checkpoint - I)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attack and Explosions
8. Performance Checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

I don't want to have my full architecture set up only to realise that something isn't really scalable or performs sub-par. Thus, time for a performance checkpoint where I'm reviewing some of my bottlenecks and try to fix them.

Zoomed-out tile images

Currently my image tiles are generated dynamically per request and stored on the Azure CDN. Although this is quite interesting from a storage point-of-view/setup time it has a drawback: the time it takes to generate an image on the furthest away zoom levels, particularly for the unlucky users that get cache misses on the CDN.

The reason is quite-simple: a tile at zoom level 7 includes about 16.000 individual hexagons, requiring some time (like a couple of seconds) to render. For comparison, a tile at zoom level 12 includes about 30 hexagons and is blazing fast to render.

Pre-generating everything is not an option so I opted for an hybrid-approach:

• Between zoom levels 7 and 10 (configurable) I generate the tile images and store them on Azure's blob storage with a CDN fetching from it.
• After 11+ they're generated dynamically and stored on the CDN (as they were)

So in practice I now have two different urls on my map:

• Dynamic url: http://az665126.vo.msecnd.net/?z={z}&x={x}&y={y}&type=jpg
• Static url: http://az679112.vo.msecnd.net/imagetiles/{quadkey}.jpg

So the question is: how many tiles will I need to pre-generate?

The math is quite straight-forward:

tiles for zoom level n = 4^n

tiles 7 = 4^7 = 16.384
tiles 8 = 4^8 = 65.536
tiles 9 = 4^9 = 262.144
tiles 10 = 4^10 = 1.048.576

total = 1.392.640

As I only generate tiles with data, and considering that 2/3 of the planet is water, the approximate number of hexagons generated would be:

approximate number = total * 0.33 ~ 460.000 images

Not too bad and completely viable, particularly as I don't expect to be re-generating these images very often.

Vector-Load to support unit movement

Something that I was noticing is that panning the map view on higher zoom levels was a little bit sluggish, particularly as it was loading the vector tiles with the hexagon data in order to calculate the movement options of the units.

I was loading this info per-tile during panning, which was blocking the UI thread. A colleague of mine suggested me to try something else: HTML5 Web Workers. You basically spawn a worker thread that is able to do some work (albeit with some limitations, like not being able to access the DOM) and using messaging to communicate with the main UI thread.

The implementation was really straightforward. Unfortunately I didn't really notice any performance improvement. Anyway, Web Workers could be an incredibly viable alternative if I ever switch to a client-based drawing logic instead of completely server-side. I've added an entry to my backlog for some WebGL experiments with this :)

I then had a different idea: instead of loading the hexagon-data tile by tile make a single request that would include the various tiles that compose the current viewport. This is triggered on the "viewchangeend" Bing Maps event, particularly using a throttled event handler.

There was a small performance benefit on this approach and it can be further optimised, particularly leveraging local-storage on the client.

Change from PNG to JPEG

At a certain point in time my tiles required transparencies but that's no longer the case. Thus, and although being hit with a small quality downgrade, I've changed my image tiles from PNG to JPEG.

This has 3 advantages:

• Storing the images on the CDN/Blob Storage will be cheaper as the JPEG images are considerably smaller
• Latency on loading the image tiles
• A more subtle advantage happens on the rendering process, as JPEG is faster to load and process, especially when the PNG counterpart has an alpha-channel.

The drawback is, as mentioned, image-quality. Here's a PNG image tile and the corresponding JPEG.

Highly compressed and noticeable image-loss but on the map as a whole is mostly negligible, particularly if there's no reference to compare. I'm more worried with performance than top-notch image quality.

Game-development Log (7. Attack and explosions)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attack and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

On my last post I've added interactivity to the units. On this post I'm taking it further by adding an attack option and corresponding explosions :)

First of all I've included an extra tank on the map that's not controllable by the player. Hence, a nice target to be attacked (although non-destructible).

The attack is triggered in the same way as a regular movement, by dragging the unit. When on-top of another unit it changes to an attack cursor, showing additional info.

I've implemented the explosion using the particle lib at http://scurker.com/projects/particles/

To place the explosion on Bing Maps I've used the pushpin's "htmlContent" property to create a new canvas where the explosion is rendered.

I've also used a custom html pushpin to display the attack details. For now only the range isn't hardcoded.

And that's it for now.

Game-development Log (6. Adding units and basic interaction)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

Time to make things more interesting by adding some interactive units to the map. Pretty beefy update that, although highly hardcoded on some places, should lay the foundation for what's to come.

Here's a small video showcasing what's currently deployed on Azure.

Game-development Log (5. CDN Caching for map tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

The map that was displayed on my previous post was using an Azure Web-Site to render the tiles. On this iteration I've included a CDN so that the tiles are cached and the latency is minimal when serving them.

Generically speaking this is my target architecture including a CDN:

1. The map displayed on the browser requests a tile to the CDN
2. The CDN checks if it contains the image or not
3. If not, it queries the tile-server for that particular image
4. The Tile Server generates it as described on my previous post
5. The CDN stores the image tile 
6. The CDN returns the image to the browser

Game-development Log (4. Rendering image tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

New iteration, tons of new stuff:

• Create a mixed-approach for the problem of having data pre-generated vs generated on-the-fly
• Binary Vector Tiles
• Render image tiles
• Blending with Bing Maps
• Improved loader process to fix/optimize roads and railways

Game-development Log (3. Loading data to Vector Tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

On this iteration I'm loading some geographical data, converting it to hexagons and exporting vector tiles with the resulting information.

Game-development Log (2. Visual Studio Online)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading Data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

IMHO a project needs planning/tracking. I'm all pro-Agile, but using that as an excuse to avoid any analysis, documentation and tracking is a common mistake. Also, it's very hard to deliver working software without a proper release cycle. With that said, one could argue that for a single-person project this isn't by any means a requirement. Regardless, I plan to do it and I'll try to follow a professional approach to building this game (which sometimes isn't particularly easy as I'm doing it on my free time after both my kids have gone to bed :D).

I'm using Visual Studio Online for work-items/release management and source-control, including its link to Azure.

Game-development Log (1. Introduction + WebAPI for map tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading Data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

Although my blog has been mostly dormant for the last few months I've kept myself busy making various experiments with mapping and gaming. I've expanded some of the ideas that I've already posted and I've prototyped some new stuff.

My intent: to create a massive online strategy game where there's a single shared persistent battleground and a pseudo-realtime gameplay mechanic. Yeah, lots of fancy words without saying much, but if it all goes as planned could be quite interesting. In a worst case scenario should be fun to build.

My plan is to document the progress on my blog (particularly as what I have now are mostly disconnected pieces) and deploy the new iterations as frequently as possible.

Generating Server-side tile maps with Node.js (Part 2 - Using Mapnik)

Part 1 - Programatically with Canvas
Part 2 - Using Mapnik

On my previous post I've shown how to use node.js to generate server-tiles, painting them "manually" using the canvas API. This is perfect for those scenarios where one wants to overlay dynamic or custom data on top of an existing map. The following images (although done on client-side with canvas) are good examples of this:

Now, suppose we don't want to generate data on top of a map but the map itself. Meaning, a tile more or less similar to this:

This is very complex as it requires loading spatial data, painting the countries, roads, rivers, labels, always taking into account the zoom level, etc. Not trivial at all.

Generating server-side tile-maps with Node.JS (Part 1 - Programatically with Canvas)

Part 1 - Programatically with Canvas
Part 2 - Using Mapnik

This will be a series of posts on leveraging Node.js to generate tile-images on server-side that will be displayed on top of Bing Maps (although applicable to any other mapping API).

For those not familiar with it, Node has gained lots of momentum on the web. It provides a simple event oriented, non-blocking architecture on which one can develop fast and scalable applications using Javascript.

It lends itself really well for generating tile-images and is in-fact being used by some large players like Mapbox.

On this first post I'm going to generate very simple tiles. I won't fetch data from the database nor do anything too fancy. Each tile will simply consist on a 256x256 png with a border and a label identifying the tile's z/x/y. Conceptually something like this:

I'm going to use an amazing Node module called Canvas which provides the whole HTML5 Canvas API on server-side. My main reason is that it's an API that I know particularly well and I can reuse lots of my existing code.

I'm going to develop on Mac OS X but most of this would be exactly the same on Windows or Linux.

Afterwards, as an extra, I'm also going to provide instructions on how to setup a Linux Virtual Machine in Azure running the resulting node application.

UTFGrid in Bing Maps (Part 2 - Bing Maps Module)

Part 1 - Introduction
Part 2 -Bing Maps Module

On this post I'm adapting my previous UTFGrid code and packaging it into a very easy to use Bing Maps Module.

I'm going to show a couple of simple demos on how to use it.

Demo #1 - Basic (try it here)

Very simple demo. Click a country and an alert message will display its name. All this without any round-trip to the server.

Note: in all these demos there are only 4 zoom levels of UTFGrid tiles. Thus, if zooming paste the satellite images, nothing should happen when clicking/hovering the map.

UTFGrid in Bing Maps (Part 1 - Introduction)

Part 1 - Introduction
Part 2 -Bing Maps Module

UTFGrid is a technology that provides the ability to have client-side interaction over raster map-tiles. Everything is very well explained in this page (which includes a working example), but the idea is simply to have an accompanying JSON file for each map-tile containing meta-information about that image. This info is also very fast to handle in client-side.

For instance, assuming this raster tile which contains Great-Britain:

The accompanying UTFGrid file will have something that looks like an ASCII-Art representation of the image (albeit a little bit stretched):

                                 !!!! !! !!!!!!!!!!!!!!!!!!!          
                                 !!!  !!! !!!!!!!!!!!!!!!!!!!         
                                  !  !!!!! !!!!!!!!!!!!!!!!!!!        
                                     !! !! !!!!!!!!!!!!!!!!!!!        
                            ##       !!    !!!!!!!!!!!!!!!!!!!        
                          #####  !!! !    !!!!!!!!!!!!!!!!!!!!        
                       ########!!!!!      !!!!!!!!!!!!!!!!!!!!        
                      #######!!!!!!!!    !!!!!!!!!!!!!!!!!!!!!!       
                     #######!!!!!!!!!    !!!!!!!!!!!!!!!!!!!!!!       
                      ######!!!!!!!!!!   !!!!!!!!! !!!!!!!!!!!!       
                    ########!!!!!!!!!!   !! !!!!  !!!!!!!!!!!!!!      
                    ######!!!!!!!!!!!!!   !  !    !!!!!!!!!!!!!!      
                      ####!!!!!!!!!!!!!!          !!!!!!!!!!!!!!!!!   
                      ##!!!!!!!!!!!!!!!!          !!!!!!!!!!!!!!!!!!  
              #  #   ####!!!!!#!!!!!!!!!    $$    !!!!!!!!!!!!!!!!!!  
            ##############!!!###!!!!!!!    $$$     !!!!!!!!!!!!!!!!!! 
            ###############!!####!!!!      $$$      !!!!!!!!!!!!!!!!!!
            ########################!      $$       !!!!!!!!!!!!!!!!!!
            ###################### #                 !!!!!!!!!!!!!!!!!
              #####################                  !!!!!!!!!!!!!!!!!
              #####################                  !!!!!!!!!!!!!!!!!
             #######################                 !!!!!!!!!!!!!!!!!
            ########################                !!!!!!!!!!!!!!!!!!
            ########################        !!      !!!!!!!!!!!!!!!!!!
              ######################       !!!!!!!!!!!!!!!!!!!!!!!!!!!
               ### #################        !!!!!!!!!!!!!!!!!!!!!!!!!!
              ## ###################         !!!!!!!!!!!!!!!!!!!!!!!!!
                #####################        !!!!!!!!!!!!!!!!!!!!!!!!!
                ####################       !!!!!!!!!!!!!!!!!!!!!!!!!!!
                ####################       !! !!!!!!!!!!!!!!!!!!!!!!!!
               #####################           !!!!!!!!!!!!!!!!!!!!!!!
             ######################            !!!!!!!!!!!!!!!!!!!!!!!
              #####################            !!!!!!!!!!!!!!!!!!!!!!!
              #####################            !!!!!!!!!!!!!!!!!!!!!!!
           ########################           !!!!!!!!!!!!!!!!!!!!!!!!
          ######################           !!!!!!!!!!!!!!!!!!!!!!!!!!!
            ################             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
           ################             !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
           ##############                !!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            ###########                 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
            ##########                   !!   !!!!!!!!!!!!!!!!!!!!!!!!
              #####                              !!!! !!!!!!!!!!!!!!!!
                                                  !!!!!!!!!!!!!!!!!!!!
                                                !!   !!!!!!!!!!!!!!!!!
                                           !  !!!!!!!!!!!!!!!!!!!!!!!!
                                            ! !!!!!!!!!!!!!!!!!!!!!!!!
                                            !!!!!!!!!!!!!!!!!!!!!!!!!!
                                            !!!!!!!!!!!!!!!!!!!!!!!!!!
                                           !!!!!!!!!!!!!!!!!!!!!!!    
                                          !!!!!!!!!!   !!!!!  !!      
                                         !!!!!!!!!!     !             
                                         !!!!!!!!!!                   
                                       !!!!!   !!!                    
                                      !!!!!                           
                                      ! !!                            
                                  !     !                             
                                                                      
                                                         % && &&      
                                                            &&&&      
                                                       %%  &&&&&      
                                                            &&&&&&&&& 
                                                         ''  &&&&&&&&&
                                                          '  &&&&&&&&&
                                                             &&&&&&&&&

The file also contains a dictionary which maps these characters to the items they represent. Thus, in this particular case the following mappings exist:

• ! maps to United Kingdom
• # to the Republic of Ireland.
• $ to Isle of Man
• % to Guernsey (I actually have a colleague that lives there)
• ' to Jersey
• & to France

Last, but not least, the file also includes a dictionary with some meta-info for those "objects". For instance, as we're talking about countries it could include stuff like "name", "population", "area", "average earnings", etc.

Unfortunately there's no support for UTFGrid in Bing Maps, hence the reason for me writing this particular blog post.

Dynamically creating hexagons from real-world data (Part 3 - Data-Services)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

In this iteration of my experiment I've add a services layer that is able to fetch real world-info in GeoJSON format, instead of having it hard-coded as on the previous post.

Also, I've added a small degree of interaction on the hexagon board creation, as the user is now able to choose where the board will be created. Nothing too fancy though.

As a finishing touch I've also improved the overall appearance of the hexagons. The following image shows the end-result.

Dynamically creating hexagons from real-word data (Part 2 - Railroads and Forests)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

In this post I'm going to add two new concepts to the hexagon representation: railroads and forests. Also, instead of having the data hardcoded, I'm going to load it from GeoJSON files. Eventually I'll change this behavior to have the data being served by a service, but for now static files will do.

Let me use another image of the wargame Memoir'44 as an example of what I'm trying to achieve, which includes railroads and forests.

Obviously I'm not targeting (for now) this degree of presentation. I just want to detect the underlying objects and represent them differently.

Dynamically creating hexagons from real-word data (Part 1 - Roads)

Part 1 - Roads
Part 2 - Railroads and Forests
Part 3 - Data-Services

As I mentioned in one of my previous posts I have some cool ideas that I would really like to try out. One of them involves creating hex-based battlegrounds dynamically around the world.

So, one would pick a region and that particular area would be overlaid with hexagons that represent real world-data like rivers, roads, forests, mountains, etc. All of this on top of Bing Maps.

It's not very easy to explain my goal so let me use the boardgame Memoir'44 as an example. Here's an actual map from that game.

Now suppose all of this was generated dynamically based on a real map, which would contain a road and some forests.

This is not an easy task and I'll break this into pieces (pun intended). On this first post I'll just address road creation.

Bing Maps Module: InfoboxAutoPan

I've just created a very, very simple Bing Maps module. I'll jump ahead to the demo before any explanation.

Demo:  http://psousa.net/demos/bingmaps/infoBoxAutoPanModule/simpleExample.html

Basically there are two maps, both with a pushpin near the edge and a click handler to display an infobox.

After clicking the pushpins this is the end-result:

Converting from Leaflet to Bing Maps

A few months ago I developed a map-based game for a contest. It was a simple multiplayer wargame on which 4 players could try to conquer Europe using a bunch of different units.

For the mapping API I had to decide between Bing Maps and Leaflet. At the time, as I expected to have to use Canvas to draw the hexagons (which Leaflet supports with canvas tile-layers) I chose Leaflet.

I've now decided to pick-up this game and evolve it into something more "tangible". Thus, and although Leaflet has nothing wrong by itself, it's "just" a map api. I need something more complete and so I've decided to port the game to Bing Maps. Afterwards I'll improve it with some really cool ideas that I have. This blog post should be more or less a catalog of the changes I had to make to convert this game from Leaflet to Bing Maps.

Creating a Bing Maps Canvas TileLayer (Part 2 - Bing Maps Module)

Part 1 - Introduction
Part 2 - Bing Maps Module

In my previous post I've developed a working Canvas TileLayer for Bing Maps. Since then I've enhanced that code and packaged it inside a working Bing Maps Module, adding some new features in the process.
This post is going to be very "demo driven", and I'll show how to use the module with all of its variants. All the demos in this post are included with the module package.

Note: If you want to read a little bit more about Bing Maps Modules you should check http://bingmapsv7modules.codeplex.com, which also includes a catalog for several modules out there.

Update 20/04/2013
This module has been added to the bingmapsv7modules project: https://bingmapsv7modules.codeplex.com/wikipage?title=Canvas Tile Layer Module