Game-development Log (14. CDN Improvements)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attack and Explosions
8. Performance Checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistency and Authentication
12. Scalling up the loader process
13. Polishing the experience
14. CDN Improvements

I haven't been really active on the development of this project as I've been occupied with other stuff. Regardless, I'm now focused on returning to active development on it.

Interestingly enough what actually triggered my return was a couple of Azure announcements last week on the CDN front. First, some context:

Although I like Azure quite a lot its CDN offering has been quite lacking to say the least. The community was quite vocal in requesting some essential features but Microsoft neglected to provide any updates or expected delivery dates, as seen here: http://feedback.azure.com/forums/169397-cdn

For example, the top voted feature request was the ability to force content to be refreshed, which is, IMHO, completely essential for a CDN offering:

http://feedback.azure.com/forums/169397-cdn/suggestions/556307-ability-to-force-the-cdn-to-refresh-any-cached-con

The feature was requested 5 years ago, eventually marked as "planned", and no further update was provided by Microsoft, similarly to the other requested features.

Well, all of this until last week, when Microsoft apparently woke up.

First they've provided feedback on most of the feature requests and provided an expectation around release dates. Not ideal (as most of these features are late by a few years) but positive nevertheless.

Then, the icing on the top of the cake was this post:

https://azure.microsoft.com/blog/2015/06/04/announcing-custom-origin-support-for-azure-cdn/

Basically Microsoft shipped three awesome features for the CDN:

I'll just copy&paste from that post:

Custom Origins Supported
Azure CDN can now be used with any origin. Previously, Azure CDN only supported a limited set of Azure Services (i.e. Web Apps, Storage, Cloud Services and Media Services) and you only had the ability to create a CDN endpoint for an Azure Service that was in your Azure Subscription. With this recent update, you can now create a CDN endpoint for any origin you like. This includes the ability to create an origin in your own data center, an origin provided by third party cloud providers, etc. and gives you the flexibility to use any origin you like with Azure CDN!

Multiple CDN Endpoints with the Same Origin
Several of you may have tried to create multiple CDN endpoints for the same origin and found this wasn’t possible due to restrictions. We have now removed the restrictions and you now have the ability to create multiple endpoints for the same origin URL. This provides you a more refined control over content management and can be used to improve performance as multiple host names can be used to access assets from the same origin.

Save Content in any Origin Folder
Previously, when you created a CDN endpoint for cloud services you were required to use “/cdn/” as the default origin path. For example, if the path for your cloud service washttp://strasbourg.cloudapp.net you were required to use http://strasbourg.cloudapp.net/cdn/ as the root path to get content from your origin when you created a CDN endpoint. This restriction has been removed and you can store content in any folder. Using the previous example, you can now use http://strasbourg.cloudapp.net/as the root path to get content from your origin.

These might seem minor changes but let me explain how they positively affect this project:

Game-development Log (8. Performance Checkpoint - I)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attack and Explosions
8. Performance Checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

I don't want to have my full architecture set up only to realise that something isn't really scalable or performs sub-par. Thus, time for a performance checkpoint where I'm reviewing some of my bottlenecks and try to fix them.

Zoomed-out tile images

Currently my image tiles are generated dynamically per request and stored on the Azure CDN. Although this is quite interesting from a storage point-of-view/setup time it has a drawback: the time it takes to generate an image on the furthest away zoom levels, particularly for the unlucky users that get cache misses on the CDN.

The reason is quite-simple: a tile at zoom level 7 includes about 16.000 individual hexagons, requiring some time (like a couple of seconds) to render. For comparison, a tile at zoom level 12 includes about 30 hexagons and is blazing fast to render.

Pre-generating everything is not an option so I opted for an hybrid-approach:

• Between zoom levels 7 and 10 (configurable) I generate the tile images and store them on Azure's blob storage with a CDN fetching from it.
• After 11+ they're generated dynamically and stored on the CDN (as they were)

So in practice I now have two different urls on my map:

• Dynamic url: http://az665126.vo.msecnd.net/?z={z}&x={x}&y={y}&type=jpg
• Static url: http://az679112.vo.msecnd.net/imagetiles/{quadkey}.jpg

So the question is: how many tiles will I need to pre-generate?

The math is quite straight-forward:

tiles for zoom level n = 4^n

tiles 7 = 4^7 = 16.384
tiles 8 = 4^8 = 65.536
tiles 9 = 4^9 = 262.144
tiles 10 = 4^10 = 1.048.576

total = 1.392.640

As I only generate tiles with data, and considering that 2/3 of the planet is water, the approximate number of hexagons generated would be:

approximate number = total * 0.33 ~ 460.000 images

Not too bad and completely viable, particularly as I don't expect to be re-generating these images very often.

Vector-Load to support unit movement

Something that I was noticing is that panning the map view on higher zoom levels was a little bit sluggish, particularly as it was loading the vector tiles with the hexagon data in order to calculate the movement options of the units.

I was loading this info per-tile during panning, which was blocking the UI thread. A colleague of mine suggested me to try something else: HTML5 Web Workers. You basically spawn a worker thread that is able to do some work (albeit with some limitations, like not being able to access the DOM) and using messaging to communicate with the main UI thread.

The implementation was really straightforward. Unfortunately I didn't really notice any performance improvement. Anyway, Web Workers could be an incredibly viable alternative if I ever switch to a client-based drawing logic instead of completely server-side. I've added an entry to my backlog for some WebGL experiments with this :)

I then had a different idea: instead of loading the hexagon-data tile by tile make a single request that would include the various tiles that compose the current viewport. This is triggered on the "viewchangeend" Bing Maps event, particularly using a throttled event handler.

There was a small performance benefit on this approach and it can be further optimised, particularly leveraging local-storage on the client.

Change from PNG to JPEG

At a certain point in time my tiles required transparencies but that's no longer the case. Thus, and although being hit with a small quality downgrade, I've changed my image tiles from PNG to JPEG.

This has 3 advantages:

• Storing the images on the CDN/Blob Storage will be cheaper as the JPEG images are considerably smaller
• Latency on loading the image tiles
• A more subtle advantage happens on the rendering process, as JPEG is faster to load and process, especially when the PNG counterpart has an alpha-channel.

The drawback is, as mentioned, image-quality. Here's a PNG image tile and the corresponding JPEG.

Highly compressed and noticeable image-loss but on the map as a whole is mostly negligible, particularly if there's no reference to compare. I'm more worried with performance than top-notch image quality.

Game-development Log (5. CDN Caching for map tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

The map that was displayed on my previous post was using an Azure Web-Site to render the tiles. On this iteration I've included a CDN so that the tiles are cached and the latency is minimal when serving them.

Generically speaking this is my target architecture including a CDN:

1. The map displayed on the browser requests a tile to the CDN
2. The CDN checks if it contains the image or not
3. If not, it queries the tile-server for that particular image
4. The Tile Server generates it as described on my previous post
5. The CDN stores the image tile 
6. The CDN returns the image to the browser

Game-development Log (3. Loading data to Vector Tiles)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

On this iteration I'm loading some geographical data, converting it to hexagons and exporting vector tiles with the resulting information.

Game-development Log (2. Visual Studio Online)

1. Introduction + WebAPI for map tiles
2. Visual Studio Online
3. Loading Data to Vector Tiles
4. Rendering image tiles
5. CDN Caching for map tiles
6. Adding Units and Basic Interaction
7. Attacks and Explosions
8. Performance checkpoint
9. Bootstraping
10. Real-time with SignalR
11. Persistence and Authentication
12. Scaling up the loader process
13. Polishing the experience

Work-in-progress: https://site-win.azurewebsites.net

IMHO a project needs planning/tracking. I'm all pro-Agile, but using that as an excuse to avoid any analysis, documentation and tracking is a common mistake. Also, it's very hard to deliver working software without a proper release cycle. With that said, one could argue that for a single-person project this isn't by any means a requirement. Regardless, I plan to do it and I'll try to follow a professional approach to building this game (which sometimes isn't particularly easy as I'm doing it on my free time after both my kids have gone to bed :D).

I'm using Visual Studio Online for work-items/release management and source-control, including its link to Azure.