tag:blogger.com,1999:blog-60296294945086265912024-03-19T07:16:08.104+00:00Pedro's Tech MumblingsMy thoughts on .NET, GIS, Javascript, Databases, and technology in generalPedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.comBlogger83125tag:blogger.com,1999:blog-6029629494508626591.post-65518854584504817392019-06-12T21:33:00.000+01:002019-06-15T16:23:02.620+01:00Drawing Lines in .NET Core: Comparing ImageSharp and System.Drawing.CommonThis is a follow-up post from an early experiment I did almost 3 years ago: <a href="https://build-failed.blogspot.com/2016/08/creating-simple-tileserver-with-net.html">https://build-failed.blogspot.com/2016/08/creating-simple-tileserver-with-net.html</a><br />
<br />
At the time, I was trying to setup a simple Tile Server in C# that ran in .NET Core in Win, Mac and Linux, generating images on-the-fly. As I didn't find any C# lib that supported drawing lines and such, I built it myself (see above blog-post for details) over a lib called "ImageProcessor", which allowed me to set individual pixels.<br />
<br />
Now, as I'm doing another project that also requires drawing stuff I decided to see how much the landscape has evolved over the last couple of years.<br />
<br />
Spoiler alert, it changed significantly. As a summary:<br />
<ul>
<li><a href="https://github.com/JimBobSquarePants/ImageProcessor">ImageProcessor</a> is now marked as "in soft archive mode", with focus shifting to another library called <a href="https://github.com/SixLabors/ImageSharp">ImageSharp</a>, from the same author</li>
</ul>
<ul>
<li>ImageSharp now supports Drawing functionality and actually seems very feature rich on that regard. Also, it's fully managed, not relying on native OS calls. If any of you worked with GDI+ in the past, you'll probably remember the mem leaks and thread-"unsafety" of it, particularly if trying to use it on the web.</li>
</ul>
<ul>
<li>Meanwhile, Microsoft released a Nuget package called "Microsoft.Drawing.Common", which is part of the Windows Compatibility Pack, aiming to help developers migrate their existing .NET code to .NET Core</li>
</ul>
<div>
<ul>
<li>As opposed to ImageSharp, Microsoft.Drawing.Common acts a a bridge to OS specific logic. In Windows it relies on GDI+, on Linux requires installing <a href="https://www.mono-project.com/docs/gui/libgdiplus/">libgdiplus</a> (from Mono).</li>
</ul>
</div>
I'll quote Scott Hanselman on this (<a href="https://www.hanselman.com/blog/HowDoYouUseSystemDrawingInNETCore.aspx">https://www.hanselman.com/blog/HowDoYouUseSystemDrawingInNETCore.aspx</a>):<br />
<blockquote class="tr_bq">
<span style="background-color: white; color: #333333; font-family: "open sans" , "helvetica neue" , "helvetica" , "arial" , sans-serif; font-size: 15.2px;"><i>There's lots of great options for image processing on .NET Core now! It's important to understand that this System.Drawing layer is great for existing System.Drawing code, but you probably shouldn't write NEW image management code with it. Instead, consider one of the great other open source options.</i></span></blockquote>
With that said, I'm very interested to understand how both libs compare, thus I've setup a very, very simple test. I'm not going to test the functional differences between both libs and <u>I'm going to focus on the use-case that's most relevant to me: drawing lines.</u><br />
<br />
All the relevant code is available on the Gitrepo: <a href="https://github.com/pmcxs/core-linedrawing-benchmark/">https://github.com/pmcxs/core-linedrawing-benchmark/</a><br />
<b><br /></b>
The following options can parametrized: number of lines being generated, image size and line width.<br />
<b><br /></b>
Ok, let's start:<br />
<b><br /></b>
<b>Basic Functionality</b><br />
<div>
<br />
Starting with a simple test, just to compare (visually) how both results fare:</div>
<div>
<br /></div>
<div>
Number of Lines: 10</div>
<div>
Image Size: 800 px</div>
<div>
Line Width: 10 px<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPjhBcV57XKJB-8g8RV025mZOhbv9Q5izw7nMP_3Yt4vczDqykHfr8kg_OuWmXuPtaeQvfqDZdHeGvK7Ie5n497Q4cWZ_oy_cq_WDKyCf50nk57DDWwzQeMKwIIM00djHiYG6y9gMMysI/s1600/test-comparison.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPjhBcV57XKJB-8g8RV025mZOhbv9Q5izw7nMP_3Yt4vczDqykHfr8kg_OuWmXuPtaeQvfqDZdHeGvK7Ie5n497Q4cWZ_oy_cq_WDKyCf50nk57DDWwzQeMKwIIM00djHiYG6y9gMMysI/s640/test-comparison.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">(default zoom)</td></tr>
</tbody></table>
First the good news: both work :)</div>
<div>
<br /></div>
<div>
I did find a small difference on the output though. Although the images are very similar, both libraries handle corners differently (at least on the default behavior).</div>
<div>
<br /></div>
<div>
System.Drawing creates a strange protrusion on sharp edges. Zooming in on the previous image:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeogCO2NzvNHXTlO0ahPPuAUrW-SE-eHb81y6aijw7AnSkUDNkrNZ9VTkbBMwDG0Fv1H6Hxn7wCGM75YdaH5ItnO15GhnSpXBEa1qTC4_lohtTR7aKsksC5KhjNIFFSZmnmJG8c-Iv9NE/s1600/test-imageSharp-zoom.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgeogCO2NzvNHXTlO0ahPPuAUrW-SE-eHb81y6aijw7AnSkUDNkrNZ9VTkbBMwDG0Fv1H6Hxn7wCGM75YdaH5ItnO15GhnSpXBEa1qTC4_lohtTR7aKsksC5KhjNIFFSZmnmJG8c-Iv9NE/s640/test-imageSharp-zoom.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">(zoomed in)</td></tr>
</tbody></table>
That seems really strange. Increasing the line thickness the effect looks even worse.<br />
<br />
<div>
Number of Lines: 10</div>
<div>
Image Size: 800 px</div>
<div>
Line Width: <b>50 px</b><br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQj3qAwtoBiFttT19iRyJM6OkE9QG6Uixnksv8k6K013c8jcRhIhX9ZZRLbVQSWTcJTTLGZYH8fE2T9ZImlHt_q9Nl8mneURR3aQx5jpagc1rOylBMFl51DMpZr97vkFnhUFOfAygY5Wk/s1600/test-comparison-width.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="800" data-original-width="1600" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQj3qAwtoBiFttT19iRyJM6OkE9QG6Uixnksv8k6K013c8jcRhIhX9ZZRLbVQSWTcJTTLGZYH8fE2T9ZImlHt_q9Nl8mneURR3aQx5jpagc1rOylBMFl51DMpZr97vkFnhUFOfAygY5Wk/s640/test-comparison-width.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">(default zoom)</td></tr>
</tbody></table>
Yeah, even on the default zoom it looks awful with System.Drawing. Basically all corners converge to a single pixel. Thus, the larger the line, the worse the effect gets.<br />
<b><br /></b>
<b>Performance</b><br />
<b><br /></b>
The code I've placed above already outputs the duration of both approaches. Unfortunately (although probably expected) System.Drawing is still faster than ImageSharp. Versions being tested:<br />
<ul>
<li>SixLabors.ImageSharp.Drawing: 1.0.0-dev002737</li>
<li>System.Drawing.Common: 4.6.0-preview5.19224.8</li>
</ul>
Before starting, I actually found this page: <a href="https://docs.sixlabors.com/articles/ImageSharp/GettingStarted.html">https://docs.sixlabors.com/articles/ImageSharp/GettingStarted.html</a> and it mentions a couple of potential issues that might cause performance problems:<br />
<blockquote class="tr_bq" style="background-color: white; box-sizing: border-box; color: #333333; font-size: 16px; margin-bottom: 10px;">
<i>A few troubleshooting steps to try:</i><br />
<ul style="box-sizing: border-box; margin-bottom: 10px; margin-top: 0px;">
<li style="box-sizing: border-box;"><i>Check the value of <a href="https://docs.microsoft.com/en-us/dotnet/api/system.numerics.vector.ishardwareaccelerated?view=netcore-2.1&viewFallbackFrom=netstandard-2.0#System_Numerics_Vector_IsHardwareAccelerated" style="background-color: transparent; box-sizing: border-box; color: #337ab7; cursor: pointer;">Vector.IsHardwareAccelerated</a>. If the output is false, it means there is no SIMD support in your runtime!</i></li>
</ul>
<ul style="box-sizing: border-box; margin-bottom: 10px; margin-top: 0px;">
<li style="box-sizing: border-box;"><i>Make sure your code runs on 64bit! Older .NET Framework versions are using the legacy runtime on 32 bits, having no built-in SIMD support.</i></li>
</ul>
</blockquote>
<br />
I can confirm my test isn't affect by those problems: That flag returns <b>true</b> and I'm running in<b> 64 bits.</b><br />
<br />
I've varied the number of lines (keeping the line width to 10px and image size to 800x800) and the results are as follows:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjqtTcaOBeC3DAIKhpDKWZJfSwRrz2AZp8fYSoy_wEHVvd1PKRyPa0A0u8ONlZScw5wEQXgs7DTKlOo5AOJjVMVDj6YP8v4B2xmR6RNWN2VR7C7KBGha3EeT77Gi-Z1rK-CeGZ_D0KDZ0/s1600/speed-comparison-drawing.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="577" data-original-width="962" height="383" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjjqtTcaOBeC3DAIKhpDKWZJfSwRrz2AZp8fYSoy_wEHVvd1PKRyPa0A0u8ONlZScw5wEQXgs7DTKlOo5AOJjVMVDj6YP8v4B2xmR6RNWN2VR7C7KBGha3EeT77Gi-Z1rK-CeGZ_D0KDZ0/s640/speed-comparison-drawing.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">line width: 10px</td></tr>
</tbody></table>
I was slightly surprised by the jump from 10 to 100 lines in ImageSharp, particularly as from 100 lines to 1000 its performance was almost the same.<br />
<br />
I then did the same test, but increasing the line width from 10px to 100px:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip_R0VwFLUJ_PBx3Mg9VuC5D7gIk9YC5TxkuaVrJeci0qTcNaXzBum-vI-N06DNziXnwCGfIJITQAo2qQEJmRjMEYwPR9yaWa20i3acxrVlkATZdASIIydU1EirKU7iPhwuYSoJuzKw78/s1600/speed-comparison-drawing-100.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="582" data-original-width="964" height="386" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip_R0VwFLUJ_PBx3Mg9VuC5D7gIk9YC5TxkuaVrJeci0qTcNaXzBum-vI-N06DNziXnwCGfIJITQAo2qQEJmRjMEYwPR9yaWa20i3acxrVlkATZdASIIydU1EirKU7iPhwuYSoJuzKw78/s640/speed-comparison-drawing-100.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">line width: 100px</td></tr>
</tbody></table>
I don't understand how ImageSharp is faster when drawing 1000 lines vs the previous test with just 100 lines:<br />
<ul>
<li>100 lines (10 pixel width): 1315 ms</li>
<li>1000 lines (100 pixel width): 776 ms</li>
</ul>
<div>
The ImageSharp performance seems to get worse the thinner the lines are, which I can try to confirm that by going to an extreme of 1px lines.<br />
<br />
<b>But, lets make things even more interesting</b>. I'll include my own custom implementation from 3 years ago to the mix. It didn't compile at first, but was easy enough to update, including using the new <a href="https://docs.sixlabors.com/articles/ImageSharp/WorkingWithPixelBuffers.html">recommendation</a> for a Span class to set pixels.<br />
<br />
The results were interesting. My custom implementation is at its peak the thinnest the line is, hence it gets some really, really strong results. I'm tempted to say there's a bug or an unintended side-effect from the current implementation in ImageSharp. I'll do some additional experiments before submitting a issue.<br />
<br />
<u>I had to change the scale to be logarithmic</u>, as otherwise the results would be hidden because of the strange behavior in ImageSharp, as it takes over 2 minutes to render 1000 lines.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLnF9cux2tWP_ghL2taOBazRdeFZ3dQtqlgOK58KrZkiHjsNqbcc2V9oWfuu9vPcbSiT-qG6-vlvyUaZBtdQ7G1nQAszrBEqygAHb6oZMpMbcnMvrORScWimgHeYubw_3ss8GVUrBAWnI/s1600/speed-comparison-drawing-1.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="580" data-original-width="966" height="384" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLnF9cux2tWP_ghL2taOBazRdeFZ3dQtqlgOK58KrZkiHjsNqbcc2V9oWfuu9vPcbSiT-qG6-vlvyUaZBtdQ7G1nQAszrBEqygAHb6oZMpMbcnMvrORScWimgHeYubw_3ss8GVUrBAWnI/s640/speed-comparison-drawing-1.PNG" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">line width: 1px</td></tr>
</tbody></table>
</div>
</div>
</div>
For actual numbers, with 1000 lines of 1px (from slowest to fastest):<br />
<ul>
<li>ImageSharp (Linux): 140000 ms</li>
<li>ImageSharp (Windows): 127000 ms</li>
<li>System.Drawing (Linux): 132 ms</li>
<li>Custom (Linux): 74 ms</li>
<li>System.Drawing (Windows): 64 ms</li>
<li>Custom (Windows): 62 ms</li>
</ul>
<div>
Increasing the width to 10px the results aren't as strong, but still quite good:</div>
<div>
<ul>
<li>ImageSharp (Linux): 1530 ms</li>
<li>ImageSharp (Windows): 1311 ms</li>
<li>Custom (Linux): 146 ms</li>
<li>Custom (Windows): 127 ms</li>
<li>System.Drawing (Linux): 123 ms</li>
<li>System.Drawing (Windows): 122 ms</li>
</ul>
</div>
<div>
<b>There's a catch though:</b><br />
<ul>
<li>This is literally the only use case I've built. No polygons, beziers, text, etc</li>
<li>Also, my "corner handling" logic is really crappy</li>
</ul>
<div>
Regardless, there could be some potential there, thus I might revisit this topic later on. I've included my custom logic on the repo I've mentioned above: <a href="https://github.com/pmcxs/core-linedrawing-benchmark/tree/master/src/ImageSharpCustomDrawing">https://github.com/pmcxs/core-linedrawing-benchmark/tree/master/src/ImageSharpCustomDrawing</a></div>
</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com13tag:blogger.com,1999:blog-6029629494508626591.post-71416603053826143262019-06-02T09:46:00.000+01:002019-06-18T21:06:10.199+01:00Building a lightweight api (ASP.NET Core 2.0) to find the geo region that contains a given CoordinateContext: Given a set of geographic areas I needed an API that would, for a supplied coordinate, return the area that contained it.<br />
<br />
So, it needed to:<br />
<div>
<br /></div>
<div>
<div>
- load a set of features from a Shapefile</div>
<div>
- provide a Web API that receives as input a latitude and longitude</div>
<div>
- return the feature that contains that point<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8ZFBRS0mK6T6DxG40bOiShPAWrp4oMbOUEK28yFcsyO-EvbKQ0eVxL98RvuxI-IC3Wnmk8PxOC1omF14EaYTx68oIVWvf-9MEmCZpRY0JJp4eejxWORUta7PDThKv_fXzzn9phw8DfAI/s1600/match.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1140" data-original-width="1254" height="581" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8ZFBRS0mK6T6DxG40bOiShPAWrp4oMbOUEK28yFcsyO-EvbKQ0eVxL98RvuxI-IC3Wnmk8PxOC1omF14EaYTx68oIVWvf-9MEmCZpRY0JJp4eejxWORUta7PDThKv_fXzzn9phw8DfAI/s640/match.jpg" width="640" /></a></div>
<br />
<br /></div>
<div>
<div>
Conceptually this is pretty standard, and I expected to have multiple implementations in c# readily available on the web. Interestingly enough, that's not the case (or I probably didn't search properly).</div>
<div>
<br />
I've decided to build a simple service to address the requirements stated above. It's very simple and available at: <a href="https://github.com/pmcxs/region-locator">https://github.com/pmcxs/region-locator</a>. Its README provides some info on how to setup and use.</div>
<div>
<br /></div>
<div>
How does it work? Breakdown below:</div>
<div>
<a name='more'></a><br />
<u>Load a set of features from a Shapefile</u></div>
<div>
<br />
To load the Shapefile I've used NetTopologySuite, particularly the NetTopologySuite.IO.Shapefile Package.<br />
<br />
I've been using NetTopologySuite for a long time and it works really well. Also, has been fully ported to .NET Standard and is also usable in Mac/Linux. The following code will read all features, including both the metadata properties as well as the corresponding geometries.<br />
<pre class="brush:csharp">FeatureCollection features = new FeatureCollection();
var geometryFactory = new GeometryFactory();
using (var shapeFileDataReader = new ShapefileDataReader(shpFilename, geometryFactory))
{
DbaseFileHeader header = shapeFileDataReader.DbaseHeader;
while (shapeFileDataReader.Read())
{
Feature feature = new Feature();
AttributesTable attributesTable = new AttributesTable();
string[] keys = new string[header.NumFields];
IGeometry geometry = (Geometry)shapeFileDataReader.Geometry;
for (int i = 0; i < header.NumFields; i++)
{
DbaseFieldDescriptor fldDescriptor = header.Fields[i];
keys[i] = fldDescriptor.Name;
attributesTable.Add(fldDescriptor.Name, shapeFileDataReader.GetValue(i + 1));
}
feature.Geometry = geometry;
IGeometry envelope = geometry.Envelope;
feature.BoundingBox = new Envelope(envelope.Coordinates[0], envelope.Coordinates[2]);
feature.Attributes = attributesTable;
features.Add(feature);
}
}
</pre>
<br />
<u>Provide a Web API that receives as input a latitude and longitude</u><br />
<u><br /></u>
I've used ASP.NET Core 2 for this. I've created a "RegionsController" with the following method<br />
<pre class="brush:csharp">[HttpGet("byCoordinates")]
public ActionResult<region> Get(string longitude, string latitude)
{
}
</region></pre>
This will define an API to be used as:<br />
http://<host>/api/regions/byCoordinates?longitude=xxx&latitude=yyy<br />
<br />
Inside this API there will be the code that does the actual filtering<br />
<br />
<u>Return the feature that contains that point</u><br />
<u><br /></u>
I've used a semi-brute force approach:<br />
- Iterate all polygons<br />
- But, as checking the intersection on a complex polygon is expensive, first check the bounding box, thus avoiding unnecessary calculation.<br />
<br />
The code for this is very simple:
<br />
<pre class="brush:csharp">for (var i = 0; i < features.Count; i++)
{
if (features[i].BoundingBox.Contains(coordinate)
&& features[i].Geometry.Contains(new Point(coordinate)))
{
matchFeature = _features[i];
break;
}
}
</pre>
I'm assuming that the regions as disjoint, hence breaking after finding the first match.<br />
The effective match is obtained using the "Contains" method.<br />
<br />
Obviously there are some additional details but you can check them at the repo itself: <a href="https://github.com/pmcxs/region-locator">https://github.com/pmcxs/region-locator</a><br />
<br />
<br />
<b>Closing remarks</b><br />
<br />
The performance is satisfactory. On my Mac it's taking around 1-2 ms for each reverse-geocoding operation with the ne_10m_admin_0_countries dataset from <a href="https://www.naturalearthdata.com/downloads/10m-cultural-vectors/">www.naturalearthdata.com</a>.<br />
<br />
For larger files I'm planning to support some fancier stuff like Quadtrees, which will reduce the number of polygons that need to be checked.<br />
<br />
Also, I'm pretty sure that I might be able to use something else other than the "Contains" method, or probably breaking up the polygons initially to make computation simpler.<br />
<br />
The README contains all required information to run this, but it's literally: clone, dotnet restore and dotnet run. Everything is setup to work out-of-the-box.</div>
</div>
</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com12tag:blogger.com,1999:blog-6029629494508626591.post-36645100821389235282019-05-25T00:21:00.000+01:002019-05-26T20:08:39.431+01:00Playing with Mapbox Vector Tiles (Part 3 - Using Mabox GL)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2017/02/playing-with-mapbox-vector-tiles-part-1.html">Part 1. End-to-End experiment</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2017/03/playing-with-mapbox-vector-tiles-part-2.html">Part 2. Generating custom tiles</a></span><br />
<span style="font-size: x-small;">Part 3. Using Mapbox GL</span></div>
<br />
On this post I'm going to pick-up what I've described on my previous two posts and creating a demo of Mapbox Vector Tiles integrated with Mapbox GL.<br />
<br />
I'll be more or less recreating what I did on this experiment: <a href="http://psousa.net/demos/maps-and-boardgames-part-2/demo3.html">http://psousa.net/demos/maps-and-boardgames-part-2/demo3.html</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih37RaiiJdyfIjy2X2pIfbUqx78nV_UfdLhnO0kfCTvA0riRafFPIlj8HO_ThLQvXP89Uts2jSCYtuwOftjucMqfVewSmc60y_EV_MYTQjXRiOu6s20eS69G8aXR6IFqhyphenhyphenmAnnPit_i7w/s1600/map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1300" data-original-width="1490" height="556" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEih37RaiiJdyfIjy2X2pIfbUqx78nV_UfdLhnO0kfCTvA0riRafFPIlj8HO_ThLQvXP89Uts2jSCYtuwOftjucMqfVewSmc60y_EV_MYTQjXRiOu6s20eS69G8aXR6IFqhyphenhyphenmAnnPit_i7w/s640/map.png" width="640" /></a></div>
<br />
That old experiment used:<br />
<ul>
<li>TileMill to create the raster tiles</li>
<li>UTFGrid tiles to provide meta-information on the hexagons</li>
<li>Leaflet as the mapping technology</li>
</ul>
This new one uses:<br />
<ul>
<li>Tipecanoe to generate the vector tiles (from geojson sources)</li>
<li>Mapbox GL JS as the mapping technology</li>
</ul>
The tricky bit is making sure that the helicopters snap to the displayed hexagons, ideally leveraging the vector data that's present on the tiles.<br />
<br />
With that said, I'm actually going to start with the end-result, followed by the break-down on how it's built.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR4ccGuhrU6StfwFTyH_HPNbhpFTfN3E9zS1DemRkPnojZ5sUH7jOKMR1HgoZg6HHcILa0NMKSnM68NjGb-a8EyGlwENZ4Eaju1tfHdf-BJyvOlH_p9vz1Ouh9xDBf8M8uo3NMh8W_oYk/s1600/map.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="720" data-original-width="1280" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgR4ccGuhrU6StfwFTyH_HPNbhpFTfN3E9zS1DemRkPnojZ5sUH7jOKMR1HgoZg6HHcILa0NMKSnM68NjGb-a8EyGlwENZ4Eaju1tfHdf-BJyvOlH_p9vz1Ouh9xDBf8M8uo3NMh8W_oYk/s640/map.gif" width="640" /></a></div>
<br />
I didn't try to replicate the experience at 100%, but it's close enough.<br />
<br />
Live demo at: <a href="http://psousa.net/demos/vector/part3/">http://psousa.net/demos/vector/part3/</a><br />
<br />
<b>How this was done:</b><br />
<br />
<a name='more'></a><b>Step 1: Creating the vector tiles</b><br />
<br />
This map has two layers: a base-map with the various countries, like what I did on my previous point and an hexagon layer.<br />
<br />
<u>Creating the geojson for the countries layer.</u><br />
<u><br /></u>
I'll summarize the steps I did for my previous post:<br />
<ul>
<li>Downloaded the file from: https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/50m/cultural/ne_50m_admin_0_countries.zip</li>
<li>Converted it to Geojson (I've used QGIS)</li>
</ul>
<div>
<u>Creating the geojson for the hexagon layer</u></div>
<ul>
<li>Used my own tool at: <a href="https://github.com/pmcxs/world-hexagon-map">https://github.com/pmcxs/world-hexagon-map</a></li>
<ul>
<li>It's not the scope of this post to explain how I've generated the Geojson layer with hexagons. For now, I'll simply include the end-result on the sample repo</li>
</ul>
</ul>
<u>Using tippecanoe to generate the vector tiles, passing both geojson files as arguments</u><br />
<br />
(please see my last post on how to install/use tippecanoe)<br />
<br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tippecanoe -o world.mbtiles -z6 world.geojson hexagons.geojson</pre>
</div>
<br />
<b>Step 2: Creating the Mapbox GL JS map</b><br />
<br />
I won't explain the code in detail (you can see it at: <a href="https://github.com/pmcxs/blog-experiments/tree/master/vector-tiles-part-3">https://github.com/pmcxs/blog-experiments/tree/master/vector-tiles-part-3</a>).<br />
<br />
As such, I'll try to keep all my new experiments in Github and leveraging the README there, instead of pasting all the instructions on the blog post.<br />
<br />
I'll highlight a couple of key points/features though:<br />
<ul>
<li>The helicopters can be dragged. When dropping them, they'll snap to the hexagon's center</li>
<li>This operation actually uses the real data in the Vector Tiles</li>
<li>The helicopters have two level of detail icons</li>
<li>The zoom level is restricted between zoom level 3 and 6</li>
</ul>
From an API Point-of-view Mapbox GL JS isn't as mature as the likes of Leaflet or OpenLayers, but its performance makes up for it.<br />
<br />
The main code is actually quite small:<br />
<br />
<div style="background: rgb(240, 240, 240); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 16.25px;"><!DOCTYPE html>
<html>
<head>
<meta charset='utf-8' />
<title>Simple Hexagon Map</title>
<meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
<script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.54.0/mapbox-gl.js'></script>
<script src="hexagons.js"></script>
<link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.54.0/mapbox-gl.css' rel='stylesheet' />
<style>
body {
margin: 0;
padding: 0;
}
#map {
position: absolute;
top: 0;
bottom: 0;
width: 100%;
}
</style>
</head>
<body>
<div id='map'></div>
<script>
var map = new mapboxgl.Map({
container: 'map',
style: 'server/style.json',
center: [0, 42],
maxZoom: 6.0,
minZoom: 3.0,
zoom: 4.5
});
var markers = [];
map.on('load', function () {
addHelicopter(166, 33);
addHelicopter(170, 30);
addHelicopter(175, 20);
addHelicopter(180, 20);
updateHelicoptersVisibility(map.getZoom());
});
map.on('zoom', function () {
updateHelicoptersVisibility(map.getZoom());
});
function addHelicopter(u, v) {
var hexagon = new HexDefinition(500);
var centerPoint = hexagon.getCenterPointXY(u, v);
var centerCoordinate = hexagon.pixelXYToLatLong(centerPoint.x, centerPoint.y, 10);
var el = document.createElement('div');
el.className = 'marker';
var marker = new mapboxgl.Marker(el)
.setLngLat([centerCoordinate.longitude, centerCoordinate.latitude])
.setDraggable(true)
.addTo(map);
marker.on('dragend', function (e) {
var coordinate = e.target.getLngLat();
var point = map.project(coordinate);
var features = map.queryRenderedFeatures(point, { layers: ['hexagons'] });
if (features.length > 0) {
var u = features[0].properties["U"];
var v = features[0].properties["V"];
var centerPoint = hexagon.getCenterPointXY(u, v);
var centerCoordinate = hexagon.pixelXYToLatLong(centerPoint.x, centerPoint.y, 10);
e.target.setLngLat([centerCoordinate.longitude, centerCoordinate.latitude]);
}
});
markers.push(marker);
}
function updateHelicoptersVisibility(zoomLevel) {
for (var i = 0; i < markers.length; i++) {
var element = markers[i].getElement();
if (zoomLevel > 4.7) {
element.style.backgroundImage = 'url(heli_1.png)';
element.style.width = '64px';
element.style.height = '64px';
}
else {
element.style.backgroundImage = 'url(heli_2.png)';
element.style.width = '32px';
element.style.height = '32px';
}
}
}
</script>
</body>
</html></pre>
</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-72431471121583669982019-04-18T20:53:00.001+01:002019-04-20T15:36:47.647+01:00Creating a large Geotif with Forest Coverage for the whole WorldFor a pet-project that I'm making I was trying to find accurate forest coverage for the whole World.<br />
<br />
A raster file seemed more adequate and I wanted something like this, but with a much higher resolution (and ideally already georeferenced)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://lcluc.umd.edu/sites/default/files/VCF.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="431" data-original-width="800" height="215" src="https://lcluc.umd.edu/sites/default/files/VCF.JPG" width="400" /></a></div>
<br />
I found the perfect data-source from Global Forest Watch: <a href="https://www.globalforestwatch.org/">https://www.globalforestwatch.org/</a><br />
<br />
Global Forest Watch provide an incredible dataset called "Tree Cover (2000)" that has a 30x30m resolution which includes the density of tree canopy coverage overall.<br />
<br />
It's too good to be true, right?<br />
<br />
Well, in a sense yes. The main problem is that it's just too much data and you can't download the image as a whole.<br />
<br />
Alternatively, they provide you an interactive map where you can download each section separately, at: <a href="http://earthenginepartners.appspot.com/science-2013-global-forest/download_v1.6.html">http://earthenginepartners.appspot.com/science-2013-global-forest/download_v1.6.html</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2YljB5SjmXQmr4w_noKKSPZ34bUU48I_TOdqF4wCkX4wnQTElAWeAci5tKIOB7wbAdP0DPjbU96NjItcxjrFJ7ixNUpM2W6kJ1uBIwXO8XZYo8IJdle2oZydijNWBIhAUn6TMbFqWI_c/s1600/grid.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="542" data-original-width="1376" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh2YljB5SjmXQmr4w_noKKSPZ34bUU48I_TOdqF4wCkX4wnQTElAWeAci5tKIOB7wbAdP0DPjbU96NjItcxjrFJ7ixNUpM2W6kJ1uBIwXO8XZYo8IJdle2oZydijNWBIhAUn6TMbFqWI_c/s640/grid.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
This consists of 504 (36x14) images, already georeferenced. For example, if you download the highlighted square above you'll get the the following picture:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy1Y4-g_4xj534Onvd7Seo6JqvI49W_armGy7ZUcn619y2YamADjl_oPnY9Zg6sea-qpknJwptd9P5PuJM-YiwXguhHD9IyIl2LIhcRorkL2poeGyNzfglvaJcyIfODQZBGaWpRxUOtPU/s1600/grid-sample.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="1146" data-original-width="1142" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiy1Y4-g_4xj534Onvd7Seo6JqvI49W_armGy7ZUcn619y2YamADjl_oPnY9Zg6sea-qpknJwptd9P5PuJM-YiwXguhHD9IyIl2LIhcRorkL2poeGyNzfglvaJcyIfODQZBGaWpRxUOtPU/s400/grid-sample.png" width="397" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/Hansen_GFC-2018-v1.6_treecover2000_50N_010W.tif" style="text-align: start;"><span style="font-size: x-small;">https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/Hansen_GFC-2018-v1.6_treecover2000_50N_010W.tif</span></a></td></tr>
</tbody></table>
It's "just" 218MB, hence you can somehow imagine the size of the whole lot. Should be massive.<br />
<br />
So, three challenges:<br />
<ol>
<li>How to download all images</li>
<li>How to merge them together to a single file</li>
<li>(Optional, but recommended) Reducing the resolution a bit to make it more manageable </li>
</ol>
<br />
<b>1. How to Download all images</b><br />
<b><br /></b>
Well, doing it manually is definitively an option, although it's probably easier to do it programmatically.
<br />
<pre class="prettyprint" style="background-color: #eeeeee; padding: 10px;"><span style="font-size: x-small;">import ssl
import urllib.request
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
sections = []
for i in range(8,-6,-1):
for j in range(-18,18):
sections.append(f'{abs(i)*10}{"N" if i >= 0 else "S"}_{str(abs(j)*10).zfill(3)}{"E" if j >= 0 else "W"}')
for section in sections:
url = 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/' + \
f'Hansen_GFC-2018-v1.6_treecover2000_{section}.tif'
with urllib.request.urlopen(url, context=ctx) as u, open(f"{section}.tif", 'wb') as f:
f.write(u.read())</span>
</pre>
The code above, in Python 3.x, iterates all the grid squares, prepares the proper download url and downloads the image.<br />
<br />
As the https certificate isn't valid you need to turn off the ssl checks, hence the code at the beginning.<br />
<br />
<b>2. How to merge them together to a single file</b><br />
<br />
It's actually quite simple, but you'll need GDAL for that, hence you'll need to install it first.<br />
<br />
<b>gdal_merge</b> is incredibly simple to use:<br />
<br />
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 21px; white-space: pre;">
<span style="color: #ce9178;">gdal_merge.py -o output-file-name.tif file1.tif file2.tif fileN.tif </span></div>
<br />
Adding to those parameters I would suggest compressing the output, as otherwise an already large file could become horrendously huge.<br />
<br />
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: Menlo, Monaco, "Courier New", monospace; font-size: 14px; line-height: 21px; white-space: pre;">
<span style="color: #ce9178;">g</span><span style="color: #ce9178;">dal_merge.py -o output-file-name.tif <files> -co COMPRESS=DEFLATE</span></div>
<br />
And that's it. I'll show how this all ties together on the Python script in the end, but you can "easily" do it manually if you concatenate the 504 file names on this command.<br />
<br />
<b>3. Reducing the resolution a bit to make it more manageable </b><br />
<b><br /></b>
As I've mentioned, the source images combined result in lots and lots of GBs, which I currently don't have available on my machine. Hence, I've reduced the resolution of each image.<br />
<br />
Please note that this isn't simply a resolution change on a Graphics Software, as it needs to preserve the geospatial information. Again, GDAL to the rescue, now using the <b>gdalwarp</b> command:<br />
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: menlo, monaco, "courier new", monospace; font-size: 14px; line-height: 21px; white-space: pre;">
<span style="color: #ce9178;">g</span><span style="color: #ce9178;">dalwarp -tr 0.0025 -0.0025 file.tif</span></div>
<br />
The first two parameters represent the pixel size. From running the command <b>gdalinfo</b> on any of the original tifs I can see that the original pixel size is:<br />
<br />
Pixel Size = (0.0002500000000000,-0.0002500000000000)<br />
<br />
Empirically I've decided to keep 1/10th of the original precision, hence I've supplied the aforementioned values (0.0025 -0.0025)<br />
<br />
As before, I would suggest compressing the content<br />
<div style="background-color: #1e1e1e; color: #d4d4d4; font-family: menlo, monaco, "courier new", monospace; font-size: 14px; line-height: 21px; white-space: pre;">
<span style="color: #ce9178;">g</span><span style="color: #ce9178;">dalwarp -tr 0.0025 -0.0025 file.tif -co COMPRESS=DEFLATE</span></div>
<br />
You do lose some quality, but it's a trade-off. If you have plenty of RAM + Disk Space you can keep an higher resolution.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMWeIk2NFoFYuQbAzsuhNhLvEupoO9coElHzIrkJzl5QhprvGrutujXwRSybdFvrnebK8mciQbaCLvsGnoElFp8F7dcEJQcC_cJNtibTrC-i9dfF9jRkAPL2KR-_AJ8_afionJVmYW8uw/s1600/detail.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="901" data-original-width="1600" height="221" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMWeIk2NFoFYuQbAzsuhNhLvEupoO9coElHzIrkJzl5QhprvGrutujXwRSybdFvrnebK8mciQbaCLvsGnoElFp8F7dcEJQcC_cJNtibTrC-i9dfF9jRkAPL2KR-_AJ8_afionJVmYW8uw/s400/detail.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Original</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><img border="0" data-original-height="905" data-original-width="1600" height="225" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi4bzCQh2TMxbWSwXSuop-iRfhBXJC-rTox47_wd8lmSKv_4niE8v3ktAiFiYnVahuSNR8iq0Ik072fRKHTY9lt-B1TBVJ_YnDvTTmKMi8PWz0z9nYQSNV6nyYWvzhPWEAVQeVVOlOjR6E/s400/detail-small.png" style="margin-left: auto; margin-right: auto;" width="400" /></td></tr>
<tr><td class="tr-caption" style="text-align: center;">1/10th of resolution</td></tr>
</tbody></table>
<b>Final script</b><br />
<b><br /></b>
The following Python 3 does everything in one go. The interesting bit is that I change the resolution of each individual tile before merging the complete map. The script also cleans up after itself, only leaving the final tif file, named "treecover2000.tif"
<br />
<pre class="prettyprint" style="background-color: #eeeeee; padding: 10px;"><span style="font-size: x-small;">import ssl
import urllib.request
import os
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
</span><span style="font-size: x-small;">extension = ".small.tif"
sections = []
for i in range(8,-6,-1):
for j in range(-18,18):
sections.append(f'{abs(i)*10}{"N" if i >= 0 else "S"}_{str(abs(j)*10).zfill(3)}{"E" if j >= 0 else "W"}')
for section in sections:
print(f'Downloading section {section}')
url = 'https://storage.googleapis.com/earthenginepartners-hansen/GFC-2018-v1.6/' + \
f'Hansen_GFC-2018-v1.6_treecover2000_{section}.tif'
with urllib.request.urlopen(url, context=ctx) as u, open(f"{section}.tif", 'wb') as f:
f.write(u.read())
os.system(f'gdalwarp -tr 0.0025 -0.0025 -overwrite {section}.tif {section}{extension} -co COMPRESS=DEFLATE')
os.system(f'rm {section}.tif')
</span><span style="font-size: x-small;">os.system(f'gdal_merge.py -o treecover2000.tif { (extension + " ").join(sections)}{extension} -co COMPRESS=DEFLATE')
os.system(f'rm *{extension}')</span>
</pre>
The "treecover2000.tif" ends-up with 751MB and looks AWESOME. Zooming in on Portugal, Spain and a bit of France<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26n16le2V28j5nf62Ozh_uHjR0tK3GM3SnaB7UfcxQQpIKv_sUIw_Lc9U_aoUrgz4VHDJGNXo0D6M5u1AlOu9Dh99a6sBAJ1oXtmJHveFWLbm4bjPhDYbCknD2X8-CTGcSee9x6DHVhA/s1600/grid-portugal.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1116" data-original-width="1600" height="446" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj26n16le2V28j5nf62Ozh_uHjR0tK3GM3SnaB7UfcxQQpIKv_sUIw_Lc9U_aoUrgz4VHDJGNXo0D6M5u1AlOu9Dh99a6sBAJ1oXtmJHveFWLbm4bjPhDYbCknD2X8-CTGcSee9x6DHVhA/s640/grid-portugal.png" width="640" /></a></div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-4450510828926831682017-03-07T00:14:00.002+00:002019-05-25T00:23:47.295+01:00Playing with Mapbox Vector Tiles (Part 2 - Generating custom tiles)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2017/02/playing-with-mapbox-vector-tiles-part-1.html">Part 1. End-to-End experiment</a></span><br />
<span style="font-size: x-small;">Part 2. Generating custom tiles</span><br />
<span style="font-size: x-small;"><a href="https://build-failed.blogspot.com/2019/05/playing-with-mapbox-vector-tiles-part-3.html">Part 3. Using Mabox GL</a></span></div>
<br />
On my previous post I've played around with various tools on the Mapbox vector tiles ecosystem. On this post I'm taking it further: generating my own vector tiles.<br />
<br />
There are various options to generate vector tiles:<br />
<ul>
<li>using a server that generates the vector tiles dynamically based on other data (such as PostGIS)</li>
</ul>
<ul>
<li>programatically generating the protobuf (rolling your own code or using existing libs)</li>
</ul>
<div>
<ul>
<li>using a tool that receives various inputs and outputs the corresponding vector tiles, which can then be used as shown on my previous post</li>
</ul>
<div>
Most options for these approaches are properly identified at this page: <a href="https://github.com/mapbox/awesome-vector-tiles">https://github.com/mapbox/awesome-vector-tiles</a></div>
<div>
<br /></div>
<div>
On this post I'm going to focus on this third option, particularly using a tool from Mapbox called <a href="https://github.com/mapbox/tippecanoe">Tippecanoe</a>. I'm not an expert on any of these alternatives (I'm learning as I'm writing this post) but Tippecanoe seems incredibly robust, including a great deal of customisation options when generating the tiles.</div>
</div>
<div>
<a name='more'></a><u><br /></u>
<u>So, first things first, what exactly is Tippecanoe?</u><br />
<br /></div>
<div>
It's simply a command-line tool that receives one or more GeoJson files as input and generates the corresponding vector tiles.</div>
<div>
<br /></div>
<div>
The <a href="https://github.com/mapbox/tippecanoe">readme</a> is great so I'm not going to duplicate everything on this post. Basically installing is as simple as:</div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">brew install tippecanoe
</pre>
</div>
<br />
And running as<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tippecanoe -o file.mbtiles <span style="color: #666666;">[</span>file.json ...<span style="color: #666666;">]</span>
</pre>
</div>
<br />
Let me create a very basic map to see how this works.<br />
<ul>
<li>I've started by downloading a world shapefile from <a href="http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries.zip">http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries.zip</a> (and extracting it obviously)</li>
</ul>
<ul>
<li>To convert a Shapefile to Geojson the easiest way is probably to use ogr2ogr from GDAL with the following command:</li>
</ul>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">ogr2ogr -f GeoJSON -t_srs crs:84 world.geojson ne_10m_admin_0_countries.shp</pre>
</div>
<i><span style="font-size: x-small;">(Alternatively, for a UI driven experience, I would suggest opening it in <a href="http://www.qgis.org/en/site/">QGis</a> and simply saving the map as GeoJSON)</span></i><br />
<ul>
<li>Now running tippecanoe on it:</li>
</ul>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tippecanoe -o world.mbtiles -z6 world.geojson
</pre>
</div>
<br />
The "z" param is used to specify the highest zoom level to which tiles are generated. The default of 14 would take a looong time to finish. For the sake of this demo I'm going to keep things small and fast. For reference each zoom level requires <b>4^(zoom level)</b> tiles.<br />
<br />
After the tiles are generated we end-up with a .mbtiles file. This is basically a SQLite database including all the generated tiles inside, making it quite convenient to store, transfer and use.<br />
<br />
I can try these vector tiles immediately but, without a style, nothing would be displayed. Fortunately tileserver-gl also provides a "data" mode where it shows the various layers of a mbtiles set.<br />
<br />
So, simply running it against my "world.mbtiles" output from the previous step:
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tileserver-gl-light world.mbtiles
</pre>
</div>
<br />
Opening <a href="http://localhost:8080/data/world">http://localhost:8080/data/world</a> will show our map with a single layer called "worldgeojson"<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgex-4JWnCYUinVugahRjEzSe-rISJV3-Sety0SjZs-D2nLDqrpFqoySgIrg4HnvvOGMekWNCf87RcfiaHdNwsoyfXxfjQHDsUdZldBuUV0b8ykcdnkp-DE9e8nVA5NBVQ0t1iQaR2YMKo/s1600/8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="523" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgex-4JWnCYUinVugahRjEzSe-rISJV3-Sety0SjZs-D2nLDqrpFqoySgIrg4HnvvOGMekWNCf87RcfiaHdNwsoyfXxfjQHDsUdZldBuUV0b8ykcdnkp-DE9e8nVA5NBVQ0t1iQaR2YMKo/s640/8.png" width="640" /></a></div>
<br />
I'm ready to create my style:<br />
<br />
Using an editor is obviously ideal (I'll get to that later) but we can easily create a really dumb style "by hand":<br />
<br />
<b>style.json</b><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">{
<span style="color: #062873; font-weight: bold;">"version"</span>: <span style="color: #40a070;">8</span>,
<span style="color: #062873; font-weight: bold;">"sources"</span>: {
<span style="color: #062873; font-weight: bold;">"myworldmap"</span>: {
<span style="color: #062873; font-weight: bold;">"type"</span>: <span style="color: #4070a0;">"vector"</span>,
<span style="color: #062873; font-weight: bold;">"url"</span>: <span style="color: #4070a0;">"mbtiles://{v3}"</span>
}
},
<span style="color: #062873; font-weight: bold;">"layers"</span>: [
{
<span style="color: #062873; font-weight: bold;">"id"</span>: <span style="color: #4070a0;">"background"</span>,
<span style="color: #062873; font-weight: bold;">"type"</span>: <span style="color: #4070a0;">"background"</span>,
<span style="color: #062873; font-weight: bold;">"paint"</span>: {
<span style="color: #062873; font-weight: bold;">"background-color"</span>: <span style="color: #4070a0;">"rgb(200, 200, 255)"</span>
}
},
{
<span style="color: #062873; font-weight: bold;">"id"</span>: <span style="color: #4070a0;">"countries"</span>,
<span style="color: #062873; font-weight: bold;">"type"</span>: <span style="color: #4070a0;">"fill"</span>,
<span style="color: #062873; font-weight: bold;">"source"</span>: <span style="color: #4070a0;">"myworldmap"</span>,
<span style="color: #062873; font-weight: bold;">"source-layer"</span>: <span style="color: #4070a0;">"worldgeojson"</span>,
<span style="color: #062873; font-weight: bold;">"paint"</span>: {
<span style="color: #062873; font-weight: bold;">"fill-color"</span>: <span style="color: #4070a0;">"rgb(200,100,50)"</span>
}
}
]
}
</pre>
</div>
<br />
This is basically:<br />
<ul>
<li>Creating a source for vector tiles (as in defining where to load the tiles from)</li>
<li>Creating a style layer to paint the background with a blue color</li>
<li>Creating a style layer that points to the "worldgeojson" layer from the vector data and filling it with a flat color</li>
</ul>
To use it with tileserver-gl I had to create a simplified configuration file:
<!-- HTML generated using hilite.me --><br />
<br />
<b>config.json</b><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">{
<span style="color: #062873; font-weight: bold;">"options"</span>: {
<span style="color: #062873; font-weight: bold;">"paths"</span>: {
<span style="color: #062873; font-weight: bold;">"root"</span>: <span style="color: #4070a0;">"."</span>,
<span style="color: #062873; font-weight: bold;">"styles"</span>: <span style="color: #4070a0;">"."</span>,
<span style="color: #062873; font-weight: bold;">"mbtiles"</span>: <span style="color: #4070a0;">"."</span>
}
},
<span style="color: #062873; font-weight: bold;">"styles"</span>: {
<span style="color: #062873; font-weight: bold;">"world"</span>: {
<span style="color: #062873; font-weight: bold;">"style"</span>: <span style="color: #4070a0;">"style.json"</span>
}
},
<span style="color: #062873; font-weight: bold;">"data"</span>: {
<span style="color: #062873; font-weight: bold;">"v3"</span>: {
<span style="color: #062873; font-weight: bold;">"mbtiles"</span>: <span style="color: #4070a0;">"world.mbtiles"</span>
}
}
}
</pre>
</div>
<br />
Running tileserver-gl as:
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tileserver-gl-light -c config.json world.mbtiles
</pre>
</div>
<br />
Opening a page at: <a href="http://localhost:8080/styles/world/#0/0/0">http://localhost:8080/styles/world/#0/0/0</a> will show our custom map with our custom style:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqg62pTmEX8gABTt0Ozcvpw6yPOwpP4BVAcnsQg8_raR0UXMHKiRkPnaK-zCDSF7ze0DNu02hqlMhDRnax2acU_-g9taVscmsPxcebe4WlMpsIUr8V9sgsZPsj1_zf7irBvmY3xMgVQZg/s1600/9.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="496" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgqg62pTmEX8gABTt0Ozcvpw6yPOwpP4BVAcnsQg8_raR0UXMHKiRkPnaK-zCDSF7ze0DNu02hqlMhDRnax2acU_-g9taVscmsPxcebe4WlMpsIUr8V9sgsZPsj1_zf7irBvmY3xMgVQZg/s640/9.png" width="640" /></a></div>
<br />
Creating the styles manually is not very practical. Fortunately there are various tools that might help with that:<br />
<ul>
<li> The best one if obviously <a href="http://www.mapbox.com/studio">www.mapbox.com/studio</a></li>
</ul>
<div>
It provides a rich UI where one can define colors, filters, fonts, etc in a very user friendly fashion.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrg9MUyMo67MObs7kZlZWYxEs6oSJ_oERYcJTt5ujsUDrBrS5agU-hdhNt5SzdfcKo-eDrqOKkuQ4Gwnx35MV79S3JWGJrYzS9HR9-I46MgDNhyphenhyphen5JPEh5ERb3GP-_Db6MD7QQKyo0jRG0/s1600/10.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="380" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjrg9MUyMo67MObs7kZlZWYxEs6oSJ_oERYcJTt5ujsUDrBrS5agU-hdhNt5SzdfcKo-eDrqOKkuQ4Gwnx35MV79S3JWGJrYzS9HR9-I46MgDNhyphenhyphen5JPEh5ERb3GP-_Db6MD7QQKyo0jRG0/s640/10.png" width="640" /></a></div>
<br />
It's particularly useful for maps created and hosted in Mapbox. Regardless, you can still edit the styles online and then download them to use with your local mbtiles, although requiring a little bit of editing on the output to point to the relevant sources.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOEshah5Wy5_ytS3ljw5zbsOAjBvN7kDNnqrB5pKXcb4ENT1uY90DCipT4Sao3Sy4NiOdeQE5gAAJMuDcoFiJiqwkts9u4g-GXweZcBGOQm6EV02Jlduig0RixRP4g3Ge3VCpe4OL2oHs/s1600/11.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="312" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgOEshah5Wy5_ytS3ljw5zbsOAjBvN7kDNnqrB5pKXcb4ENT1uY90DCipT4Sao3Sy4NiOdeQE5gAAJMuDcoFiJiqwkts9u4g-GXweZcBGOQm6EV02Jlduig0RixRP4g3Ge3VCpe4OL2oHs/s320/11.png" width="320" /></a></div>
<br />
<ul>
<li>A simple alternative is <a href="https://github.com/erikandre/mapbox-gl-style-editor">https://github.com/erikandre/mapbox-gl-style-editor</a>. </li>
</ul>
This is just a page with the map on one side and the Json style on the other. Simple and effective, although doesn't provide many bells and whistles.<br />
<ul>
</ul>
<ul>
<li>The best alternative to Mapbox Studio is definitely <a href="http://maputnik.com/">http://maputnik.com/</a>. </li>
</ul>
This is actually an open-source editor that provides some of the functionality of Mapbox Studio without any restrictions.<br />
<div>
<br />
Using it is very simple. You can either go directly to <a href="http://maputnik.com/editor/">http://maputnik.com/editor/</a> or clone the git repo from <a href="https://github.com/maputnik/editor">https://github.com/maputnik/editor</a> and run it as:</div>
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">npm install
npm start
</pre>
</div>
<br />
By default the editor runs on port 8888. The first thing to do is to add a source for our generated tiles. If you still have tileserver-gl running from one of the previous steps you'll have a tilejson file automatically prepared at: <a href="http://localhost:8080/data/v3.json">http://localhost:8080/data/v3.json</a>.<br />
<br />
Go to "Source" on the top and create a new source as:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizxytl2NbW05k6Xr7VwGQUOZdj9ctb3xGrUROMtdfMmgMC6ijkrOQSrU5l03TWtWktecBnDJlw4s3nBOO_Bnk_DvXI_6tMdArsxs6ywTezq0Mz7YBP2GTsfnkMXGfg2WCfiebxAX0JYQI/s1600/12.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="530" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizxytl2NbW05k6Xr7VwGQUOZdj9ctb3xGrUROMtdfMmgMC6ijkrOQSrU5l03TWtWktecBnDJlw4s3nBOO_Bnk_DvXI_6tMdArsxs6ywTezq0Mz7YBP2GTsfnkMXGfg2WCfiebxAX0JYQI/s640/12.png" width="640" /></a></div>
<br />
<ul>
<li>Source ID: mysource</li>
<li>Source Type: Vector (TileJSON URL)</li>
<li>TileJSON URL: http://localhost:8080/data/v3.json</li>
</ul>
<div>
Afterwards we're ready to create a new layer that references data from this particular source. Press <b>Add Layer </b>on the left and add the layer data:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSuqjgTsRNTLj0lOCw65MYgDMLFIJejxyRWKDF6MuQJEYjyDyNW45LVFk2_OLoQSLYgqM4-X5pkhyphenhyphenBAZ5ALKXcCzDxu071H3I0G0LdRCEiLcuFDW3ecHkBBnwoRSAOu_vQIS8dNVchcdQ/s1600/13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSuqjgTsRNTLj0lOCw65MYgDMLFIJejxyRWKDF6MuQJEYjyDyNW45LVFk2_OLoQSLYgqM4-X5pkhyphenhyphenBAZ5ALKXcCzDxu071H3I0G0LdRCEiLcuFDW3ecHkBBnwoRSAOu_vQIS8dNVchcdQ/s400/13.png" width="400" /></a></div>
<div>
<br /></div>
<br />
Also, create another layer for the background:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlZP3vB0ZiFvXmnbnYm18tHT5SRUYYksZtlH5l9IAgTpKJx52pNoTK3QAvodvT7n2bryVCtzFA6gKpdR7tKYWtVWtm1s2vaHehVaAsfR-yZHXVaPFnEPt-kcrGwz2byuu57jeHG9fcG3w/s1600/14.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="211" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlZP3vB0ZiFvXmnbnYm18tHT5SRUYYksZtlH5l9IAgTpKJx52pNoTK3QAvodvT7n2bryVCtzFA6gKpdR7tKYWtVWtm1s2vaHehVaAsfR-yZHXVaPFnEPt-kcrGwz2byuu57jeHG9fcG3w/s400/14.png" width="400" /></a></div>
<br />
Now just choose a flat color for these layers. You'll end-up with something like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWhXeb96-XZup6jklxZBxY9FYcTrVsZKu3KSAaQbbgfVSj-f18biNU8obMbj0jhkJ0TbTG8ZFEj06rEK-lWHOpzVcZj4mvwF4ykzgF9Da8Mv6WjowjTgUZ3dyswxoXc7Zzte5YapqQSok/s1600/15.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="540" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhWhXeb96-XZup6jklxZBxY9FYcTrVsZKu3KSAaQbbgfVSj-f18biNU8obMbj0jhkJ0TbTG8ZFEj06rEK-lWHOpzVcZj4mvwF4ykzgF9Da8Mv6WjowjTgUZ3dyswxoXc7Zzte5YapqQSok/s640/15.png" width="640" /></a></div>
<br />
That's it for now. On my third and final post I'm going to build up on these techniques and do something more advanced/interesting.<br />
<br />
<br />
<br />
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com3tag:blogger.com,1999:blog-6029629494508626591.post-43677851303346878842017-02-28T01:21:00.001+00:002019-05-25T00:24:15.225+01:00Playing with Mapbox Vector Tiles (Part 1 - End-to-end experiment)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;">Part 1. End-to-End experiment </span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2017/03/playing-with-mapbox-vector-tiles-part-2.html">Part 2. Generation custom tiles</a></span><br />
<a href="https://build-failed.blogspot.com/2019/05/playing-with-mapbox-vector-tiles-part-3.html"><span style="font-size: x-small;">Part 3. Using Mabox GL</span></a></div>
<br />
<div style="text-align: justify;">
<u>So, what are vector tiles?</u></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
For those that are not familiar with this approach, it provides an alternative to raster tiles where the main difference is that vector tiles provide data instead of a rendered image, although still using a similar xyz tiling structure.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
This provides some benefits, such as the ability to have meta-data associated with the tile and being able to generate different presentations for the same information.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
One of the problems is that there isn't an official standard for vector tiles. Regardless, with time, a particular vector tile format seems to have gained more traction than all others: the <a href="https://www.mapbox.com/vector-tiles/">Mapbox Vector Tiles</a>.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Mapbox has provided a consistent ecosystem around it and various providers have started to support it and, on this post, I'm going to play around with some of the existing technology around Mapbox Vector Tiles.</div>
<a name='more'></a><div style="text-align: justify;">
First, showing an actual example from Mapbox: <a href="https://www.mapbox.com/mapbox-gl-js/examples/">https://www.mapbox.com/mapbox-gl-js/examples/</a></div>
<div style="text-align: justify;">
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjh0kF77RgLeV44uc314uyHCyum2KyIODxSQ4omGkdjIuCK73BV3Mb0xeUGXgQWxzbJ3oYYtrPzix7bRGbEqDbn8fjaItOx9hyjFV-1nQSxljgWVp7tBT-6uyY3yr4vHprPwup248_0jUk/s1600/zoom.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="337" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjh0kF77RgLeV44uc314uyHCyum2KyIODxSQ4omGkdjIuCK73BV3Mb0xeUGXgQWxzbJ3oYYtrPzix7bRGbEqDbn8fjaItOx9hyjFV-1nQSxljgWVp7tBT-6uyY3yr4vHprPwup248_0jUk/s640/zoom.gif" width="640" /></a></div>
<br /></div>
<div style="text-align: justify;">
This page uses:</div>
<div style="text-align: justify;">
<ul>
<li>Vector tiles hosted by Mapbox</li>
<li>Mapbox GL: a browser plugin to render Mapbox Vector Tiles using WebGL</li>
</ul>
</div>
<div style="text-align: justify;">
The interesting part is that everything is being rendered in client-side, thus allowing for a really smooth transition between zoom levels and lots of flexibility, particularly as the styles are also applied in client-side.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<u>So, how can we use this technology for our own maps?</u></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>1. Using Mapbox Studio</b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
The obvious choice is using <a href="https://www.mapbox.com/mapbox-studio/">Mapbox Studio</a>, an online service where you can load data, style and publish the maps without much fuss. Mapbox provides various <a href="https://www.mapbox.com/pricing/">price ranges</a>, with a free tier that allows 50.000 map views per month, which seems pretty reasonable to test it out.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<b>2. Serving the Mapbox Vector Tiles (mvt) ourselves</b></div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Although Mapbox provides a paid service they don't force you to use it. The mvt spec is open and other libs/apps are able to read/write that format. Mapbox mantains a list of such projects here: <a href="https://github.com/mapbox/awesome-vector-tiles">https://github.com/mapbox/awesome-vector-tiles</a></div>
<div style="text-align: justify;">
<br />
So, to setup a local server there are various choices but I found the easiest one to be <a href="https://github.com/klokantech/tileserver-gl">tileserver-gl</a>. Setting it up was a breeze using npm:</div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">npm install -g tileserver-gl-light
</pre>
</div>
<div style="text-align: justify;">
<br />
and to run it:</div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tileserver-gl-light <map>
</pre>
</div>
<div style="text-align: justify;">
<br />
<div class="confluence-information-macro confluence-information-macro-information" style="background: rgb(252, 252, 252); border-radius: 5px; border: 1px solid rgb(170, 184, 198); color: #333333; font-family: Arial, sans-serif; font-size: 14px; margin: 10px 0px 1em; min-height: 20px; padding: 10px 10px 10px 36px; position: relative; text-align: start;">
<div class="confluence-information-macro-body" style="margin: 0px; padding: 0px;">
<div style="padding: 0px;">
<b>Note:</b> There's a "tileserver-gl" and a "tileserver-gl-light". The full version also supports a model were it renders the vector tiles in server-side and serves them as rasters to the client. I'm using the light version for simplicity.</div>
</div>
</div>
So now we need the actual map data to display. <a href="https://openmaptiles.org/downloads">https://openmaptiles.org/downloads</a>/ provides MVT files for the whole world, ready to use. I've downloaded the Portuguese map using the following command:</div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">curl -o portugal.mbtiles https://openmaptiles.os.zhdk.cloud.switch.ch/v3.3/extracts/portugal.mbtiles
</pre>
</div>
<span style="text-align: justify;"><br /></span>
<span style="text-align: justify;">Then I can just run the server as:</span><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">tileserver-gl-light portugal.mbtiles</pre>
</div>
<br />
It should be running properly on http://localhost:8080<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip9tl_cNeg0QrACIVqKaBMmOvkByIXlYY8FEx-94WB_jJa2P-YdrW78NZz77GH2DEnHu8eAToLCOXLjKLFX6YWuXjyFHlcMDLHUPNh2ZEtzHMGFHsKfgcxP9s2ACGGsimla9pN1R6wdeU/s1600/1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEip9tl_cNeg0QrACIVqKaBMmOvkByIXlYY8FEx-94WB_jJa2P-YdrW78NZz77GH2DEnHu8eAToLCOXLjKLFX6YWuXjyFHlcMDLHUPNh2ZEtzHMGFHsKfgcxP9s2ACGGsimla9pN1R6wdeU/s640/1.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://localhost:8080/styles/osm-bright/?vector#5.91/40.357/-5.857" style="text-align: start;"><span style="font-size: x-small;">http://localhost:8080/styles/osm-bright/?vector#5.91/40.357/-5.857</span></a></td></tr>
</tbody></table>
Zooming in we can see that the tile info is being fetched from the server and rendered accordingly.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK0i89sFNr4L5PIDHPzlddl5tRYW94DuAVxF2h-P66MfE82golmr02jMNb_YP5DZG3863vsC8Jcnv_mndeD3wctNjtHy60v7R8YYIUW1LGtBArYmheNjlStqWcwhaTH6NQ-M1j5iqvCVc/s1600/2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK0i89sFNr4L5PIDHPzlddl5tRYW94DuAVxF2h-P66MfE82golmr02jMNb_YP5DZG3863vsC8Jcnv_mndeD3wctNjtHy60v7R8YYIUW1LGtBArYmheNjlStqWcwhaTH6NQ-M1j5iqvCVc/s640/2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://localhost:8080/styles/osm-bright/?vector#15.91/38.7109/-9.3284" style="text-align: start;"><span style="font-size: x-small;">http://localhost:8080/styles/osm-bright/?vector#15.91/38.7109/-9.3284</span></a></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
<span style="text-align: justify;">Also, to prove that the rendering is actually being done in client-side, one can actually use a different style for the same data, which yields totally different results. Tile-server gl actually includes two different styles by default: <b>osm-bright</b> and <b>klokantech-basic</b>. Changing the "osm-bright" part of the url to "klokantech-basic":</span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKMa72SVWwwoDppFrdwevl7jreVf65yQgCc-VtOTDs3E5gEXpEqSuGI6ct8v1bl0ZTAo8O3TVMdIbPw7FuqeAOfBLz287v1F6KldhDsOmzZRg9p-HyekZyJzKX3eDIwzh0Dv5Z0Kn6_u0/s1600/3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKMa72SVWwwoDppFrdwevl7jreVf65yQgCc-VtOTDs3E5gEXpEqSuGI6ct8v1bl0ZTAo8O3TVMdIbPw7FuqeAOfBLz287v1F6KldhDsOmzZRg9p-HyekZyJzKX3eDIwzh0Dv5Z0Kn6_u0/s640/3.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://localhost:8080/styles/klokantech-basic/?vector#15.91/38.7109/-9.3284">http://localhost:8080/styles/klokantech-basic/?vector#15.91/38.7109/-9.3284</a></td></tr>
</tbody></table>
<span style="text-align: justify;"><b>3. Changing the style</b></span><br />
<span style="text-align: justify;"><br /></span>
Now I'm going to create a new style for the map. Creating one from scratch is very hard-work so I'll simply copy an existing one and change it slightly.<br />
<br />
Some steps required to do this in tileserver-gl:<br />
<ul>
<li>Run the server with a --verbose argument to know the default config</li>
</ul>
<div>
It will show the default configuration file, including a section with the root path under /options/paths/root<br />
<br /></div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">Automatically creating config file <span style="color: #007020; font-weight: bold;">for</span> portugal.mbtiles
{
<span style="color: #4070a0;">"options"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"paths"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"root"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"/Users/pedrosousa/.nvm/versions/node/v6.10.0/lib/node_modules/tileserver-gl-light/node_modules/tileserver-gl-styles"</span>,
<span style="color: #4070a0;">"fonts"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"fonts"</span>,
<span style="color: #4070a0;">"styles"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"styles"</span>,
<span style="color: #4070a0;">"mbtiles"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"/Users/pedrosousa/Projects/vectorTiles"</span>
}
},</pre>
</div>
<ul>
<li>On that root folder there is a "styles" subolder with a folder for each style, particulary "klokantech-basic" and "osm-bright". I'm going to copy "klokantech-basic" as it's simpler to a new folder called "custom"</li>
</ul>
<ul>
<li>Inside I'm just going to change a couple of properties:</li>
<ul>
<li>Name: "Custom"</li>
<li>Background-Color to white: "rgba(255, 255, 255, 1)" </li>
<li>Water Fill-Color to be a dark blue: "rgba(12, 55, 84, 1)"</li>
</ul>
</ul>
<ul>
<li>Last, I need to add this new style to the tileserver-gl configuration file. As I've been using the default one I actually need to create a new file. The easiest way is to copy the output obtained on the first step and just add the new style copying one of the others. For reference this is my file:</li>
</ul>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">{
<span style="color: #4070a0;">"options"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"paths"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"root"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"/Users/pedrosousa/.nvm/versions/node/v6.10.0/lib/node_modules/tileserver-gl-light/node_modules/tileserver-gl-styles"</span>,
<span style="color: #4070a0;">"fonts"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"fonts"</span>,
<span style="color: #4070a0;">"styles"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"styles"</span>,
<span style="color: #4070a0;">"mbtiles"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"/Users/pedrosousa/Projects/vectorTiles"</span>
}
},
<span style="color: #4070a0;">"styles"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"klokantech-basic"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"style"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"klokantech-basic/style.json"</span>,
<span style="color: #4070a0;">"tilejson"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"bounds"</span><span style="color: #666666;">:</span> [
<span style="color: #666666;">-</span><span style="color: #40a070;">31.6575302</span>,
<span style="color: #40a070;">29.7288021</span>,
<span style="color: #666666;">-</span><span style="color: #40a070;">6.0891591</span>,
<span style="color: #40a070;">42.2543112</span>
]
}
},
<span style="color: #4070a0;">"custom"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"style"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"custom/style.json"</span>,
<span style="color: #4070a0;">"tilejson"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"bounds"</span><span style="color: #666666;">:</span> [
<span style="color: #666666;">-</span><span style="color: #40a070;">31.6575302</span>,
<span style="color: #40a070;">29.7288021</span>,
<span style="color: #666666;">-</span><span style="color: #40a070;">6.0891591</span>,
<span style="color: #40a070;">42.2543112</span>
]
}
},
<span style="color: #4070a0;">"osm-bright"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"style"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"osm-bright/style.json"</span>,
<span style="color: #4070a0;">"tilejson"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"bounds"</span><span style="color: #666666;">:</span> [
<span style="color: #666666;">-</span><span style="color: #40a070;">31.6575302</span>,
<span style="color: #40a070;">29.7288021</span>,
<span style="color: #666666;">-</span><span style="color: #40a070;">6.0891591</span>,
<span style="color: #40a070;">42.2543112</span>
]
}
}
},
<span style="color: #4070a0;">"data"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"v3"</span><span style="color: #666666;">:</span> {
<span style="color: #4070a0;">"mbtiles"</span><span style="color: #666666;">:</span> <span style="color: #4070a0;">"portugal.mbtiles"</span>
}
}
}</pre>
</div>
<div>
<ul>
<li>Now you need to launch the server with a "-c" property to specify the new configuration file that was just created. Assuming it's called "config.json":</li>
</ul>
<div style="background: rgb(240, 240, 240); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 16.25px;">tileserver-gl-light portugal.mbtiles -c config.json</pre>
</div>
</div>
<ul>
<li>Voilá. Now opening the browser with the corresponding "/custom" style will show the updated map style</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJ7CsM72LYp3uizciQ9pm0DnRJQ-cpdVc8Hp15FXkXB9Muk6KmuWQEEQuB6-UBu7vYvyNC_2FegCpQb2PdyHhT_vKWrake8ueLrA6HAS0P9h-7is6swpOcnTmrDf0vdDFtgWfVwC8qNS8/s1600/5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="539" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJ7CsM72LYp3uizciQ9pm0DnRJQ-cpdVc8Hp15FXkXB9Muk6KmuWQEEQuB6-UBu7vYvyNC_2FegCpQb2PdyHhT_vKWrake8ueLrA6HAS0P9h-7is6swpOcnTmrDf0vdDFtgWfVwC8qNS8/s640/5.png" width="640" /></a></div>
<div>
<br /></div>
<div>
<b>4. Serving the vector tiles directly without a tile-server</b></div>
<div>
<br /></div>
<div>
Although I would recommend having a tile-server (particularly with a nginx front-facing it) sometimes one might want to serve the individual tiles directly.</div>
<div>
<br /></div>
<div>
The easiest way is probably to use another tool from mapbox called <a href="https://github.com/mapbox/mbutil">mb-util</a> to extract the tiles from the .mbtiles file.</div>
<div>
<br />
I've used the steps that I've found on this page: <a href="https://github.com/klokantech/vector-tiles-sample">https://github.com/klokantech/vector-tiles-sample</a></div>
<div>
<br />
1. After installing mbutil you can run the following command on the folder where you have your mbtiles file</div>
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">mb-util --image_format<span style="color: #666666;">=</span>pbf portugal.mbtiles portugal</pre>
</div>
<br />
2. We still need to extract them as they're usually gzipped.
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">gzip -d -r -S .pbf *
</pre>
</div>
<br />
3. After extracting the pbf extension is lost. The following command iterates the various files and puts a .pbf extension on them again:
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">find . -type f -exec mv <span style="color: #4070a0;">'{}'</span> <span style="color: #4070a0;">'{}'</span>.pbf <span style="color: #4070a0; font-weight: bold;">\;</span>
</pre>
</div>
<br />
Ok, now that we have our tiles we're ready to serve them. We just need a very simple index.html file:
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #007020;"><!DOCTYPE html></span>
<span style="color: #062873; font-weight: bold;"><html></span>
<span style="color: #062873; font-weight: bold;"><head></span>
<span style="color: #062873; font-weight: bold;"><meta</span> <span style="color: #4070a0;">charset='utf-8'</span> <span style="color: #062873; font-weight: bold;">/></span>
<span style="color: #062873; font-weight: bold;"><title></span>Vector Map with Mapbox GL JS<span style="color: #062873; font-weight: bold;"></title></span>
<span style="color: #062873; font-weight: bold;"><meta</span> <span style="color: #4070a0;">name='viewport'</span> <span style="color: #4070a0;">content='initial-scale=1,maximum-scale=1,user-scalable=no'</span> <span style="color: #062873; font-weight: bold;">/></span>
<span style="color: #062873; font-weight: bold;"><style></span>
<span style="color: #062873; font-weight: bold;">body</span> {
<span style="color: #007020; font-weight: bold;">margin</span><span style="color: #666666;">:</span> <span style="color: #40a070;">0</span>;
<span style="color: #007020; font-weight: bold;">padding</span><span style="color: #666666;">:</span> <span style="color: #40a070;">0</span>;
}
<span style="color: #06287e;">#map</span> {
<span style="color: #007020; font-weight: bold;">position</span><span style="color: #666666;">:</span> <span style="color: #007020; font-weight: bold;">absolute</span>;
<span style="color: #007020; font-weight: bold;">top</span><span style="color: #666666;">:</span> <span style="color: #40a070;">0</span>;
<span style="color: #007020; font-weight: bold;">bottom</span><span style="color: #666666;">:</span> <span style="color: #40a070;">0</span>;
<span style="color: #007020; font-weight: bold;">width</span><span style="color: #666666;">:</span> <span style="color: #40a070;">100</span><span style="color: #666666;">%</span>;
}
<span style="color: #062873; font-weight: bold;"></style></span>
<span style="color: #062873; font-weight: bold;"></head></span>
<span style="color: #062873; font-weight: bold;"><body></span>
<span style="color: #062873; font-weight: bold;"><script </span><span style="color: #4070a0;">src='https://api.mapbox.com/mapbox-gl-js/v0.32.1/mapbox-gl.js'</span><span style="color: #062873; font-weight: bold;">></script></span>
<span style="color: #062873; font-weight: bold;"><div</span> <span style="color: #4070a0;">id='map'</span><span style="color: #062873; font-weight: bold;">></div></span>
<span style="color: #062873; font-weight: bold;"><script </span><span style="color: #4070a0;">type="text/javascript"</span><span style="color: #062873; font-weight: bold;">></span>
<span style="color: #007020; font-weight: bold;">var</span> map <span style="color: #666666;">=</span> <span style="color: #007020; font-weight: bold;">new</span> mapboxgl.Map({
container<span style="color: #666666;">:</span> <span style="color: #4070a0;">'map'</span>,
center<span style="color: #666666;">:</span> [<span style="color: #40a070;">12</span>, <span style="color: #40a070;">1</span>],
zoom<span style="color: #666666;">:</span> <span style="color: #40a070;">1</span>,
style<span style="color: #666666;">:</span> <span style="color: #4070a0;">'style.json'</span>
});
<span style="color: #062873; font-weight: bold;"></script></span>
<span style="color: #062873; font-weight: bold;"></body></span>
<span style="color: #062873; font-weight: bold;"></html></span>
</pre>
</div>
Most of the relevant information comes from the style.json file that I'm referencing on the map definition.<br />
<br />
I'm not going to put the entire style file here as it's quite big and is mostly a copy from the <b style="text-align: justify;">klokantech-basic</b><span style="text-align: justify;"> I've mentioned above. The relevant bits are:</span>
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">{
<span style="color: #062873; font-weight: bold;">"sources"</span>: {
<span style="color: #062873; font-weight: bold;">"openmaptiles"</span>: {
<span style="color: #062873; font-weight: bold;">"type"</span>: <span style="color: #4070a0;">"vector"</span>,
<span style="color: #062873; font-weight: bold;">"tiles"</span>: [
<span style="color: #4070a0;">"http://localhost:8000/portugal/{z}/{x}/{y}.pbf"</span>
]
}
}
}</pre>
</div>
Basically setting up the source of the tiles to the folder that we've extracted.<br />
<br />
Now running a simple web-server on the folder (ex: using Python Simple Server)
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">python -m SimpleHTTPServer
</pre>
</div>
<br />
Opening http://localhost:8000 should show a map similar to the one we had before using tileserver-gl, although with a slightly different url pattern. The tiles are being served directly from the web-server without any translation layer in the middle<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvSW9rQcbh4L2q5nJq0-Uehy-4cnB13eQgz5u2BsKAysZrVQHtkkDXMHRqdDZQ1Sv_CL5yk0t8V_i0mhlgZ9cq6ozVq_-t5dSuODdDNssz-OEdA6jOFcJPEZC9S7APaPI8XIOamH8htSM/s1600/6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="592" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjvSW9rQcbh4L2q5nJq0-Uehy-4cnB13eQgz5u2BsKAysZrVQHtkkDXMHRqdDZQ1Sv_CL5yk0t8V_i0mhlgZ9cq6ozVq_-t5dSuODdDNssz-OEdA6jOFcJPEZC9S7APaPI8XIOamH8htSM/s640/6.png" width="640" /></a></div>
<br />
Ok, that's it for now. Next post: generating my own vector tiles<br />
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-42209244980863890612016-10-25T20:15:00.001+01:002016-10-25T20:17:15.688+01:00Splitting vector and raster files in QGIS (Part 2 - Creating a proper QGIS plugin)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/12/splitting-vector-and-raster-files-in.html">Part 1. Splitting vector and raster files in QGIS</a></span><br />
<span style="font-size: x-small;">Part 2. Creating a proper QGIS plugin</span></div>
<br />
On my previous post I've created a python script which could be executed inside QGIS in order to split the various layers onto smaller, more manageable chunks.<br />
<br />
I've been wanting to create a proper QGIS plugin to package that logic for some time but I've been postponing it as I was expecting it to be slightly tricky, mostly due to the fact that I've never done a QGIS plugin before and anticipated that it could quite challenging.<br />
<br />
Truth be told, it was actually quite simple. Due to a mix of awesome online documentation and some starter tools this was mostly a straightforward process.<br />
<br />
Let me start by showing the end-result and them I'll explain how it was built.<br />
<br />
<a name='more'></a><b>The plugin:</b><br />
<br />
It's called "Geo Grid Cut" and it basically receives as input all the layers of a map. For example, assuming we have the following layers open on QGIS<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhARXj7cNlgiq4CXxhohH0DmKCVL-LT-rS3wKZdZOlvV31aVEbx0mx3Eb1LzyRm8k8GZyeHz9RgNq6vASsYPqE4sKtZ-8xOxOLGN_ujugTAHVRwPSInjYBaMNz_OgJDmc6zCIMdLggbqKA/s1600/layers.tiff" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid0ykS2Cgdt-MWwTE006OlySUPFuZ3xsClMj1CZhAna3UJTMzVTw7ESpTnbtcQFPVfnUspzE2r6_SQP89-ZvGV5QqcHZn3d-I0Qj-czZo7LeOHWMNINSM48cc-UAMcmz_UkbNnbOWnNxg/s1600/map_animated.gif" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="251" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEid0ykS2Cgdt-MWwTE006OlySUPFuZ3xsClMj1CZhAna3UJTMzVTw7ESpTnbtcQFPVfnUspzE2r6_SQP89-ZvGV5QqcHZn3d-I0Qj-czZo7LeOHWMNINSM48cc-UAMcmz_UkbNnbOWnNxg/s400/map_animated.gif" width="400" /></a><img border="0" height="69" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhARXj7cNlgiq4CXxhohH0DmKCVL-LT-rS3wKZdZOlvV31aVEbx0mx3Eb1LzyRm8k8GZyeHz9RgNq6vASsYPqE4sKtZ-8xOxOLGN_ujugTAHVRwPSInjYBaMNz_OgJDmc6zCIMdLggbqKA/s200/layers.tiff" width="200" /></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
After installing (I'll talk about that later) the plugin can be accessed on the "plugins" menu on option "Geo Grid Cut":<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBIpg8F1h8jfnoyk0N8Hw93s5R5_78g8WifYCTHnlyLTXQkKtheRJpHiWgyK4b1HwZDECCF0D58IsAOZliufLjyceyZdRGGdBsEK2DTC-DdNOF6FB7hz2yzRYU4TKvW4-ug0UQHUwXTYA/s1600/plugin-choose.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBIpg8F1h8jfnoyk0N8Hw93s5R5_78g8WifYCTHnlyLTXQkKtheRJpHiWgyK4b1HwZDECCF0D58IsAOZliufLjyceyZdRGGdBsEK2DTC-DdNOF6FB7hz2yzRYU4TKvW4-ug0UQHUwXTYA/s640/plugin-choose.jpg" width="640" /></a></div>
<br />
It provides various options but for now I'm just going to split a small custom area in Europe<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihGPEPbnha3QFNYsQIuAH3LQMcjmKdV-nWIMttRUy67Mw8ZOFd-bQWxvp_vYmI6OtAsN5deQ35HMYuiKRO5y2XljpLJkRNKULhjzvvIJt41uloIhW5sahqoTVkR4EY1kiQdL1LVqX5c9s/s1600/cut_params.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEihGPEPbnha3QFNYsQIuAH3LQMcjmKdV-nWIMttRUy67Mw8ZOFd-bQWxvp_vYmI6OtAsN5deQ35HMYuiKRO5y2XljpLJkRNKULhjzvvIJt41uloIhW5sahqoTVkR4EY1kiQdL1LVqX5c9s/s400/cut_params.jpg" width="383" /></a></div>
<br />
For this particular example the end-result is 6 zip files, each corresponding to an area of 10x10 (degrees) that includes the corresponding features of all 4 layers inside that area.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0DBHCmtzqv3GJ2rDeu0srKqzOKFalgzI4hn1yx43GBUGnB7Du-NLWy5iLgGAGgQxV4kfqzVDLj-z4XEmlJOG8JdMeDk7xZKOptTl7rzWit4cwQeEMDNJc3HE4n77maAIq-8luM4dQgyU/s1600/map_split.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="438" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh0DBHCmtzqv3GJ2rDeu0srKqzOKFalgzI4hn1yx43GBUGnB7Du-NLWy5iLgGAGgQxV4kfqzVDLj-z4XEmlJOG8JdMeDk7xZKOptTl7rzWit4cwQeEMDNJc3HE4n77maAIq-8luM4dQgyU/s640/map_split.jpg" width="640" /></a></div>
<br />
<b>Creating the plugin:</b><br />
<b><br /></b>
I was very pleasantly surprised that this process is so well documented here: <a href="http://www.qgistutorials.com/en/docs/building_a_python_plugin.html">http://www.qgistutorials.com/en/docs/building_a_python_plugin.html</a><br />
<br />
Besides providing a detailed explanation it's also very "iterative" in the sense that in 5 min you have something up and running.<br />
<br />
As that page includes most required info I'll just add some additional considerations:<br />
<ul>
<li>After installing the Qt Creator and the Python Bindings for Qt my "make" command was not working. On MacOsX a restart was needed</li>
<li>(at least on MacOSX) Be careful with the casing on the plugin name and the corresponding folder on disk. If you name your plugin as "MyFirstPlugin" be sure to also name the repo as "MyFirstPlugin". For simplicity I would simply keep everything lowercase</li>
<li>Most existing plugins are open-source, which is particularly useful to see how things are done</li>
<li>By default you'll have a python "<plugin>_dialog_base.py" and a "<plugin>.py" file. Keep the UI logic on the corresponding "dialog" python file. Keeps thinks cleaner</li>
</ul>
<br />
<b>Additional plugin parameters:</b><br />
<br />
The plugin includes some additional configuration options:<br />
<ul>
<li>Extent - Defines the bounding box that will be split into smaller chunks</li>
<ul>
<li>Layers - Automatically sets the extent to cover all the layers on the map (default)</li>
<li>Canvas - Will use the current viewport to set the extent</li>
<li>Custom - Set the coordinates for the bounding box manually</li>
</ul>
<li>Grid Size</li>
<ul>
<li>Width - Width of each cell of the grid (in degrees)</li>
<li>Height - Height of each cell of the grid (in degrees)</li>
<li>Buffer - Defines the overlap (for each direction) when creating the grid cells. For example, if a buffer is set to anything > 0 an area around the edges will be shared by various cells. This is useful to prevent gaps between cells when processing the resulting output.</li>
</ul>
<li>Output</li>
<ul>
<li>Folder - Base folder where the output will be placed</li>
<li>Compress output - If checked will create a zip file for each cell. Otherwise an uncompressed folder will be created</li>
<li>Create boundaries description file - For each cell creates a description file that indicates the coordinates for that cell boundaries</li>
</ul>
</ul>
<br />
<b>Source-code and Installing</b><br />
<br />
The source-code is available at: <a href="https://github.com/pmcxs/geogridcut">https://github.com/pmcxs/geogridcut</a><br />
<br />
To install locally the easiest way is to:<br />
<ul>
<li>Go to the QGIS folder and clone the plugin's repo with its default name</li>
</ul>
<pre class="prettyprint">cd /Users/<username>/.qgis2/python/plugins
git clone https://github.com/pmcxs/geogridcut
</pre>
<br />
<ul>
<li>Launch (or relaunch) QGIS and go to "Plugins > Manage and Install Plugins"</li>
<li>Under the "Installed" section you'll see "Geo Grid Cut" there. Press the checkbox near its name to activate it</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf9u0iZXEIKRIT4NqX-egOwF591npINgXrb-GOgEpJDBXFRow3lmXWbJ2cNDMVyi-cC2qGS1jK7vhjmly1iYxj7phab3gALzBwS96oSYxFeCncay2lbpUMPorMMj9zE5XM3_vgSSKxe1g/s1600/activateplugin.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="334" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjf9u0iZXEIKRIT4NqX-egOwF591npINgXrb-GOgEpJDBXFRow3lmXWbJ2cNDMVyi-cC2qGS1jK7vhjmly1iYxj7phab3gALzBwS96oSYxFeCncay2lbpUMPorMMj9zE5XM3_vgSSKxe1g/s640/activateplugin.jpg" width="640" /></a></div>
<div>
<ul>
<li>The plugin should now appear on the plugins menu, ready to use :)</li>
</ul>
</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-24967438879590686912016-08-02T21:28:00.000+01:002016-09-07T09:40:08.582+01:00Creating a simple TileServer with .NET Core 1.0 (Part 2 - Drawing Lines)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2016/06/creating-simple-tileserver-with-net.html">Part 1. Setting up the the project</a></span><br />
<span style="font-size: x-small;">Part 2. Improving drawing logic</span></div>
<br />
On my last post I've setup an asp.net core project which outputs simple maps tiles dynamically generated using the ImageProcessor library (which now supports .NET Core).<br />
<br />
As I've mentioned that lib doesn't yet include drawing functionality, so on this post I'll try to address that as a pre-requisite to being able to draw proper map tiles. For now I'll focus on drawing lines, including support for variable width and anti-aliasing, doing some benchmarking along the way to make sure that the performance is adequate.<br />
<br />
By-the-way, since my first post the proper 1.0 release has been launched, so I'm also updating the code to reflect the latest bits :)<br />
<br />
Lots of stuff to do so let's get started:<br />
<a name='more'></a><br />
<u>1. Drawing simple lines</u><br />
<br />
For drawing lines I've used <a href="https://en.wikipedia.org/wiki/Bresenham%27s_line_algorithm">Bresenham's line algorithm</a>. It's really simple to implement and provides accurate and fast results. The implementation (for all 4 quadrants) is:
<br />
<pre>void DrawLine(Image image, int x1, int y1, int x2, int y2, Color color)
{
int w = x2 - x1;
int h = y2 - y1;
int dx1 = 0, dy1 = 0, dx2 = 0, dy2 = 0;
if (w < 0) dx1 = -1; else if (w > 0) dx1 = 1;
if (h < 0) dy1 = -1; else if (h > 0) dy1 = 1;
if (w < 0) dx2 = -1; else if (w > 0) dx2 = 1;
int longest = Math.Abs(w);
int shortest = Math.Abs(h);
if (!(longest > shortest))
{
longest = Math.Abs(h);
shortest = Math.Abs(w);
if (h < 0) dy2 = -1; else if (h > 0) dy2 = 1;
dx2 = 0;
}
int numerator = longest >> 1;
for (int i = 0; i <= longest; i++)
{
image.SetPixel(x1, y1, color);
numerator += shortest;
if (!(numerator < longest))
{
numerator -= longest;
x1 += dx1;
y1 += dy1;
}
else
{
x1 += dx2;
y1 += dy2;
}
}
}
</pre>
I'm drawing the following lines with it:<br />
<br />
<pre>_lineDrawing.DrawLine(image, 0, 0, 255, 255, Color.Red);
_lineDrawing.DrawLine(image, 0, 255, 255, 0, Color.Red);
_lineDrawing.DrawLine(image, 0, 63, 255, 191, Color.Red);
_lineDrawing.DrawLine(image, 0, 191, 255, 63, Color.Red);
_lineDrawing.DrawLine(image, 0, 127, 255, 127, Color.Red);
_lineDrawing.DrawLine(image, 127, 0, 127, 255, Color.Red);
_lineDrawing.DrawLine(image, 63, 0, 191, 255, Color.Red);
_lineDrawing.DrawLine(image, 191, 0, 63, 255, Color.Red);
</pre>
The end-result is this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQmTBi_iGEmaifpeHschCKl9_N5hyphenhyphenBS8Eby_01tnMDKtw1Pdb-qbAVgrVssXNNtkmr0ucLsrXrQ94ZMDJkzamoCXLffvvVMwcJoPIKfjNv7COL_kBWTGoQAAE-6ahlWQgPtPeUYe4oB_E/s1600/star_lines.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQmTBi_iGEmaifpeHschCKl9_N5hyphenhyphenBS8Eby_01tnMDKtw1Pdb-qbAVgrVssXNNtkmr0ucLsrXrQ94ZMDJkzamoCXLffvvVMwcJoPIKfjNv7COL_kBWTGoQAAE-6ahlWQgPtPeUYe4oB_E/s1600/star_lines.png" /></a></div>
Regarding performance, on my laptop (a pretty standard machine) it takes in average 15 ms to render this particular tile. The actual drawing logic for these 8 lines takes less than 0.5 ms and outputting the png around 12 ms.<br />
<br />
<u>2. Adding thickness to the line </u><br />
<br />
This was actually trickier than I had anticipated. I had two different approaches to this:<br />
<br />
First approach: <br />
<ul>
<li>Drawing a regular simple line</li>
<li>For each point create 2 perpendicular lines (one for each direction) a draw it up to a certain distance</li>
</ul>
Second approach:<br />
<ul>
<li>Calculate the perpendicular points at the start and end of the line</li>
<li>Draw various parallel lines to the main one, until achieving a certain width.</li>
</ul>
Both approaches apparently worked, although I was obtaining consistently better performance with the second approach.<br />
<br />
Unfortunately there was a problem I hadn't anticipated. When drawing parallel lines it's possible that you end up with spaces between them if the perpendicular (normal) points represent both an horizontal and vertical step.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqQGHbNWNKQzw3TTMG1JLiHn47IYBsTYhzFlR0fiM1ResxhCUQHv5EpQEKguxuLKRMtjjOuRVkFSD6PokwXuAmCSByMt2YtCnUo2Sh6MLEtg_xzReKkcoEMVXLpvSeDchqbcvRbFJ1wTA/s1600/gaps.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="301" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqQGHbNWNKQzw3TTMG1JLiHn47IYBsTYhzFlR0fiM1ResxhCUQHv5EpQEKguxuLKRMtjjOuRVkFSD6PokwXuAmCSByMt2YtCnUo2Sh6MLEtg_xzReKkcoEMVXLpvSeDchqbcvRbFJ1wTA/s320/gaps.PNG" width="320" /></a></div>
<br />
To solve this I've changed the drawing logic to pick the points on the normal without doing any diagonals. I've actually used a modified Bresenham's algorithm for this. The end-result is conceptually like this:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8gtfOyDlmWyeamW6CCYJwVAmHCYZPlexkEGRz66mRmwXx_cnKzRBN6LB3h-Geo69QzZoMl-fL9JvSmC1zRTA6HWoqFBVNYsvzELnCNstdJqsoZF-0c3pP_rj8mkVejHH8sFVVRvTTbgA/s1600/no_gaps.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8gtfOyDlmWyeamW6CCYJwVAmHCYZPlexkEGRz66mRmwXx_cnKzRBN6LB3h-Geo69QzZoMl-fL9JvSmC1zRTA6HWoqFBVNYsvzELnCNstdJqsoZF-0c3pP_rj8mkVejHH8sFVVRvTTbgA/s1600/no_gaps.PNG" /></a></div>
<br />
Applying this logic to the previous star pattern with a width of 5:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPRa0M384Aud781Ul2D0gyLE4NF29pI_ORP0VabBu-BHT31L4L65hl38pncWRooW64zKuuYndjfjPPKZBDJb5p3R4mq-_ExsGg5kgo9znTdOxf4W9i4ZBYOsup2PVMaukalql-KjkLztQ/s1600/star_lines_bold.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPRa0M384Aud781Ul2D0gyLE4NF29pI_ORP0VabBu-BHT31L4L65hl38pncWRooW64zKuuYndjfjPPKZBDJb5p3R4mq-_ExsGg5kgo9znTdOxf4W9i4ZBYOsup2PVMaukalql-KjkLztQ/s1600/star_lines_bold.png" /></a></div>
Regarding performance, adding the thickness to the drawing algorithm was not noticeable in the overall time, and the full drawing logic still takes less than 1 ms.<br />
<br />
Unfortunately, when zooming in we can see it's quite pixelated (aliased).<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd8IkbKgO1ga5p2ktO3le_EgPNTzDZTFSuk-EW00d6ouvJt78lnnairt7DXoRZjbfyGbv-KZ8kxk5be5Sjq36M58-CQvEbMjobK28EwXSGNSaG62RqbcshSprIimzB0PXAcwX_ddcXl9o/s1600/jagged.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjd8IkbKgO1ga5p2ktO3le_EgPNTzDZTFSuk-EW00d6ouvJt78lnnairt7DXoRZjbfyGbv-KZ8kxk5be5Sjq36M58-CQvEbMjobK28EwXSGNSaG62RqbcshSprIimzB0PXAcwX_ddcXl9o/s1600/jagged.PNG" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Original (zoomed)</td></tr>
</tbody></table>
<br />
<u>3. Adding anti-aliasing support</u><br />
<br />
I also have two different approaches to solve this problems:<br />
<br />
First approach:<br />
<ul>
<li>When drawing the parallel lines use a different algorithm to draw the ones on both edges with a line algorithm that supports anti-aliasing, such as <a href="https://en.wikipedia.org/wiki/Xiaolin_Wu%27s_line_algorithm">Xiaolin Wu's</a></li>
</ul>
<br />
Second approach:<br />
<ul>
<li>Draw everything on an higher resolution and then scale the image down. </li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxoR9aLPeqei2rZzQGr4eHiWfNwk-mxsT8WWPw5dysyuQCgpvKeZKuFTIM-e9fUO3_qmfXQcAPxOl2YvIel8sqJbWaMvFrlH8k2o6bRy7zdz-exKrVSO1t-1dnR2-M53J1Exb1qlpKnxA/s1600/jagged.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a> <br />
Went with approach two (scale down) and this is the end-result:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoLxbeBJifFMsUTbDwomjVf7NrDhkhdBjn77Z3JqVNhoo1e_ZEmv5V0tl6SUU_vQd433HdN9RPgpsQOtIIWp4WEz2CDHJB_PDMpc2AFQFYAwlrIXLQn6vjSDQLP06YthBUjFHjS1y9a-g/s1600/star_lines_bold_antialiasing.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoLxbeBJifFMsUTbDwomjVf7NrDhkhdBjn77Z3JqVNhoo1e_ZEmv5V0tl6SUU_vQd433HdN9RPgpsQOtIIWp4WEz2CDHJB_PDMpc2AFQFYAwlrIXLQn6vjSDQLP06YthBUjFHjS1y9a-g/s1600/star_lines_bold_antialiasing.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Downscalling (zoomed)</td></tr>
</tbody></table>
I was pretty happy with the result. Unfortunately from a performance point-of-view it's far from ideal and the tile rendering time more than doubled.<br />
<br />
So, back to the drawing board (ah!) to try the other approach: supporting anti-aliasing on the line drawing algorithm itself, using <a href="https://en.wikipedia.org/wiki/Xiaolin_Wu%27s_line_algorithm">Xiaolin Wu's</a> line algorithm:<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7mcbsw-_cDrNBeI3IbuXLBC4tHMFpxE9qQJOg8K8IE4dbPw9jpHT0H_EzNmjhF0fkImzduCS3N-Mzm5g97uq2nY8G55jjJmynW58OWfMnSIKLLGna4BtWe5FF8cIgIqIwkNPwVO30_Ic/s1600/star_lines_xiaolinWu_without.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7mcbsw-_cDrNBeI3IbuXLBC4tHMFpxE9qQJOg8K8IE4dbPw9jpHT0H_EzNmjhF0fkImzduCS3N-Mzm5g97uq2nY8G55jjJmynW58OWfMnSIKLLGna4BtWe5FF8cIgIqIwkNPwVO30_Ic/s200/star_lines_xiaolinWu_without.png" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Bresenham's line (zoomed)</td></tr>
</tbody></table>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDEVWJqf0yg_nBhGUGY79Lq8GnVPlnP-YmutEvcsk8dhUBBNWbQ_oe3TECHprfNaXt15Jm6vCsRBx1L5oIV-CgBJ-_s7Es31pzI41Wy7SWCK4P3wazxZaoGcnkrUXYkwmPAtm13P1nZiM/s1600/star_lines_xiaolinWu_with.png" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="199" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDEVWJqf0yg_nBhGUGY79Lq8GnVPlnP-YmutEvcsk8dhUBBNWbQ_oe3TECHprfNaXt15Jm6vCsRBx1L5oIV-CgBJ-_s7Es31pzI41Wy7SWCK4P3wazxZaoGcnkrUXYkwmPAtm13P1nZiM/s200/star_lines_xiaolinWu_with.png" width="200" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Xiaolin Wu's line (zoomed)</td></tr>
</tbody></table>
<br />
The implementation is slightly more complex than Bresenham, but its performance is pretty reasonable.<br />
<br />
For the full implementation please check my github repo.<br />
<br />
<br />
<br />
<br />
On the top images you can see how it compares with Bresenham algorithm, providing a subtle anti-aliasing.<br />
<br />
So, to support anti-aliased lines with variable widths I'm going to:<br />
<ul>
<li>Use the first strategy I've shown do draw multiple parallel lines to achieve a thick line</li>
<li>Drawing the lines furthest away from the main line using Xiaolin Wu's line algorithm</li>
<li>Drawing all the other lines with Bresenham's line algorithm</li>
</ul>
The end-result is (zooming in):<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt49tTGi-oHz3GQgeb7FvzfiuUNzdgT4SvB_n-fWnFx7UdBMXq9bhyphenhyphenncEpZUM7fbujXRgZN6SJHloS3DNGGpJacZCoRArJzlMGavW9Y7OlA_iITQYTTooVS0y4KwSm-X4Or6s0JOIxvbs/s1600/star_lines_bold_antialiasing_xiaolinWu_zoomed.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjt49tTGi-oHz3GQgeb7FvzfiuUNzdgT4SvB_n-fWnFx7UdBMXq9bhyphenhyphenncEpZUM7fbujXRgZN6SJHloS3DNGGpJacZCoRArJzlMGavW9Y7OlA_iITQYTTooVS0y4KwSm-X4Or6s0JOIxvbs/s1600/star_lines_bold_antialiasing_xiaolinWu_zoomed.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mixing Bresenham and Xiaolin Wu (zoomed)</td></tr>
</tbody></table>
The anti-aliasing is not as strong as the downscaled version but its performance is much better. Also, on a non-zoomed view the aliasing is mostly negligible:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRJB8-8TLS2SIwAJ8oa-lwr0jRne7hXb-FC0bDIh4WOprHyeVMnpp7LBNAKNUUopkQovBnWXisiEEGNwXytbtl9LsEaxiwzBw_j3FSwHGc7fG0jZEA3JosnkKSR6jZma4B_wLgWpeuOLk/s1600/star_lines_bold_antialiasing_xiaolinWu.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRJB8-8TLS2SIwAJ8oa-lwr0jRne7hXb-FC0bDIh4WOprHyeVMnpp7LBNAKNUUopkQovBnWXisiEEGNwXytbtl9LsEaxiwzBw_j3FSwHGc7fG0jZEA3JosnkKSR6jZma4B_wLgWpeuOLk/s1600/star_lines_bold_antialiasing_xiaolinWu.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Mixing Bresenham and Xiaolin Wu (original size)</td><td class="tr-caption" style="text-align: center;"></td></tr>
</tbody></table>
<u><br /></u>
Also, in order to properly support transparent pixels I had to include a proper color blending routine. Otherwise, when setting a semi-transparent pixel on top of an existing color it would replace it altogether.<br />
<br />
Thus, in these cases I've just added the following code:
<br />
<pre class="prettyprint">public static Color BlendColor(Color bg, Color fg)
{
var r = new Color();
r.A = 1 - (1 - fg.A) * (1 - bg.A);
if (r.A < 1.0e-6) return r; // Fully transparent -- R,G,B not important
r.R = fg.R * fg.A / r.A + bg.R * bg.A * (1 - fg.A) / r.A;
r.G = fg.G * fg.A / r.A + bg.G * bg.A * (1 - fg.A) / r.A;
r.B = fg.B * fg.A / r.A + bg.B * bg.A * (1 - fg.A) / r.A;
return r;
}
</pre>
<br />
This tile gives an actual funky effect on top of a map<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI52B2a1b6o1JgXES14LJSOzJHyVu9Sxu9a1JnwWtrHm4xJYFAR0gpQ1hEf3dQRjVjm-UuldxNqxS2tOgFHTAfvOeTS3k34LPjk0shZZ3WRmTfQZxYDAAXBQk2BjHXR6b4PauBpWIVT2o/s1600/pattern_tile.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="468" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhI52B2a1b6o1JgXES14LJSOzJHyVu9Sxu9a1JnwWtrHm4xJYFAR0gpQ1hEf3dQRjVjm-UuldxNqxS2tOgFHTAfvOeTS3k34LPjk0shZZ3WRmTfQZxYDAAXBQk2BjHXR6b4PauBpWIVT2o/s640/pattern_tile.PNG" width="640" /></a></div>
<br />
<u>4. Drawing hexagons</u><br />
<br />
Now that I've got a reasonable toolset I'm able to create the hexagon tiles layer to overlay on the map. I'm not going to post all the relevant code here so feel free the take a look at the source-code on the corresponding github repo.<br />
<br />
The end-result is far from astonishing but shows the effect I was going for.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlLmfzWXhHlC6yplV-RIOLRwknvg_eMnobjxyhECpfHF-cwgnqBJfIr17ZtNbWZYrb6wJdI4m3QrP-rr2V1NDkLfYbCOlvchrkQLRLXMTnEqS7-9ta-sT9HM-M_xjBj__JVAbNIeLJ_V0/s1600/hexagon1.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlLmfzWXhHlC6yplV-RIOLRwknvg_eMnobjxyhECpfHF-cwgnqBJfIr17ZtNbWZYrb6wJdI4m3QrP-rr2V1NDkLfYbCOlvchrkQLRLXMTnEqS7-9ta-sT9HM-M_xjBj__JVAbNIeLJ_V0/s1600/hexagon1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="331" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlLmfzWXhHlC6yplV-RIOLRwknvg_eMnobjxyhECpfHF-cwgnqBJfIr17ZtNbWZYrb6wJdI4m3QrP-rr2V1NDkLfYbCOlvchrkQLRLXMTnEqS7-9ta-sT9HM-M_xjBj__JVAbNIeLJ_V0/s400/hexagon1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">zoom level 6</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEje484ArIZt6JYHjwy2GsyQEFqGUXXx_vFaRoJtH9sYwGWe96jMDZyysj62b19AnVjGhjq1nM06MhN7x8pV3hcQscG_Wu9ynnAVf5bKlqcDxqiUwLQBYYYt3C_bcxmkWTs2u3kYFOu9sIc/s1600/hexagon2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="332" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEje484ArIZt6JYHjwy2GsyQEFqGUXXx_vFaRoJtH9sYwGWe96jMDZyysj62b19AnVjGhjq1nM06MhN7x8pV3hcQscG_Wu9ynnAVf5bKlqcDxqiUwLQBYYYt3C_bcxmkWTs2u3kYFOu9sIc/s400/hexagon2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">zoom level 8</td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: left;">
This was a fun experiment but I expect it to get obsolete in the short-term, particularly as soon as Microsoft includes proper drawing methods in System.Drawing. Thus I'm not planning in taking it any further. Regardless, I might reuse some of these bits on other stuff.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<span style="background-color: #fefdfa; color: #333333; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18.2px;">The complete code can be viewed at: </span><a href="https://github.com/pmcxs/CoreTiles" style="background-color: #fefdfa; color: #d52a33; font-family: Arial, Tahoma, Helvetica, FreeSans, sans-serif; font-size: 13px; line-height: 18.2px; text-decoration: none;">https://github.com/pmcxs/CoreTiles</a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com9tag:blogger.com,1999:blog-6029629494508626591.post-31795361110676835602016-06-10T23:02:00.001+01:002016-08-02T21:30:35.288+01:00Creating a simple TileServer with .NET Core RC2 (Part 1 - Setting up the project)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;">Part 1. Setting up the the project</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2016/08/creating-simple-tileserver-with-net.html">Part 2. Improving drawing logic</a></span></div>
<br />
As most of you might have heard Microsoft has recently released .Net core RC2. Although it's still subject to lots of changes I think now is a good time to get on the bandwagon, particularly with the various improvements that have been done.<br />
<div>
<br /></div>
<div>
On this series I'm going to do a cross-platform tile-server that generates tiles with hexagons displayable on a map. As I'm just learning .NET core this will be a learning exercise for me and I'll post all of the steps that I've done in order to achieve the end-result.<br />
<a name='more'></a></div>
<div>
<br />
<u>1. First step, install .NET core and Visual Studio</u></div>
<div>
<br /></div>
<div>
The instructions on Microsoft's page are actually quite good. I've tried them both on Windows and Mac and worked like a charm. Just don't omit any of the steps (like uninstalling previous versions):</div>
<div>
<a href="https://www.microsoft.com/net/core">https://www.microsoft.com/net/core</a></div>
<div>
<br /></div>
<div>
This page also explains how to setup Visual Studio 2015 and Visual Studio Code. I've actually setup both and I'm forcing myself to also use Visual Studio Code as an exercise to properly understand how the plumbing and client tools work.</div>
<div>
<br /></div>
<div>
<u>2. Create the project structure</u></div>
<div>
<br /></div>
<div>
My project will be structured as:</div>
<div>
<ul>
<li>A class library to hold the drawing logic</li>
<li>A class library to hold the hexagon logic</li>
<li>A unit test project to validate the drawing logic</li>
<li>A unit test project to validate the hexagon logic</li>
<li>A server that will generate images for specific tile coordinates</li>
</ul>
</div>
<div>
You may ask: "Do you need so many projects for such a simple project?". Definitely not, although useful for a learning exercise.<br />
<br />
The convention for .net core projects is to create a folder for the main projects (under /src) and a folder for the corresponding tests (under /test). Also, a global.json is defined at the top to specify these two main folders. The project structure will be:<br />
<br /></div>
<pre style="border: black solid 1px; padding: 3px;">/CoreTiles
|__global.json
|__/src
|__/CoreTiles.Drawing
|__<files>
|__project.json
|__/CoreTiles.Hexagon
|__<files>
|__project.json
|__/CoresTiles.Server
|__<files>
|__project.json
|__/test
|__/CoreTiles.Drawing.Tests
|__<files>
|__project.json
|__/CoreTiles.Hexagon.Tests
|__<files>
|__project.json
</pre>
I've started by creating the tree structure in the file-system. Then, inside each project folder creating an empty project using the dotnet cli tool:
<br />
<pre style="border: black solid 1px; padding: 3px;">dotnet new</pre>
This creates a console application that ouputs "Hello World", containing both a "project.json" and a "Program.cs" files. A couple of changes need to be done to the default generated projects:<br />
<ul>
<li>The class libraries (CoreTiles.Drawing and CoreTiles.Hexagon) are not executable, so the project.json should be changed. Basically specifying that this project doesn't have an entry execution point and making it more compatible across the board, as "netstandard" can be implemented by multiple .NET platforms. Additional info <a href="https://github.com/dotnet/corefx/blob/master/Documentation/architecture/net-platform-standard.md">here</a>. </li>
</ul>
<div>
From:</div>
<pre style="border: black solid 1px; padding: 3px;">{
"version": "1.0.0-*",
<span style="color: red;">"buildOptions": {
"emitEntryPoint": true
},</span>
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.0-rc2-3002702"
}
},
"frameworks": {
"netcoreapp1.0": {
"imports": "dnxcore50"
}
}
}
</pre>
To:
<br />
<pre style="border: black solid 1px; padding: 3px;">{
"version": "1.0.0-*",
"dependencies": {
<span style="color: #6aa84f;">"NETStandard.Library": "1.5.0-rc2-24027" </span>
},
"frameworks": {
"<span style="color: #6aa84f;">netstandard1.5</span>": {
"imports": "dnxcore50"
}
}
}</pre>
<ul>
<li>The test projects are slightly different, as they're actually executables. Regardless, the Main method is provided by the test runner, which in this case is xunit. So, the entry point should be removed and the xunit dependencies should be added:</li>
</ul>
From:
<br />
<pre style="border: black solid 1px; padding: 3px;">{
"version": "1.0.0-*",
"buildOptions": {
"emitEntryPoint": true
},
"dependencies": {
"Microsoft.NETCore.App": {
"type": "platform",
"version": "1.0.0-rc2-3002702"
}
},
"frameworks": {
"netcoreapp1.0": {
"imports": "dnxcore50"
}
}
}
</pre>
To:
<br />
<pre style="border: black solid 1px; padding: 3px;">{
"version": "1.0.0-*",
<span style="color: #6aa84f;"> "testRunner": "xunit",</span>
"dependencies": {
"Microsoft.NETCore.App": {
"type":"platform",
"version": "1.0.0-rc2-3002702"
},
<span style="color: #6aa84f;"> "xunit":"2.1.0",
"dotnet-test-xunit": "1.0.0-rc2-build10015"</span>
},
"frameworks": {
"netcoreapp1.0": {
"imports": [
"dnxcore50",
<span style="color: #6aa84f;">"portable-net45+win8"</span>
]
}
}
}</pre>
<ul>
<li>The server project (CoreTiles.Server) is an actual console application, so doesn't need to be changed (for now)</li>
</ul>
<div>
<u>3. Create the Server Web API </u></div>
<div>
<br /></div>
<div>
Start by adding the various dependencies to project.json
<br />
<pre style="border: black solid 1px; padding: 3px;">"dependencies": {
"Microsoft.NETCore.App": {
"version": "1.0.0-rc2-3002702",
"type": "platform"
},
"Microsoft.AspNetCore.Mvc": "1.0.0-rc2-final",
"Microsoft.AspNetCore.Server.Kestrel": "1.0.0-rc2-final"
}
</pre>
On RC1 a MVC project did the wiring up of the Startup class automatically. On RC2 this is a regular console app. As such, all the plumbing needs to be done explicitly on Main:</div>
<pre style="border: black solid 1px; padding: 3px;">public class Program
{
public static void Main(string[] args)
{
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseStartup<Startup>()
.Build();
host.Run();
}
}
</pre>
We need to register the MVC framework on the pipeline
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">public void ConfigureServices(IServiceCollection services)
{
// Add framework services.
services.AddMvc();
}
// This method gets called by the runtime.
// Use this method to configure the HTTP request pipeline.
public void Configure(
IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
app.UseMvc();
}
</pre>
Now creating a controller that will return a tile images, depending on specified parameters (z/x/y).<br />
<br />
I've created a simple TileController with this signature:
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">[Route("[controller]")]
public class TileController : Controller
{
[HttpGet("{z}/{x}/{y}")]
public IActionResult Get(int z, int x, int y)
{
return Ok();
}
}
</pre>
<br />
Now I'm just missing:<br />
<ul>
<li>Returning an actual image</li>
<li>Creating a map view that uses this API </li>
</ul>
The map image being returned needs to be generated dynamically. Luckily there's a component that<br />
does exactly that, including support for .NET Core: <a href="https://github.com/JimBobSquarePants/ImageProcessor">ImageProcessor.</a><br />
Unfortunately ImageProcessor doesn't yet include drawing capabilities so I'll need to extend it to include some simple drawing actions (like drawing lines, rectangles, polylines, etc).<br />
<br />
Thus, for this first post I'll just return a simple red square for each tile. I'll do the proper hexagon drawing on the next post, on which I'll do some extensions to ImageProcessor.
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">[Route("[controller]")]
public class TileController : Controller
{
private const int TileSize = 256;
[HttpGet("{z}/{x}/{y}")]
public async Task<IActionResult> Get(int z, int x, int y)
{
using (Image image = new Image(TileSize, TileSize))
using (var outputStream = new MemoryStream())
{
//Drawing code goes here
image.SaveAsPng(outputStream);
var bytes = outputStream.ToArray();
Response.ContentType = "image/png";
await Response.Body.WriteAsync(bytes, 0, bytes.Length);
return Ok();
}
}
}
</pre>
As the Image class of ImageProcessor only supports putting and getting individual pixels creating a square method is quite simple:<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">public static void DrawRectangle(this Image image, int x, int y,
int width, int height, Color color)
{
//Draw horizontal lines
for(var i = x; i < x + width; i++)
{
image.SetPixel(i, y, color);
image.SetPixel(i, y + height - 1, color);
}
//Draw vertical lines
for(var j=y+1; j < y + height; j++)
{
image.SetPixel(x, j, color);
image.SetPixel(x + width - 1, j, color);
}
}
</pre>
Updated the Tile drawing logic to:
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">[Route("[controller]")]
public class TileController : Controller
{
private const int TileSize = 256;
[HttpGet("{z}/{x}/{y}")]
public async Task<IActionResult> Get(int z, int x, int y)
{
using (Image image = new Image(TileSize, TileSize))
using (var outputStream = new MemoryStream())
{
<b><span style="color: #990000;">image.DrawRectangle(0,0,256,256,Color.Red);</span></b>
image.SaveAsPng(outputStream);
var bytes = outputStream.ToArray();
Response.ContentType = "image/png";
await Response.Body.WriteAsync(bytes, 0, bytes.Length);
return Ok();
}
}
}
</pre>
Running the CoreTile.Server app (dotnet run) and opening a browser at: http://localhost:5000/tile/0/0/0 shows this:<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht1JA2OWYHdH710bhCr3hs3Qqej-qAN7IRr-P0E_O7QYubp6MikFlQtFwoquNl9qYOzQmDOuO01TtGZ48Xt2_8yhJPV9HEDYsPfvRMiCP7cKJOFdWmATT-LZp14Y7wV_dviBihOTMzqag/s1600/square.PNG" imageanchor="1"><img border="0" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht1JA2OWYHdH710bhCr3hs3Qqej-qAN7IRr-P0E_O7QYubp6MikFlQtFwoquNl9qYOzQmDOuO01TtGZ48Xt2_8yhJPV9HEDYsPfvRMiCP7cKJOFdWmATT-LZp14Y7wV_dviBihOTMzqag/s320/square.PNG" width="320" /></a><br />
Now I'll create a web-page to show a simple map using this new tile-layer.<br />
<br />
I'm going to use Openlayers 3 for this, particularly as I've been wanting to experiment with it for some time.<br />
<br />
First I need to change the Startup class to define a proper default routing logic as well as being able to serve static files from the server.
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">public void Configure(IApplicationBuilder app)
{
app.UseStaticFiles();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Home}/{action=Index}");
});
}
}
</pre>
Then creating a simple controller to serve the main View
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;">public class HomeController : Controller
{
public IActionResult Index()
{
return View();
}
}
</pre>
And finally the actual view
<br />
<pre class="prettyprint" style="border: black solid 1px; padding: 3px;"><!DOCTYPE html>
<html>
<head>
<title>Canvas Tiles</title>
<link rel="stylesheet" href="http://openlayers.org/en/v3.16.0/css/ol.css"
type="text/css">
<script src="http://openlayers.org/en/v3.16.0/build/ol.js">
</script>
</head>
<body>
<div id="map" class="map"></div>
<script>
var osmSource = new ol.source.OSM();
var map = new ol.Map({
layers: [
new ol.layer.Tile({
source: osmSource
}),
new ol.layer.Tile({
source: new ol.source.XYZ({
url: "/tile/{z}/{x}/{y}"
})
}),
],
target: 'map',
controls: ol.control.defaults({
attributionOptions: ({
collapsible: false
})
}),
view: new ol.View({
center: ol.proj.transform(
[-0.1275, 51.507222], 'EPSG:4326', 'EPSG:3857'),
zoom: 10
})
});
</script>
</body>
</html>
</pre>
<br />
Running the application and opening http://localhost:5000 shows the basemap with the dynamically generated tiles<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTTO1cW0UZ747dCYf2tGdibBKxx6mx9Bm_SlFeHNKtQtZl-TbB-vbQsuwUqnhomerdX-8tT1fSyMDjquWiwRefrCBpceuFcLmZPkm-fNrG3fXj2kklwbtHEmtjZY6BHgCL-IgWbyf_AN4/s1600/simple_tile.PNG" imageanchor="1"><img border="0" height="326" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTTO1cW0UZ747dCYf2tGdibBKxx6mx9Bm_SlFeHNKtQtZl-TbB-vbQsuwUqnhomerdX-8tT1fSyMDjquWiwRefrCBpceuFcLmZPkm-fNrG3fXj2kklwbtHEmtjZY6BHgCL-IgWbyf_AN4/s400/simple_tile.PNG" width="400" /></a><br />
<br />
I've tried this both on a Mac and a Windows. I'm assuming it should also work without any problem in Linux.<br />
<br />
The complete code can be viewed at: <a href="https://github.com/pmcxs/CoreTiles">https://github.com/pmcxs/CoreTiles</a><br />
<br />
Reference:<br />
<ul>
<li><a href="https://github.com/dotnet/core-docs/blob/master/docs/core-concepts/testing/unit-testing-with-dotnet-test.md">https://github.com/dotnet/core-docs/blob/master/docs/core-concepts/testing/unit-testing-with-dotnet-test.md</a></li>
<li><a href="http://martinwilley.com/blog/2016/05/22/DotNetCoreClassLibraries.aspx">http://martinwilley.com/blog/2016/05/22/DotNetCoreClassLibraries.aspx</a></li>
<li><a href="http://dotnet.github.io/docs/core-concepts/libraries/libraries-with-cli.html">http://dotnet.github.io/docs/core-concepts/libraries/libraries-with-cli.html</a></li>
<li><a href="https://github.com/dotnet/core-docs/tree/master/samples">https://github.com/dotnet/core-docs/tree/master/samples</a></li>
</ul>
<div>
<br /></div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com3tag:blogger.com,1999:blog-6029629494508626591.post-65166329833814404712015-06-14T04:21:00.002+01:002015-06-14T04:21:45.787+01:00Game-development Log (14. CDN Improvements)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/11/game-development-log-11-persistency-and.html">11. Persistency and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2015/01/game-development-log-12-scaling-up.html">12. Scalling up the loader process</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html">13. Polishing the experience</a></span><br />
<span style="font-size: x-small;">14. CDN Improvements</span></div>
<br />
I haven't been really active on the development of this project as I've been occupied with other stuff. Regardless, I'm now focused on returning to active development on it.<br />
<br />
Interestingly enough what actually triggered my return was a couple of Azure announcements last week on the CDN front. First, some context:<br />
<br />
Although I like Azure quite a lot its CDN offering has been quite lacking to say the least. The community was quite vocal in requesting some essential features but Microsoft neglected to provide any updates or expected delivery dates, as seen here:
<a href="http://feedback.azure.com/forums/169397-cdn">http://feedback.azure.com/forums/169397-cdn</a><br />
<br />
For example, the top voted feature request was the ability to force content to be refreshed, which is, IMHO, completely essential for a CDN offering:<br />
<br />
<a href="http://feedback.azure.com/forums/169397-cdn/suggestions/556307-ability-to-force-the-cdn-to-refresh-any-cached-con">http://feedback.azure.com/forums/169397-cdn/suggestions/556307-ability-to-force-the-cdn-to-refresh-any-cached-con</a><br />
<br />
The feature was requested 5 years ago, eventually marked as "planned", and no further update was provided by Microsoft, similarly to the other requested features.<br />
<br />
Well, all of this until last week, when Microsoft apparently woke up.<br />
<br />
First they've provided feedback on most of the feature requests and provided an expectation around release dates. Not ideal (as most of these features are late by a few years) but positive nevertheless.<br />
<br />
Then, the icing on the top of the cake was this post:<br />
<br />
<a href="https://azure.microsoft.com/blog/2015/06/04/announcing-custom-origin-support-for-azure-cdn/">https://azure.microsoft.com/blog/2015/06/04/announcing-custom-origin-support-for-azure-cdn/</a><br />
<br />
Basically Microsoft shipped three awesome features for the CDN:<br />
<br />
I'll just copy&paste from that post:<br />
<blockquote style="background-color: white; box-sizing: border-box; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 0.65rem; padding: 0px; text-rendering: optimizeLegibility;">
<b>Custom Origins Supported</b><br />
Azure CDN can now be used with any origin. Previously, Azure CDN only supported a limited set of Azure Services (i.e. Web Apps, Storage, Cloud Services and Media Services) and you only had the ability to create a CDN endpoint for an Azure Service that was in your Azure Subscription. With this recent update, you can now create a CDN endpoint for any origin you like. This includes the ability to create an origin in your own data center, an origin provided by third party cloud providers, etc. and gives you the flexibility to use any origin you like with Azure CDN!</blockquote>
<blockquote style="background-color: white; box-sizing: border-box; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 0.65rem; padding: 0px; text-rendering: optimizeLegibility;">
<b>Multiple CDN Endpoints with the Same Origin</b><br />
Several of you may have tried to create multiple CDN endpoints for the same origin and found this wasn’t possible due to restrictions. We have now removed the restrictions and you now have the ability to create multiple endpoints for the same origin URL. This provides you a more refined control over content management and can be used to improve performance as multiple host names can be used to access assets from the same origin.</blockquote>
<blockquote style="background-color: white; box-sizing: border-box; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 0.65rem; padding: 0px; text-rendering: optimizeLegibility;">
<b>Save Content in any Origin Folder</b><br />
Previously, when you created a CDN endpoint for cloud services you were required to use “/cdn/” as the default origin path. For example, if the path for your cloud service was<a href="http://strasbourg.cloudapp.net/" style="background: transparent; box-sizing: border-box; color: #00abec; line-height: inherit; text-decoration: none;"><span style="box-sizing: border-box; color: #0563c1; font-family: Calibri;">http://strasbourg.cloudapp.net</span></a> you were required to use <a href="http://strasbourg.cloudapp.net/cdn/" style="background: transparent; box-sizing: border-box; color: #00abec; line-height: inherit; text-decoration: none;"><span style="box-sizing: border-box; color: #0563c1; font-family: Calibri;">http://strasbourg.cloudapp.net/cdn/</span></a> as the root path to get content from your origin when you created a CDN endpoint. This restriction has been removed and you can store content in any folder. Using the previous example, you can now use <a href="http://strasbourg.cloudapp.net/" style="background: transparent; box-sizing: border-box; color: #00abec; line-height: inherit; text-decoration: none;"><span style="box-sizing: border-box; color: #0563c1; font-family: Calibri;">http://strasbourg.cloudapp.net/</span></a>as the root path to get content from your origin.</blockquote>
These might seem minor changes but let me explain how they positively affect this project:<br />
<br />
<a name='more'></a><br />
<b>Multiple CDN Endpoints with the Same Origin</b><br />
<b><br /></b>
Completely related with my blog-post on improving tile-loading at the browser (<a href="http://build-failed.blogspot.pt/2015/03/improve-tile-loading-at-browser.html">http://build-failed.blogspot.pt/2015/03/improve-tile-loading-at-browser.html</a>), a really simple performance improvement relies on having multiple urls for the map-tiles.<br />
<br />
This technique is called domain sharding. Different domains are used to fetch the same information, thus bypassing the "same-domain" browser limitation. This limitation is implemented differently on the various browsers, but all include a hard-limit on the number of requests that can be done concurrently to the same url. With this approach I'm basically quadrupling this number.<br />
<br />
I've created 4 different CDN urls:<br />
<br />
<ul>
<li>http://az710822.vo.msecnd.net/imagetiles/{quadkey}.jpg </li>
<li>http://az768596.vo.msecnd.net/imagetiles/{quadkey}.jpg</li>
<li>http://az769152.vo.msecnd.net/imagetiles/{quadkey}.jpg</li>
<li>http://az769848.vo.msecnd.net/imagetiles/{quadkey}.jpg</li>
</ul>
<br />
All of them are pointing to the same blob storage containg all the tiles. The difference is that, on client-side, the url used to request each tile is dependent on the last digit of the quadkey that identifies the tile.<br />
<br />
Thus:<br />
<ul>
<li>http://az710822.vo.msecnd.net/imagetiles/{quadkey}.jpg (quadkeys ending in 0)</li>
<li>http://az768596.vo.msecnd.net/imagetiles/{quadkey}.jpg (quadkeys ending in 1)</li>
<li>http://az769152.vo.msecnd.net/imagetiles/{quadkey}.jpg (quadkeys ending in 2)</li>
<li>http://az769848.vo.msecnd.net/imagetiles/{quadkey}.jpg (quadkeys ending in 3)</li>
</ul>
Example:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZzAXsjZICynfDgH8lwg7hXk2nr6MVBY4Wfz3PiemVao004TlMj2HyNx45b0MTZc8m5ZVWSx8jhavyFLB4O8BPY7QZyads-zjytkwVdYKD_1tdQ4r8D4XPCWt_RcINmyNNV75uw_hFTKw/s1600/88.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZzAXsjZICynfDgH8lwg7hXk2nr6MVBY4Wfz3PiemVao004TlMj2HyNx45b0MTZc8m5ZVWSx8jhavyFLB4O8BPY7QZyads-zjytkwVdYKD_1tdQ4r8D4XPCWt_RcINmyNNV75uw_hFTKw/s1600/88.png" /></a></div>
<br />
<ul>
<li><a href="http://az769152.vo.msecnd.net/imagetiles/0311332.jpg">http://az769152.vo.msecnd.net/imagetiles/0311332.jpg</a></li>
<li><a href="http://az769848.vo.msecnd.net/imagetiles/0311333.jpg">http://az769848.vo.msecnd.net/imagetiles/0311333.jpg</a></li>
<li><a href="http://az710822.vo.msecnd.net/imagetiles/0313110.jpg">http://az710822.vo.msecnd.net/imagetiles/0313110.jpg</a></li>
<li><a href="http://az768596.vo.msecnd.net/imagetiles/0313111.jpg">http://az768596.vo.msecnd.net/imagetiles/0313111.jpg</a></li>
</ul>
<br />
<div>
Notice the different urls for each tile.</div>
<div>
<br /></div>
The interesting part is that, cost-wise, this approach doesn't affect my Azure bill as the set of cached tiles for each CDN has no duplicates, thus not requiring additional disk space.<br />
<br />
<b>Save Content in any Origin Folder</b><br />
<br />
Serving custom tiles is a challenge, particularly due to the ridiculous amount of tiles required to serve higher zoom levels. The math is simple: for each zoom level you need the following number of tiles:<br />
4^zoom level<br />
<br />
zoom 0 = 4^0 => 1 tile<br />
zoom 1 = 4^1 => 4 tiles<br />
zoom 2 = 4^2 => 16 tiles<br />
...<br />
zoom 20 = 4^20 => 1.099.511.627.776 tiles (yes, we're talking about trillions here).<br />
<br />
This is challenging both in terms of disk space and time spent to generate the tiles. So, a common approach is to pre-render lower zoom levels (ex: 0-10) and serve the higher zoom level dynamically on demand (also, its also quite desirable to cache these while serving them).<br />
<br />
For this project I'm pre-generating the tiles up to the zoom level 12 (which is still a reasonable number) and dynamically generating higher zoom levels.<br />
<br />
Initially I did setup a CDN to the service that is dynamically generating the tiles but it was quite limiting as it only worked with Cloud-Services, required the /cdn suffix and didn't work well with the routing I had, requiring me to create a custom route with query-string parameters which, although supported by the CDN, wasn't working properly. So, eventually I gave up and was serving the dynamic tiles directly from my webapi hosted directly from a WebProject.<br />
<br />
Example:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggw_Aa1-nGXrqK3N8vgCEB29FuJlUto-fAYyjhIMuCMS1VfA8WBbZl7VRhxNFPvxVnY7NNNTjFT9nKi2A-OOaaotZpRvnNd46tiDy3Da94ntM8xFoZcII_EUjXMKkJLImUnnwCl2HbdLE/s1600/79.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggw_Aa1-nGXrqK3N8vgCEB29FuJlUto-fAYyjhIMuCMS1VfA8WBbZl7VRhxNFPvxVnY7NNNTjFT9nKi2A-OOaaotZpRvnNd46tiDy3Da94ntM8xFoZcII_EUjXMKkJLImUnnwCl2HbdLE/s1600/79.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><a href="http://tile-win.azurewebsites.net/13/3923/3085.jpg" style="font-size: medium; text-align: start;">http://tile-win.azurewebsites.net/13/3923/3085.jpg</a></td></tr>
</tbody></table>
<br />
With the new Azure feature I can now point a CDN to my existing webapi. Thus I can obtain the same image from the CDN.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggw_Aa1-nGXrqK3N8vgCEB29FuJlUto-fAYyjhIMuCMS1VfA8WBbZl7VRhxNFPvxVnY7NNNTjFT9nKi2A-OOaaotZpRvnNd46tiDy3Da94ntM8xFoZcII_EUjXMKkJLImUnnwCl2HbdLE/s1600/79.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggw_Aa1-nGXrqK3N8vgCEB29FuJlUto-fAYyjhIMuCMS1VfA8WBbZl7VRhxNFPvxVnY7NNNTjFT9nKi2A-OOaaotZpRvnNd46tiDy3Da94ntM8xFoZcII_EUjXMKkJLImUnnwCl2HbdLE/s1600/79.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8000001907349px;"><a href="http://tile-win.azurewebsites.net/13/3923/3085.jpg" style="font-size: medium; text-align: start;">http://az768968.vo.msecnd.net/13/3923/3085.jpg</a></td></tr>
</tbody></table>
So, what's the performance comparison?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwWTTr38PtQuZ5EPXG-HTFT5-MSaRvBFzvAf9gFmBeYW_Az_bzP8lsPIvr1j22zjarJmXqh2Q4vG4UQMRS0VDsbH25qZGLp-Qkvo2BXfW1aQTceVDaMsZ7JYxSB6EfwgMbnJt08X_3UMs/s1600/89.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwWTTr38PtQuZ5EPXG-HTFT5-MSaRvBFzvAf9gFmBeYW_Az_bzP8lsPIvr1j22zjarJmXqh2Q4vG4UQMRS0VDsbH25qZGLp-Qkvo2BXfW1aQTceVDaMsZ7JYxSB6EfwgMbnJt08X_3UMs/s1600/89.png" /></a></div>
<br />
<br />
<b>Also, not CDN related</b><br />
<br />
On this update I also:<br />
<br />
<ul>
<li>Completely refactored my loading logic. Now I have separate projects for each step, namely:</li>
<ul>
<li>Converting Geographical data to Vector Tiles</li>
<li>Generating Image Tiles from Vector Tiles</li>
<li>Pushing Tiles to Azure Blob Storage </li>
</ul>
</ul>
<div>
This change was really important as it allows me to streamline the creation and publication of the tiles going forward.</div>
<div>
<ul>
<li>Fixed one of the most tricky bugs I had on the loading logic.</li>
</ul>
<div>
I had this problem:</div>
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5AeDLRmqbgIrI9hSh2qOlJelzuBmcd_FicDhDm4Jko-50AM0CgnndeOHuJdhLmn_73Eq7VeHoidRkw9FljwNBQkXPyt8ssMm16cHq3WPlk8hMYdPaJyjQQyPdBkp5xdwCmMh_yYOxOao/s1600/86.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5AeDLRmqbgIrI9hSh2qOlJelzuBmcd_FicDhDm4Jko-50AM0CgnndeOHuJdhLmn_73Eq7VeHoidRkw9FljwNBQkXPyt8ssMm16cHq3WPlk8hMYdPaJyjQQyPdBkp5xdwCmMh_yYOxOao/s1600/86.PNG" /></a></div>
<div>
<br /></div>
<div>
As I explained on one of the posts in this series, I split the geographical data into manageable chunks. When processing the individual tiles some artifacts would be visible on the seams between those chunks.</div>
<div>
<br /></div>
<div>
I detected this problem a long time ago but I didn't have an immediate solution for it. With the loader refactor I managed to find an elegant solution for the problem. The same area now appears as:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEib9u2WlfTsZGRFQsXAFOc4d8z7AUZM6PuItFqX5LyRkDVdxtd0LmQmbaIXAYS9NzWMg6nVEPVk6vIRe4ty6PG0ClhuCL8aeIt-c9HYarn7qkFve-9ys-ceJwnl8QiGXyVGCbC0lNbue5Y/s1600/87.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEib9u2WlfTsZGRFQsXAFOc4d8z7AUZM6PuItFqX5LyRkDVdxtd0LmQmbaIXAYS9NzWMg6nVEPVk6vIRe4ty6PG0ClhuCL8aeIt-c9HYarn7qkFve-9ys-ceJwnl8QiGXyVGCbC0lNbue5Y/s1600/87.PNG" /></a></div>
<div>
Note: Eventually will still appear incorrect on the main website due to the CDN caching, which is currently setup to about 1 week.</div>
<br />
<br />
Next steps:<br />
<br />
<ul>
<li>Improvement to game mechanics</li>
</ul>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com2tag:blogger.com,1999:blog-6029629494508626591.post-11713614254202883322015-05-03T09:32:00.003+01:002015-05-03T09:32:15.871+01:00Using Genetic Algorithms to solve the Traveling Salesman Problem on Bing MapsA couple of years ago I was really into Genetic Algorithms and Ant Colony Systems, mostly focusing on solving known NP-Complex problems such as the TRP (Traveling Salesman Problem) and the VRP (Vehicle Routing Problem).<br />
<br />
As I have a couple of interesting use-cases that could benefit from these types of algorithms what better way to refresh my knowledge than making a simple mapping experiment solving the TRP problem?<br />
<br />
Here's a short summary of these concepts:<br />
<ul>
<li>TRP - Optimization problem that tries to find the shortest route that passes on all supplied points</li>
</ul>
<div>
<ul>
<li>Genetic Algorithm</li>
<ul>
<li>There's a population were each element represents a solution to the problem</li>
<li>The algorithm progresses through various iterations, called generations</li>
<li>On each generation the various elements of the population mate and create new elements</li>
<li>The fittest elements survive and the weakest die</li>
</ul>
</ul>
It obviously has much more going on than that, including stuff like roulette selection, mutation, elitism, etc.<br />
<br /></div>
<div>
</div>
<div>
I'll create a map where the user can input waypoints and the algorithm will find the shortest path.</div>
<a name='more'></a><br />
<div>
<u>1. Creating the problem space</u></div>
<div>
<br /></div>
<div>
I'll simply add a map on which the user can click to add new points. Clicking on an existing point removes it.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQbJfskUK_CpU2qFzIYPOf2YaaLegXemCBX9_bu1RuBkrB2Ca_BvmkiYopU38GWehyZQTlFYQ3Bj2FfrI0uX8XZUXQ-3nHAwjO-exFCHC4ZXiZYO2pFRNomKlHB5x0X_8wXH4PyrnvKNg/s1600/3.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhQbJfskUK_CpU2qFzIYPOf2YaaLegXemCBX9_bu1RuBkrB2Ca_BvmkiYopU38GWehyZQTlFYQ3Bj2FfrI0uX8XZUXQ-3nHAwjO-exFCHC4ZXiZYO2pFRNomKlHB5x0X_8wXH4PyrnvKNg/s1600/3.PNG" height="325" width="400" /></a></div>
<div>
<br /></div>
<div>
Now, drawing a path for the points. This polyline will represent the route to be optimized.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwyv3UW3C6s1yKS_tb0eY15ODLVEcT43XUPYz_rn1ofyUmjUk7BdS3TByqYQhIZ1lc6mA3THZFinmm-qeFhvqbY7LVcsJbpyHz3yo2gj2z97RGcCb9HP8HbwTuYkiaGjYhh-XUPFOAfmA/s1600/4.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwyv3UW3C6s1yKS_tb0eY15ODLVEcT43XUPYz_rn1ofyUmjUk7BdS3TByqYQhIZ1lc6mA3THZFinmm-qeFhvqbY7LVcsJbpyHz3yo2gj2z97RGcCb9HP8HbwTuYkiaGjYhh-XUPFOAfmA/s1600/4.PNG" height="400" width="373" /></a></div>
<div>
<br /></div>
<div>
I've actually changed the default pushpin to have one with a centered anchor point<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdK8zNKiu4o9eQ9JjkN8H55nqRIg9xrdLMphriG5r7eEtYLZTcxoM8OH8Wr6v55tpcDbkSvlaB0qnd7o_IGGPAXQf0ADEmxCk0OHsUPzulu7zkY7IZObfX4OOSCaNC1se7HN8UDPJwpXM/s1600/5.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdK8zNKiu4o9eQ9JjkN8H55nqRIg9xrdLMphriG5r7eEtYLZTcxoM8OH8Wr6v55tpcDbkSvlaB0qnd7o_IGGPAXQf0ADEmxCk0OHsUPzulu7zkY7IZObfX4OOSCaNC1se7HN8UDPJwpXM/s1600/5.PNG" height="367" width="400" /></a></div>
<br />
Now I just need the algorithm :)</div>
<div>
<br /></div>
<div>
<u>2. Creating the optimization algorithm</u></div>
<div>
<br /></div>
<div>
Regarding genetic algorithms, and although conceptually they might look sophisticated and complex, in reality they're incredibly simple. Actually, most of the algorithms based on real-life models (like ant-colonies, genetic, simulated annealing) are very easy to grasp and implement.</div>
<div>
<br /></div>
<div>
Regardless, various JS libraries already exist and instead of reinventing the wheel I've chosen one called "<a href="https://github.com/subprotocol/genetic-js">genetic-js</a>". I've never used it before but it does look really cool: clean API, good documentation and a nice set of features.</div>
<div>
<br /></div>
<div>
I'll need to implement:<br />
<ul>
<li>a seed function:</li>
</ul>
<div>
Used to generate the various individuals of the population. In this particular case each element will include all the waypoints on a random order.</div>
</div>
<div>
<ul>
<li>a fitness function:</li>
</ul>
<div>
Measures the fitness of an individual. Will represent the total distance of a route, hence the smaller the better. The distance will be, for now, linear, not taking into consideration the driving route. I'm using <a href="http://en.wikipedia.org/wiki/Vincenty%27s_formulae">Vincenty's formulae</a> for the distance calculation.</div>
</div>
<div>
<ul>
<li>a crossover function:</li>
</ul>
<div>
This function represents how two children are generated after two individuals have mated. I'm just going to make a dumb crossover function that just picks a random segment from a parent and mixes it with the other parent. As a side-note this is typically where the algorithm is optimized by using smarter mating strategies instead of just relying on chance. These are called "greedy" implementations vs "pure" ones.</div>
</div>
<div>
<ul>
<li>a mutation function:</li>
</ul>
<div>
There's a small probability of an individual simply mutating. This function represents how that will happen. I'm going to do a basic mutation function that just swaps random points of the route. Similarly to the crossover function, the mutation function also benefits a lot from greedy approaches. Anyway, not really the scope of this post to create an optimized algorithm so the random version will do.<br />
<br /></div>
</div>
<div>
Also, some additional parameters should be set, mostly around probabilities, selection logic for mating, etc.</div>
<div>
<br /></div>
<div>
To trigger the process I've added a simple button and a couple of labels to update the progress.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXr89YfROmsHqabYlL1IGFWHJ2Ruv4BN8goVjntQHFjTQOP1s5CCv_byIynDkMVTX5DPDDB54KNer2xInElIwuU3NKTweDKekllhgAHgzyw7Y-pcmhv2LAETWREBzmOIpzFowy5XoER-o/s1600/6.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXr89YfROmsHqabYlL1IGFWHJ2Ruv4BN8goVjntQHFjTQOP1s5CCv_byIynDkMVTX5DPDDB54KNer2xInElIwuU3NKTweDKekllhgAHgzyw7Y-pcmhv2LAETWREBzmOIpzFowy5XoER-o/s1600/6.PNG" height="91" width="640" /></a></div>
Now, testing the sucker:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSo3GehFya0QjrybEyytLrNgrvldHIVZ2iPYCoS8Ht9_Efb4KuFeRYFfZZe-alfxLVfHzz7aWlYRWuWLTasuRTgAztKa4gyVDp90eAVASTBq-RZ0TF1OP13DTUg3Jdc0Dq_kKBY8axdto/s1600/7.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSo3GehFya0QjrybEyytLrNgrvldHIVZ2iPYCoS8Ht9_Efb4KuFeRYFfZZe-alfxLVfHzz7aWlYRWuWLTasuRTgAztKa4gyVDp90eAVASTBq-RZ0TF1OP13DTUg3Jdc0Dq_kKBY8axdto/s1600/7.PNG" height="333" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="font-size: 12.8000001907349px;">Initial setup with 20 points<div class="separator" style="clear: both;">
<br /></div>
</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRKTyhxTUHYhu8OmP_5FMKD8qmCDilNNTV3xtFA0K_Za8dvChRUDVfpi0IoAAjahirdWoloORQMm94nKVv_jmaOoPpvmjkZPRbXgvKRlx5zh6eHhPY59ehFzDX-FQ8MohsZAoONy-VdO4/s1600/8.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRKTyhxTUHYhu8OmP_5FMKD8qmCDilNNTV3xtFA0K_Za8dvChRUDVfpi0IoAAjahirdWoloORQMm94nKVv_jmaOoPpvmjkZPRbXgvKRlx5zh6eHhPY59ehFzDX-FQ8MohsZAoONy-VdO4/s1600/8.PNG" height="335" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">During the Execution</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUg_7cjL7dMkAWpkJaw3UsIZgZgPhUgyqfmJHcHPwtIM9unO1t4mYMpaNEh4cSZ241OFw8cTIzP6YGtXWTWTl5rixWuvM4ZvHjFNQUbBTxIbEPMu_NUOiE4Bt0cnYDkMxssDhHMYuIz0U/s1600/9.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUg_7cjL7dMkAWpkJaw3UsIZgZgPhUgyqfmJHcHPwtIM9unO1t4mYMpaNEh4cSZ241OFw8cTIzP6YGtXWTWTl5rixWuvM4ZvHjFNQUbBTxIbEPMu_NUOiE4Bt0cnYDkMxssDhHMYuIz0U/s1600/9.PNG" height="333" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Final Route found</td></tr>
</tbody></table>
The current algorithm typically takes a lot of iterations (generations) to reach good solutions. There are very detailed studies on proper crossover and mutation functions to achieve good TSP results. Eventually I might update this sample to improve the algorithm, but for now you can test this live as-is at: <a href="http://psousa.net/demos/bingmaps/trp/">http://psousa.net/demos/bingmaps/trp/</a></div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-39746638663731151142015-03-21T20:28:00.001+00:002015-03-21T20:28:44.257+00:00Improve tile-loading at the browserA <a href="http://wiki.openstreetmap.org/wiki/Slippy_Map">Slippy Map</a>, such as Bing Maps or Google Maps, is composed of multiple tiles. Each tile, typically a 256x256 image, is individually fetched from the server.<br />
<br />
As displays are now supporting incredibly high resolutions, this means that tons of server requests will be required to fill a single map view. For example, on my laptop with retina display, opening a single full screen map will result in about 48 individual tiles being requested simultaneously.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioYiE_uD_j0kT8ARglK77Cu-BDVottGarg6cVEzEsndnvwEqJ2ixs0LN_XEkCaspuziNOPJIKszG7Z7x57lALVs30jvbxod02X-fwlA0zuQ2Mh1gJPIQ4nXDok_ZlhftNGzvSNN0Zg4YY/s1600/1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEioYiE_uD_j0kT8ARglK77Cu-BDVottGarg6cVEzEsndnvwEqJ2ixs0LN_XEkCaspuziNOPJIKszG7Z7x57lALVs30jvbxod02X-fwlA0zuQ2Mh1gJPIQ4nXDok_ZlhftNGzvSNN0Zg4YY/s1600/1.png" height="358" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
This is a problem as by default web browsers will limit the number of active connections for each domain. This value varies per browser, but we're talking about an average of 6 concurrent downloads per domain, which is quite low. So, assuming all tiles are served from the same domain, lots of throttling will occur.<br />
<br />
So, how to cope with this?<br />
<br />
<u>1. Tile Size</u><br />
<br />
If you control the tile generation process a "simple" option will be to generate bigger tiles, hence reducing the number of requests. Bing Maps, for instance, supports setting different sizes for the tiles (<a href="https://msdn.microsoft.com/en-us/library/gg427599.aspx">reference</a>).<br />
<div>
<br /></div>
For example, setting the tile size to be 512 instead of 256:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDHlQaxh77eCXGWipPxBJhDeSMibEX7Ua6CyDAp76PWHvvxJU2vfo-e0_s0iccrRy3o1c0NQi4xArPZZT1m9zcMtt3ZakE_0aZXEf8XTdG-gidnpUy3dQSihJ8PUJwwaPvyB6g7FMnLNo/s1600/2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiDHlQaxh77eCXGWipPxBJhDeSMibEX7Ua6CyDAp76PWHvvxJU2vfo-e0_s0iccrRy3o1c0NQi4xArPZZT1m9zcMtt3ZakE_0aZXEf8XTdG-gidnpUy3dQSihJ8PUJwwaPvyB6g7FMnLNo/s1600/2.png" height="360" width="640" /></a></div>
<br />
<pre class="prettyprint">var MM = Microsoft.Maps;
var map = new MM.Map(document.getElementById("mapDiv"), {
center: new MM.Location(45.0, 10),
zoom: 5,
credentials:"your key here"});
var tileSource = new MM.TileSource({
width: 512,
height: 512,
uriConstructor: function(tile) {
return "images/square512.png";
}});
var tileLayer = new MM.TileLayer({ mercator: tileSource});
map.entities.push(tileLayer);
</pre>
In this particular case we're talking about 15 tiles being requested (albeit each one being bigger), which is a big difference from the previous 48.<br />
<br />
<u>2. Serve tiles from different urls</u><br />
<u><br /></u>
A technique called domain sharding can be also be used, on which different domains are used to fetch the same information, thus bypassing the "same-domain" browser limitation.<br />
<br />
A good example of this can be seed on Bing Maps, as it's using this technique to speed up serving the tiles.<br />
<br />
Taking a look at the web-traffic for the base tiles we can see 4 different hostnames being used:<br />
<br />
<ul>
<li>https://t0.ssl.ak.dynamic.tiles.virtualearth.net</li>
<li>https://t1.ssl.ak.dynamic.tiles.virtualearth.net</li>
<li>https://t2.ssl.ak.dynamic.tiles.virtualearth.net</li>
<li>https://t3.ssl.ak.dynamic.tiles.virtualearth.net</li>
</ul>
<br />
The corresponding hostname is determined by the last digit of the quadkey that identifies the tile. For example, tile 0331 will use t1, tile 0330 will use t0, and so on.Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-55048370162046683882015-03-16T19:17:00.003+00:002015-03-16T19:17:54.497+00:00Game-development Log (13. Polishing the experience)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/11/game-development-log-11-persistency-and.html">11. Persistency and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2015/01/game-development-log-12-scaling-up.html">12. Scalling up the loader process</a></span><br />
<span style="font-size: x-small;">13. Polishing the experience</span></div>
<br />
Although I haven't been really active on my blog I've been playing a lot with this project adding tons of new stuff. I'm going to detail the various elements that I've updated/implemented during the last month:<br />
<ul>
<li>Pre-Generating additional Zoom Level images</li>
<li>Representing Altitude</li>
<li>Unit Energy</li>
<li>Unit Direction</li>
<li>Unit LOD Icons</li>
<li>Infantry Unit Type</li>
<li>Movement Restriction</li>
<li>Coordinate display on higher zoom levels</li>
</ul>
<a name='more'></a><b>Pre-Generating additional Zoom Level images</b><br />
<b><br /></b>
Currently my map-tiles are always generated in server-side. Some of these tiles, from zoom level 7 to 10, are pre-generated and stored on Azure's blob storage.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYmIVa9Zf_5UiL07AngHBgq32TVAYOgRMOcU60fjCwKOFJJP4tsIvU6xO5XZJ5TlQlrTsabNSnt3nja6pmnDunuyvQnMhjuKiuKYBrhRcoM5lrmg7uGZx5Xqgb208Brrano4bP_U-wS0I/s1600/76.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYmIVa9Zf_5UiL07AngHBgq32TVAYOgRMOcU60fjCwKOFJJP4tsIvU6xO5XZJ5TlQlrTsabNSnt3nja6pmnDunuyvQnMhjuKiuKYBrhRcoM5lrmg7uGZx5Xqgb208Brrano4bP_U-wS0I/s1600/76.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Zoom Level 7 (static)</td></tr>
</tbody></table>
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvHVTKVNgMsvdNqsQJ4ynD7WHoXhCMERpNXct61g9rqC1l1PdVhJ92AKNhjOVh9lMz28cCeCyivWsMXoitb74AtgT4erFoW8sfGhLakHhncOFWNT6dVv0U7rL2I4QglPGPDl1qfZB0tM0/s1600/77.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvHVTKVNgMsvdNqsQJ4ynD7WHoXhCMERpNXct61g9rqC1l1PdVhJ92AKNhjOVh9lMz28cCeCyivWsMXoitb74AtgT4erFoW8sfGhLakHhncOFWNT6dVv0U7rL2I4QglPGPDl1qfZB0tM0/s1600/77.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Zoom Level 10 (static)</td></tr>
</tbody></table>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
The additional zoom levels, from 11 to 13, are generated dynamically through a tile service.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKgsP2v7nIdkXLljO2Ea-xS9DayoFnsIwC9MhU-D9p9Mzd9tNK6j4dgScj5B27kYe8Wh6JdDbot7w5npErXfU7tgVInLn8EUT0ED3ybGQMfQHFNR56osQ3ceOeK64hyqaxFj-_Kpn5Wow/s1600/78.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKgsP2v7nIdkXLljO2Ea-xS9DayoFnsIwC9MhU-D9p9Mzd9tNK6j4dgScj5B27kYe8Wh6JdDbot7w5npErXfU7tgVInLn8EUT0ED3ybGQMfQHFNR56osQ3ceOeK64hyqaxFj-_Kpn5Wow/s1600/78.jpg" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Zoom Level 12 (dynamic)</td></tr>
</tbody></table>
Although this works it's not really cost-effective, as it would require new machines to be spawned to handle the additional load of having additional users interact with the tile-service for the closer zoom levels.<br />
<br />
So, two additional options here:<br />
<ul>
<li>Draw the map-tiles in client-side (WebGL/Canvas), having the clients fetch the vector data.</li>
<li>Pre-generate the other zoom levels</li>
</ul>
<div>
Regarding the WebGL approach, I did spend some time making a couple of experiments (resulting on some of my last posts), but in the end I wasn't fully satisfied with the approach, nor its performance (although I might pick up on this latter on).</div>
<div>
<br /></div>
<div>
I've decided to keep the server-side approach I've been using so far, but pre-generating more zoom levels. But, in order to do so, I had to improve my loader tool, including tons of changes/refactors and segregating the image generation process so that I can, for instance, use Azure Web Jobs to offload the image-tile generation.</div>
<div>
<br /></div>
<div>
Currently, on my laptop, it takes around 15 minutes to generate an area such as the United Kingdom. Not incredibly fast but completely parallelizable and I'm able to load separate areas simultaneously.<br />
<br /></div>
<div>
Currently on Azure I've imported the following region:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3HCxB36iqldi067rLNB7UVrmOMouadVg8B8p2vJEU9lF3jwI6GtwLK7AUhKKTuotmYGDYByA3U8gJ87Y4ZdVoOpNJrsBnyt4tQWRggCaFGbP150fwdAIEmTNxhIbFbPBt1XxDQBHUNzo/s1600/66.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3HCxB36iqldi067rLNB7UVrmOMouadVg8B8p2vJEU9lF3jwI6GtwLK7AUhKKTuotmYGDYByA3U8gJ87Y4ZdVoOpNJrsBnyt4tQWRggCaFGbP150fwdAIEmTNxhIbFbPBt1XxDQBHUNzo/s1600/66.png" height="384" width="400" /></a></div>
<br />
I'll soon import the rest of the world.<br />
<br />
<b>Representing Altitude</b><br />
<b><br /></b>I was already representing the altitude on the land hexagons. Thus, according to their height, the hexagons would be slightly brightened.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixTLunpzGhgWkqDzMi9K-ENT1enkF5xwsw5EJJXdT_flpWdFP-MtYU5ppV8KXFfKGJ-zS9HYm3q9yBLM5EuMf3TcreEy06hv5t-Hg2cy9v7ryOhqHh6CbiNrgSfdeQG8nzaH8NIHvpSMk/s1600/79.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEixTLunpzGhgWkqDzMi9K-ENT1enkF5xwsw5EJJXdT_flpWdFP-MtYU5ppV8KXFfKGJ-zS9HYm3q9yBLM5EuMf3TcreEy06hv5t-Hg2cy9v7ryOhqHh6CbiNrgSfdeQG8nzaH8NIHvpSMk/s1600/79.PNG" height="533" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A couple of areas with a lighter green, representing some hills (old version)</td></tr>
</tbody></table>
I've kept that cosmetic subtle effect but added a whole new thing that also affects gameplay: slopes. Thus, I detect if there's any altitude change between two hexagons and if so, I create a slope. The end-result is something like this:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6juQpR04aLpcnI9i53-4WwBW563m2z7UcXhpyrKxa21TqXMNivwROSg3Ty5oMhgTxoxbwQUkzXVOX2fEgPnnKbs6XSoXzkVvZ8K4crtJ6y0cwCWBiQT1Mfzc9OLI7Ga4YhyFo4RtC5Hs/s1600/80.PNG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6juQpR04aLpcnI9i53-4WwBW563m2z7UcXhpyrKxa21TqXMNivwROSg3Ty5oMhgTxoxbwQUkzXVOX2fEgPnnKbs6XSoXzkVvZ8K4crtJ6y0cwCWBiQT1Mfzc9OLI7Ga4YhyFo4RtC5Hs/s1600/80.PNG" height="536" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">The effect is now much less subtle (new version)</td></tr>
</tbody></table>
This will impact movement (already does for tanks) and line-of-sight (planned).<br />
<br />
Multiple levels can also be stacked. In this image I have 6 different height levels:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPF0iEkZ3mg3TMLxLhnMaualz5cmgoBVzInEmwi4E6QfIxyou7oHj4-Ro-shSOkV-YcFd31lefor6c0dyCHrAxxcPiYRANJiW6pWHVsnV-mTMG8OW_-LXEGNaBLH6pLm9xm1MNsqq_tiE/s1600/85.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPF0iEkZ3mg3TMLxLhnMaualz5cmgoBVzInEmwi4E6QfIxyou7oHj4-Ro-shSOkV-YcFd31lefor6c0dyCHrAxxcPiYRANJiW6pWHVsnV-mTMG8OW_-LXEGNaBLH6pLm9xm1MNsqq_tiE/s1600/85.PNG" height="396" width="640" /></a></div>
<br />
<br />
<b>Unit Energy</b><br />
<b><br /></b>I've added the concept of energy to the units. This includes a typical visual representation with a green bar that turns to yellow and finally to red when there's just a little bit of energy left.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQd6nyIlD4EAyD1m2b_Xj_TNulWhZdXKE9SfSnfW4gN0xVlJ6t_UiG_OT35HmObvMKGQUs9gIC3AbMPnauxdnlIIGSdn7xjuVpfes2jl_Q1pimTixZIrrEm3cM1rB75S7M5gFy37kjUus/s1600/73.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQd6nyIlD4EAyD1m2b_Xj_TNulWhZdXKE9SfSnfW4gN0xVlJ6t_UiG_OT35HmObvMKGQUs9gIC3AbMPnauxdnlIIGSdn7xjuVpfes2jl_Q1pimTixZIrrEm3cM1rB75S7M5gFy37kjUus/s1600/73.PNG" height="282" width="400" /></a></div>
<br />
Currently units may attack other units, removing their energy, without any restriction on friendly fire whatsoever. When the unit reaches zero energy it's removed.<br />
<br />
<b>Unit Direction</b><br />
<br />
Previously the icon was always facing north, regardless of the direction they had moved. Now the icon is updated to reflect the last movement it did.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEir0Yo13c2xkK0wGdpVWUvr3sc51TrYY849SkrWQ0LTPR0NBLGrv1Y9gg0LjC3JgxvlbMKk91Pxi0eVrAC6veG93AJPaOC28aDEv_yh8dzjoLnX7ZveQCT9YaJc6ePNdN4JGLzk1Y1Z6z8/s1600/81.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEir0Yo13c2xkK0wGdpVWUvr3sc51TrYY849SkrWQ0LTPR0NBLGrv1Y9gg0LjC3JgxvlbMKk91Pxi0eVrAC6veG93AJPaOC28aDEv_yh8dzjoLnX7ZveQCT9YaJc6ePNdN4JGLzk1Y1Z6z8/s1600/81.PNG" height="274" width="320" /></a></div>
<br />
<b>Unit LOD icons</b><br />
<b><br /></b>The engine now supports representing different Level-of-Detail (LOD) icons for the units according to the zoom level. Currently I've only setup 2 levels but could use additional levels if required. This transition is also used to represent the zoom level on which interaction is not longer possible.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTERIyddcC6HCtuDyjMgGv1zu9IfpWNXa__RcdfZuiLv_J5gDSxLdcYFYME0WH1eDqSH-RZR4UNYJpz6Ujk61M2y4f-0Ln2i-_UIDuwGKkIYUKs0hDUH82S78CLNo8vHaaMBWSI8xMxs/s1600/74.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTTERIyddcC6HCtuDyjMgGv1zu9IfpWNXa__RcdfZuiLv_J5gDSxLdcYFYME0WH1eDqSH-RZR4UNYJpz6Ujk61M2y4f-0Ln2i-_UIDuwGKkIYUKs0hDUH82S78CLNo8vHaaMBWSI8xMxs/s1600/74.PNG" height="334" width="400" /></a></div>
<br />
<b><br /></b>
<b><br /></b>
<b>Infantry Unit Type</b><br />
<b><br /></b>
I've also added a new unit type: infantry. It doesn't have any terrain restriction and can only move one hexagon at a time.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgloExuEH9FvkyqB7tbDai7nOnh57i45A386PAxGwhlARsx4GO_arJKOz5WNdanbID07QQz995PZhyYENmq3tDEivr6n0ya6Dq8fF3CfroF3KAtC05SaHOhmqGqhD6lQqEJNM-htGTFdiM/s1600/82.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgloExuEH9FvkyqB7tbDai7nOnh57i45A386PAxGwhlARsx4GO_arJKOz5WNdanbID07QQz995PZhyYENmq3tDEivr6n0ya6Dq8fF3CfroF3KAtC05SaHOhmqGqhD6lQqEJNM-htGTFdiM/s1600/82.PNG" height="288" width="320" /></a></div>
<br />
<br />
<b>Unit Movement Restriction for tanks</b><br />
<b><br /></b>
I've added a couple of movement restrictions for the tanks and now they <u>can't</u>:<br />
- Enter forests<br />
- Climb slopes (except using a road)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii2U2FqlFDlyPQw9yUum8dsjgLT92sVuM6UqBVnAOA9mSYDoW4W_t9AJssoEPbSm9nmpJ_Baj8jQu3hBfNhOFN74rxDMvsvCZQOaEtejXG6D5mYtGr4jlcPqBdqmizWTE3ANg5grHNMYM/s1600/83.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEii2U2FqlFDlyPQw9yUum8dsjgLT92sVuM6UqBVnAOA9mSYDoW4W_t9AJssoEPbSm9nmpJ_Baj8jQu3hBfNhOFN74rxDMvsvCZQOaEtejXG6D5mYtGr4jlcPqBdqmizWTE3ANg5grHNMYM/s1600/83.png" height="307" width="320" /></a></div>
<br />
<br /></div>
<div>
<b>Coordinate display on higher zoom levels</b></div>
<div>
<b><br /></b></div>
<div>
The maximum zoom level (13) now displays the U,V coordinates for the hexagons. This could eventually have a great strategic value, particularly for team-play. Will allow stuff like: "rally with me at 8310 788" or "enemy spotted at 8719 667", etc.</div>
<div>
<br /></div>
<div>
Also, it's quite useful for debugging/development purposes :)</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkcsmUQNRDwhu5CfGlikQ0pZKajBZOiqLpQJ7D5MHY1Aw_9IWKvbt1eahZFABqadDMVuEHqlBe5N0uceNetYzWFuKqBsSDGj_3zojA4TVf7uYVSHTVW4s6o5BVdmwnzsU5ErG_OCGlIIE/s1600/84.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkcsmUQNRDwhu5CfGlikQ0pZKajBZOiqLpQJ7D5MHY1Aw_9IWKvbt1eahZFABqadDMVuEHqlBe5N0uceNetYzWFuKqBsSDGj_3zojA4TVf7uYVSHTVW4s6o5BVdmwnzsU5ErG_OCGlIIE/s1600/84.PNG" height="484" width="640" /></a></div>
<div>
<br /></div>
<div>
Anyway, you can see this work-in-progress live at: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
Next steps:<br />
<ul>
<li>Generate tiles for the rest of the world</li>
<li>Cool-down time after moving/attacking</li>
<li>Line-of-sight</li>
<li>Unit visibility (ex: an infantry unit inside an hexagon forest will be hidden from other players)</li>
</ul>
</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-43524050040552942272015-02-03T23:32:00.000+00:002015-02-04T15:59:10.949+00:00Displaying 3d objects on Bing Maps (using Three.js)On my previous post I've made a couple of experiments on displaying 2d content on top of Bing Maps using Pixi. Performance was top-notch, particularly if the browser supported WebGL (fallbacking to Canvas otherwise).<br />
<br />
This time I'm trying to take the WebGL experiment even further by adding 3d content on top of a map. For that I'm going to use the most popular WebGL 3D lib out there: <a href="http://threejs.org/">Three.js</a><br />
<br />
Let me start by showing the end-result.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9c4oIRFcb3tL98JbhVJsRP8uD3ycN2PCtR75RobTiVcomWyfCNrxVWBNUr-iAgIjEfUOEoAjU-Xh7fG-WsizJkj2rsGDFAEIhZcKRYtvAINegg152hoNWtTF7bDBv-RS1GQyTTUjY9SM/s1600/three1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg9c4oIRFcb3tL98JbhVJsRP8uD3ycN2PCtR75RobTiVcomWyfCNrxVWBNUr-iAgIjEfUOEoAjU-Xh7fG-WsizJkj2rsGDFAEIhZcKRYtvAINegg152hoNWtTF7bDBv-RS1GQyTTUjY9SM/s1600/three1.png" /></a></div>
<br />
I've placed some boxes on top of a map. Although the map by itself is 2D the boxes are rendered in 3D using WebGL. Thus, as further away from the screen center (the vanishing point) the more pronounced the depth effect will be.<br />
<a name='more'></a><b>So, how to do this on top of Bing Maps?</b><br />
<br />
As usual, I've decomposed this into simpler steps.<br />
<br />
<u>1. Create a DOM element on top of Bing maps to place the Three.JS renderer</u><br />
<br />
This code is very similar to its Pixi counterpart (from my previous post).
<br />
<pre class="prettyprint">var mapDiv = map.getRootElement();
var width = map.getWidth();
var height = map.getHeight();
renderer = new THREE.WebGLRenderer( { alpha: true } );
renderer.shadowMapEnabled = true;
renderer.setSize(width, height);
mapDiv.parentNode.lastChild.appendChild( renderer.domElement );
renderer.domElement.style.position = "absolute";
renderer.domElement.style.top = "0px";
renderer.domElement.style.left = "0px";
</pre>
The renderer should be created with "alpha", otherwise the map won't be visible.<br />
<br />
With this code a canvas element is generated on top of the map for the renderer.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXRzS2z83qetyrAb5kSDHw5vz78twbEsJV_o5M9PZTXl1ECLpCwVFaXZkYF3kuPkhOUfbaaZBw2i9I5kDs21rvu-ZC4jM47WL1jBjp-lAuPWrrHy-Rt4BJjxKfHcEKaAZSPddVABvs340/s1600/inspector.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXRzS2z83qetyrAb5kSDHw5vz78twbEsJV_o5M9PZTXl1ECLpCwVFaXZkYF3kuPkhOUfbaaZBw2i9I5kDs21rvu-ZC4jM47WL1jBjp-lAuPWrrHy-Rt4BJjxKfHcEKaAZSPddVABvs340/s1600/inspector.PNG" height="181" width="640" /></a></div>
<br />
<br />
<u>2. Add boxes, lights and a camera</u><br />
<br />
Nothing really special there. Just added these objects directly without any "spatial" positioning. Used:<br />
<ul>
<li>THREE.PerspectiveCamera for the camera</li>
<li>THREE.BoxGeometry for the boxes</li>
<li>THREE.AmbientLight/THREE.PointLight for the lights</li>
<li>A simple texture for the boxes</li>
</ul>
<br />
<u>3. Mapping boxes position to geo coordinates</u><br />
<br />
Now this was the tricky part. All the magic is done on function "updatePosition". What I do, per box, is:<br />
<ul>
<li>Obtain the geo-coordinate associated with the box</li>
<li>Obtain the screen coordinate for that coordinate</li>
<ul>
<li>Using map.tryLocationToPixel</li>
</ul>
<li>Obtain the Three.js world coordinates for that pixel</li>
<ul>
<li>Uses a technique called raycasting, trying to find the interception between a projected ray from the obtained screen coordinate to an invisible plane object that I've created.</li>
<li>The interception will be the world coordinate to where the box needs to be moved.</li>
</ul>
<li>As the pivot point of the box is on its center the box needs to be shifted upwards to half of its height (I'm actually doing this just once during the loading phase)</li>
<li>During zoom the box needs to be scaled. I've added a parent object for each box (at the bottom) to work out as a new pivot point.</li>
</ul>
<div>
Anyway, here's a video of this experiment in action
<iframe allowfullscreen="" frameborder="0" height="400" src="https://www.youtube.com/embed/ROVDm4BPDxM" width="600"></iframe></div>
<br />
And also a working page: <a href="http://psousa.net/demos/bingmaps/webgl/three/three3.html">http://psousa.net/demos/bingmaps/webgl/three/three3.html</a> where you can take a look at the code. Just remember to open this using Chrome :)<br />
<br />
<div style="-webkit-text-stroke-width: 0px; color: black; font-family: 'Times New Roman'; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">
So this is basically it. I believe it has the potential for some interesting use cases and WebGL is here to stay. More and more people are using compatible browsers and Microsoft decision to simplify upgrading to Windows 10 will certainly bump this number even further.</div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com7tag:blogger.com,1999:blog-6029629494508626591.post-58473555407282609992015-01-06T19:37:00.001+00:002015-01-06T19:37:09.513+00:00Displaying WebGL on Bing Maps (using Pixi.js)Something that has been on my backlog for some time is trying to mix Bing Maps and WebGL, similarly to what I've done for an "old" Google Maps <a href="http://build-failed.blogspot.com/2013/02/displaying-webgl-data-on-google-maps.html">experiment</a>.<br />
<br />
That previous demo was done on top of a Google maps sample, hence just requiring some small tweaks and improvements. Also, was very low-level and not really practical to adapt to more "real-world" usage, as it required programming the shaders, computing the transformation matrixes, etc.<br />
Thus, I was trying to find a alternative WebGL JS lib that was:<br />
<ul>
<li>Fast</li>
<li>Easy to use, albeit still providing some low-level control, namely on primitives drawing</li>
</ul>
After some research I ended up with two candidates:<br />
<ul>
<li><a href="http://lib.ivank.net/">IvanK Lib</a></li>
<li><a href="http://www.pixijs.com/">Pixi.js</a></li>
</ul>
IvanK Lib is pretty good (and fast) but Pixi.js takes the cake with tons of functionality and a large community using it.<br />
<br />
I'm going to enumerate the various experiments I did, showing a sample page for each.<br />
<br />
<a name='more'></a><br />
<b>Recommendation: </b>use Chrome as otherwise it might be painfully slow/non-working.<br />
<br />
<b>1. Create a pixi stage on top of Bing Maps.</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi1.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi1.html</a>
<br />
<pre class="prettyprint">var mapDiv = map.getRootElement();
stage = new PIXI.Stage();
// create a renderer instance mapping (pun intended) the size of the map.
renderer = PIXI.autoDetectRenderer(
map.getWidth(),
map.getHeight(),
{transparent: true});
// add the renderer view element to the DOM, making it sit on top of the map
mapDiv.parentNode.lastChild.appendChild(renderer.view);
renderer.view.style.position = "absolute";
renderer.view.style.top = "0px";
renderer.view.style.left = "0px";
renderer.render(stage);</pre>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZVSELqW2H_cjjzgkcnIjEtrpr-rN4YDoJYq_EJl6MW5ZE4YXSefp-QwwwCGehrvGoXNLwcKtWw4__DT8ZM23u3iVDouRqDhUtfHxVL64bBMPbsftCzSQVrBYSaU6tb2XzErzHk_xpj3g/s1600/pixi1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZVSELqW2H_cjjzgkcnIjEtrpr-rN4YDoJYq_EJl6MW5ZE4YXSefp-QwwwCGehrvGoXNLwcKtWw4__DT8ZM23u3iVDouRqDhUtfHxVL64bBMPbsftCzSQVrBYSaU6tb2XzErzHk_xpj3g/s1600/pixi1.png" /></a></div>
<br />
Yep, nothing visible. Regardless, if you open a DOM inspector you can see a canvas element that was generated on top of the map.<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvfOt67o3VmRbjdoueGDemKKmM9EIVVUKfUpp1yof1mIW-Btrqa8sYXSV9CP-N1z7ip2gFl_s4PQACmAh_ca7Z5DoFKvrHVs2MVF-79F3AEyc67BAhal4fdIg98Wv7Ph4WA6RVOZD3gwc/s1600/dom.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvfOt67o3VmRbjdoueGDemKKmM9EIVVUKfUpp1yof1mIW-Btrqa8sYXSV9CP-N1z7ip2gFl_s4PQACmAh_ca7Z5DoFKvrHVs2MVF-79F3AEyc67BAhal4fdIg98Wv7Ph4WA6RVOZD3gwc/s1600/dom.png" /></a><br />
<br />
<b>2. Add a sprite to the map.
</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi2.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi2.html</a><br />
<pre class="prettyprint">var texture = PIXI.Texture.fromImage("img/bunny.png");
var bunny = new PIXI.Sprite(texture);
// center the sprite anchor point
bunny.anchor.x = 0.5;
bunny.anchor.y = 0.5;
bunny.lat = 40.0;
bunny.lon = -8.5;
var pixelCoordinate = map.tryLocationToPixel(
new MM.Location(bunny.lat, bunny.lon),
MM.PixelReference.control);
bunny.position.x = pixelCoordinate.x;
bunny.position.y = pixelCoordinate.y;
stage.addChild(bunny);
</pre>
Although the bunny is properly added on top of the map it's not georeferenced. Thus, if the map is moved the bunny stays on the same screen position.<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4c-ZHLgCzgXy6JZeqlBRKV1UCOc7zGldCR8GL4JwnftheStYiT53wQUKhbIimXfWCcTPPVuPVhxto2AfDHZzjnza249QVc8RqzKBcJjwB71qm-Wt5UUrvz2DKMMGpWz4E4E7_wMTYd64/s1600/pixi2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4c-ZHLgCzgXy6JZeqlBRKV1UCOc7zGldCR8GL4JwnftheStYiT53wQUKhbIimXfWCcTPPVuPVhxto2AfDHZzjnza249QVc8RqzKBcJjwB71qm-Wt5UUrvz2DKMMGpWz4E4E7_wMTYd64/s1600/pixi2.PNG" /></a><br />
<br />
<b>3. Listen to the <i>viewchange</i> event and update the sprite position</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi3.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi3.html</a>
<br />
<pre class="prettyprint">MM.Events.addHandler(map, 'viewchange', updatePosition);
(...)
function updatePosition(e) {
var pixelCoordinate = map.tryLocationToPixel(
new MM.Location(bunny.lat, bunny.lon),
MM.PixelReference.control);
bunny.position.x = pixelCoordinate.x;
bunny.position.y = pixelCoordinate.y;
renderer.render(stage);
}
</pre>
<b>4. Do the same thing for 1000 sprites</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi4.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi4.html</a><br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhF3zxUnBvALY8Q6bYEiDvi4B0bgX_pXhowFF8a6HMyPnu9R_X00f2TbqDFAi_t6k6lHrgBBvn_o67aTIbkG8OJHZIhtvMLH2G5ai6mlK60TC8AJ7ygXaUC0uYH9WWZK3k7RnLXhZDJTE/s1600/pixi4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhhF3zxUnBvALY8Q6bYEiDvi4B0bgX_pXhowFF8a6HMyPnu9R_X00f2TbqDFAi_t6k6lHrgBBvn_o67aTIbkG8OJHZIhtvMLH2G5ai6mlK60TC8AJ7ygXaUC0uYH9WWZK3k7RnLXhZDJTE/s1600/pixi4.png" /></a></div>
Depending on your machine (and graphics card) should still behave nicely. Regardless, when displaying lots of similar sprites pixi.js supports the concept of <a href="http://www.goodboydigital.com/pixijs/docs/classes/SpriteBatch.html">SpriteBatch</a>:<br />
<blockquote class="tr_bq">
<span style="background-color: #f0f1f8; color: #333333; font-family: Helvetica, Arial, sans-serif; font-size: 13px; line-height: 18.2000007629395px;">The SpriteBatch class is a really fast version of the DisplayObjectContainer built solely for speed, so use when you need a lot of sprites or particles</span></blockquote>
<b>5. Use SpriteBatch</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi5.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi5.html</a>
<br />
<pre class="prettyprint">container = new PIXI.SpriteBatch();
stage.addChild(container);
for (var i = 0; i < 1000; i++) {
var bunny = new PIXI.Sprite(texture);
// center the sprites anchor point
bunny.anchor.x = 0.5;
bunny.anchor.y = 0.5;
// move the sprite t the center of the screen
bunny.lat = 40.0 + Math.random() * 20;
bunny.lon = -8.5 + Math.random() * 50;
var pixelCoordinate = map.tryLocationToPixel(
new MM.Location(bunny.lat, bunny.lon),
MM.PixelReference.control);
bunny.position.x = pixelCoordinate.x;
bunny.position.y = pixelCoordinate.y;
container.addChild(bunny);
bunnies.push(bunny);
}
</pre>
It's really simple to use. Instead of adding the sprites to the stage add them to a SpriteBatch. Now, the problem is that this code is still updating the position of each individual sprite when moving/zooming the map.<br />
<br />
<b>6. Scale the SpriteBatch instead of reposition individual sprites</b>
<br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi6.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi6.html</a><br />
<pre class="prettyprint">function updatePosition(e) {
if(!e.linear) //zooming animation
{
var currentWidth = getCurrentWidth();
var diff = startWidth / currentWidth;
container.scale.x = diff;
container.scale.y = diff;
var divTopLeft = map.tryLocationToPixel(startPosition, MM.PixelReference.control);
var x = divTopLeft.x;
var y = divTopLeft.y;
container.position.x = x;
container.position.y = y;
renderer.render(stage);
}
}
</pre>
This sample doesn't update the individual sprites and scales the SpriteBatch as a whole. This provides a good performance impact, although the sprites will look pixelated on higher zoom levels.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCTeAielmHDONS3zucJe2USNfDw0eXfWGR0M8JC_rH3tZX_zFdKy0NoJqh6G_MDbmiJtJj3MoSNpPwksRN-5PDqz-IJK5WV5XECeyG6HoecO2JHGVRRpmRjF5MFXp74NvEOmZKLdKGElM/s1600/pixi6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiCTeAielmHDONS3zucJe2USNfDw0eXfWGR0M8JC_rH3tZX_zFdKy0NoJqh6G_MDbmiJtJj3MoSNpPwksRN-5PDqz-IJK5WV5XECeyG6HoecO2JHGVRRpmRjF5MFXp74NvEOmZKLdKGElM/s1600/pixi6.png" /></a></div>
An improved solution would be to use this mechanism on panning/zooming, and having different LOD (Level-of-Detail) Sprites, which would be redrawn when the zoom animation finished.<br />
<br />
Now, instead of drawing sprites I'm going to show how to draw primitives (in this case rectangles).<br />
<br />
<b>7. Draw primitives</b><br />
<a href="http://psousa.net/demos/bingmaps/webgl/pixi/pixi7.html">http://psousa.net/demos/bingmaps/webgl/pixi/pixi7.html</a>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidRJG7OQtnFTZfaWEAgXvs412Jo4ht7zHrHr7bx_gZ2of9oNvvUmjj4D3OgRNSJtTqQfncfEJ6KHb5mBNN6PBrxgwEdIJDM_tH1RuKFP1-Bz8q9epMkm5iZphhK28iyV2fHcQqIML4eJU/s1600/pixi7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEidRJG7OQtnFTZfaWEAgXvs412Jo4ht7zHrHr7bx_gZ2of9oNvvUmjj4D3OgRNSJtTqQfncfEJ6KHb5mBNN6PBrxgwEdIJDM_tH1RuKFP1-Bz8q9epMkm5iZphhK28iyV2fHcQqIML4eJU/s1600/pixi7.png" /></a></div>
<br />
<pre class="prettyprint">graphics = new PIXI.Graphics();
var referencePixel = map.tryLocationToPixel({ latitude: 44, longitude: -9.5}, MM.PixelReference.control);
graphics.beginFill(0xFFFFFF);
for(var i=0; i < 20000; i++) {
graphics.drawRect(referencePixel.x + Math.random()* 1200.0, referencePixel.y + Math.random()*900.0, 2, 2);
}
graphics.endFill();
</pre>
I'm basically creating 20000 small rectangles using pixi.js. On higher zoom levels precision isn't lost as this is vector data (as opposed to the raster data from the previous examples).<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiI2CWOMIouLyrQQ3lNFbDLT-A3evGHkIs66OgZliCvHWs53DqfRU3gBqJj_DxG0h42m1Sm42hIS-dbFSBAAOEULKwPZGc2iDaV7MORMsB2Q3ImUrcFqpIbR-T7HvlJHIPRoBLSLy9TVcg/s1600/pixi71.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiI2CWOMIouLyrQQ3lNFbDLT-A3evGHkIs66OgZliCvHWs53DqfRU3gBqJj_DxG0h42m1Sm42hIS-dbFSBAAOEULKwPZGc2iDaV7MORMsB2Q3ImUrcFqpIbR-T7HvlJHIPRoBLSLy9TVcg/s1600/pixi71.png" /></a></div>
<br />
All of this is obviously non-production code with various bugs. Regardless, future looks promising :)Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com4tag:blogger.com,1999:blog-6029629494508626591.post-3230694475385899952015-01-01T19:26:00.000+00:002015-03-16T19:20:25.597+00:00Game-development Log (12. Scaling up the loader process)<div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/11/game-development-log-11-persistency-and.html">11. Persistency and Authentication</a></span><br />
<span style="font-size: x-small;">12. Scalling up the loader process</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html">13. Polishing the experience</a></span></div>
<br />
During the last month I've been mostly refactoring the loader process. As it was I had lots of trouble loading larger areas. I've detailed the problem (and the solution) on a <a href="http://build-failed.blogspot.pt/2014/12/splitting-vector-and-raster-files-in.html">previous post</a>. I've also optimized the loading process and I'm able to load the whole world in about 1 day, already including pre-generating the lower zoom level images.<br />
<br />
Regardless, I've haven't yet imported the whole world as everything is still work in progress. I'm planning to add some additional stuff to the map, like mountains, regions, cities and overall improved aesthetics.<br />
<br />
I've currently imported to Azure a rectangle that goes from coordinate [15E 60N] to [15W 30N]. Basically this area:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVkPFG0nIYxawLC3Y8b2M6H9o0qQSPI5VZWQrfJvPk2KF91atYRshGbYJas6tgChwt_PIipUnZMHuvkI35HIqYt98Z6GkJIeDmymPbnK0XS69mVBo7wYyvkQIBjLjQYz0-NoFiH1JhSWw/s1600/66.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVkPFG0nIYxawLC3Y8b2M6H9o0qQSPI5VZWQrfJvPk2KF91atYRshGbYJas6tgChwt_PIipUnZMHuvkI35HIqYt98Z6GkJIeDmymPbnK0XS69mVBo7wYyvkQIBjLjQYz0-NoFiH1JhSWw/s1600/66.png" height="614" width="640" /></a></div>
<br />
If you zoom in inside this area the Bing Maps tiles are replaced with my own (after zoom level 7). For example, zooming in into London:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdPlNW-oWjBeXodkglhsADfJURuGS2lselZ7bdMoNJDXy08qDN9U7xSqa2yKRaM3cN1B3OStBFnEPQCT_-nEYfOEFD6M4gfIBFs9iGUkyom7jVbYnB4tV_QWYXbMt7PAxlrfnPQfrmYxU/s1600/72.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdPlNW-oWjBeXodkglhsADfJURuGS2lselZ7bdMoNJDXy08qDN9U7xSqa2yKRaM3cN1B3OStBFnEPQCT_-nEYfOEFD6M4gfIBFs9iGUkyom7jVbYnB4tV_QWYXbMt7PAxlrfnPQfrmYxU/s1600/72.png" height="126" width="640" /></a></div>
<br />
Or a random area on Norway:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5wd-wlhyPpoZKUJasodvKJ9akP6oXjtg7gdUbzIBLAg1Rz_v3um9-7sFEq2HBRqE-nSjOiU2FoQxfBwW_xSlRsLBZSDAeO02hjuzdsMt9ohpMckdd8YnC1CjGxzcXFtkQv5nWzRCS-GA/s1600/68.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5wd-wlhyPpoZKUJasodvKJ9akP6oXjtg7gdUbzIBLAg1Rz_v3um9-7sFEq2HBRqE-nSjOiU2FoQxfBwW_xSlRsLBZSDAeO02hjuzdsMt9ohpMckdd8YnC1CjGxzcXFtkQv5nWzRCS-GA/s1600/68.png" height="126" width="640" /></a></div>
<br />
Or Béchar in Algeria:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGAUhPgs80iTbsQA2VomprIAc4W1yYikg0ew5FosdyBUDfxeKiqWHGoa0sYq-DoG7A7JvaZtfsrGmxFK-A5rvzGZXpceO-3GSoVNbsgapR95DesHDtPm3QhqfQhQpxGLbZGs4ESo0vEbk/s1600/69.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGAUhPgs80iTbsQA2VomprIAc4W1yYikg0ew5FosdyBUDfxeKiqWHGoa0sYq-DoG7A7JvaZtfsrGmxFK-A5rvzGZXpceO-3GSoVNbsgapR95DesHDtPm3QhqfQhQpxGLbZGs4ESo0vEbk/s1600/69.png" height="128" width="640" /></a></div>
<br />
The most important thing to note is that those hexagons are not simply cosmetic. All information is being pushed to client-side so that the units will behave differently depending on the terrain. For example, a tank can only cross a river through a bridge:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxXWcuc5xPbXrNe8NQY27TcPUWnqsZQM5O47NY-9i3_dD2y_8ARgzMMZYTT4Kpqph_GqyvoktZx6WfXpdiQi6tQ-CGkEmfC3FNzkG95y30kHEaJ6buDr0ZXF47WwaOAyl-L1mpio0Cgac/s1600/70.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxXWcuc5xPbXrNe8NQY27TcPUWnqsZQM5O47NY-9i3_dD2y_8ARgzMMZYTT4Kpqph_GqyvoktZx6WfXpdiQi6tQ-CGkEmfC3FNzkG95y30kHEaJ6buDr0ZXF47WwaOAyl-L1mpio0Cgac/s1600/70.png" /></a></div>
<br />
I've also added:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpbuYtNnDpw5AvSjaPVG3L0QOUgvd_u2U7kPZrj7GWb-5Ihu_w2q33UUa0Lg5JTqIpOWvRujXf0vr0gLn8DIMKN40J0J-HdDzOFOZhc_0e6vfIqmZsf2XC3NCmxZlle05TaBPcj4wxa30/s1600/71.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhpbuYtNnDpw5AvSjaPVG3L0QOUgvd_u2U7kPZrj7GWb-5Ihu_w2q33UUa0Lg5JTqIpOWvRujXf0vr0gLn8DIMKN40J0J-HdDzOFOZhc_0e6vfIqmZsf2XC3NCmxZlle05TaBPcj4wxa30/s1600/71.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Deserts</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1Ewcv3DYo5Z5WNqF3FECNY72Ho9kkkBcNGdZIOSow7eM5IlG0OKFTU9hHYoqRwo8s1fobg5sGIfNjwAxNJAgeIKGFVR8czjWFyxrGx7TjJIme1d8RbPaP9bA5GTQlCHmMUz0QqWPss-Y/s1600/72.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1Ewcv3DYo5Z5WNqF3FECNY72Ho9kkkBcNGdZIOSow7eM5IlG0OKFTU9hHYoqRwo8s1fobg5sGIfNjwAxNJAgeIKGFVR8czjWFyxrGx7TjJIme1d8RbPaP9bA5GTQlCHmMUz0QqWPss-Y/s1600/72.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Two levels of forests: dense and sparse</td></tr>
</tbody></table>
<br />
<div>
Anyway, you can play around with it at: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net/</a></div>
<br />
<br />
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-70236686278024970822014-12-25T12:26:00.003+00:002014-12-25T12:31:54.611+00:00Processing GeoTiff files in .NET (without using GDAL)I've recently created a process that is able to import geoferenced raster data, namely GeoTiff files... with a catch:<br />
<div>
As I didn't find a proper lib to load GeoTiff files I was converting the source files to an <a href="http://www.gdal.org/frmt_xyz.html">ASCII Gridded XYZ</a> format prior to importing. Despite the fancy name it's simply an ASCII file where each line contains:</div>
<pre class="prettyprint"><span style="font-family: 'Times New Roman'; white-space: normal;"><LONGITUDE> <LATITUDE> <VALUE1> [ <VALUE 2>] [<VALUE 3>]</span>
</pre>
<div>
Each line on this file corresponds to a pixel on a raster image, thus it's incredibly easy to parse in C#. The code would be something like this:</div>
<pre class="prettyprint">foreach (var line in File.ReadLines(filePath))
{
if (line == null)
{
continue;
}
var components = line.Split(
new[] { ' ' },
StringSplitOptions.RemoveEmptyEntries);
var longitude = double.Parse(components[0]);
var latitude = double.Parse(components[1]);
IEnumerable<string> values = components.Skip(2);
//... process each data item
yield return dataItem;
}
</pre>
<div>
Nice and easy. But this has two disadvantages:<br />
<br />
First, it requires an extra step to convert to XYZ from Geotiff, although easily done with the GDAL:<br />
<pre class="prettyprint">gdal_translate -of XYZ <input.tif> <output.xyz>
</pre>
</div>
<div>
Another disadvantage is that the XYZ becomes much larger than its TIFF counterpart. Depending on the source file the difference could be something like 10 GB vs 500 MB (yes, 50x bigger)</div>
<div>
<br />
So, I would really like to be able to process a GeoTiff file directly in c#. One obvious candidate is using GDAL, particularly its C# bindings. Although these bindings work relatively well they rely on an unmanaged GDAL installation. For me this is a problem as I want to host this on Azure WebSites, where installing stuff is not an option.<br />
<div>
So, I embraced the challenge of being able to process a GeoTIFF file in a fully managed way.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://stickerish.com/wp-content/uploads/2011/04/ChallengeAcceptedBlackTextSS.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://stickerish.com/wp-content/uploads/2011/04/ChallengeAcceptedBlackTextSS.png" height="320" width="295" /></a></div>
<br />
<a name='more'></a>The file that I'll be loading is this one. It's an height map for the entire world (21600x10800, 222MB)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXeL3zWvwGHxP5s8QZ4IyNXBmJVkOBZC1OpZjpSjz8ucddaDFRhDpKYPEhSa-7ZZDkOfcxzFEO6gSerFtKaFopvel2aFPwoKSAUeU5STXtZR2kB9fvBFYJbEyn9QVbSRnng-m_NfWYQFA/s1600/64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXeL3zWvwGHxP5s8QZ4IyNXBmJVkOBZC1OpZjpSjz8ucddaDFRhDpKYPEhSa-7ZZDkOfcxzFEO6gSerFtKaFopvel2aFPwoKSAUeU5STXtZR2kB9fvBFYJbEyn9QVbSRnng-m_NfWYQFA/s1600/64.png" /></a></div>
<br />
<u>First step, loading the TIFF file, still ignoring any geo data.</u></div>
<div>
<br /></div>
Fortunately I found a really nice lib that does exactly what I wanted: be able to load a tiff file in c#, fully managed. It's called <a href="http://bitmiracle.com/libtiff/">LibTiff.NET</a>.<br />
<br />
Although I'm not really found of its API, it does include tons of functionality and documentation. Most demos load the full image into memory (which isn't really viable for larger tiff files) but it does include the capability to process a file one line at a time.<br />
<br />
My starting point was this example <a href="https://bitmiracle.github.io/libtiff.net/?topic=html/e4f25423-eede-4ef6-a920-9cb539d056c6.htm">here</a>. The important bits are:<br />
<pre class="prettyprint">using (Tiff tiff = Tiff.Open(@"<file.tif>", "r"))
{
int width = tiff.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
byte[] scanline = new byte[tiff.ScanlineSize()];
for (int i = 0; i < height; i++)
{
tiff.ReadScanline(scanline, i);
}
}
</pre>
This sample opens a tiff and iterates each line. This code is used to open 8bit tiff files. 16 bits files would require an additional operation:</div>
<pre class="prettyprint">using (Tiff tiff = Tiff.Open(@"<file.tif>", "r"))
{
int width = tiff.GetField(TiffTag.IMAGEWIDTH)[0].ToInt();
byte[] scanline = new byte[tiff.ScanlineSize()];
ushort[] scanline16Bit = new ushort[tiff.ScanlineSize() / 2];
for (int i = 0; i < height; i++)
{
tiff.ReadScanline(scanline, i);
Buffer.BlockCopy(scanline, 0, scanline16Bit, 0, scanline.Length);
}
}
</pre>
<u>Second step, loading geographical data and corresponding height value</u><br />
<br />
The aforementioned lib doesn't include any support for GeoTiff. So, time to take a look at the official docs.<br />
<br />
According to the official FAQ (<a href="http://www.remotesensing.org/geotiff/faq.html">http://www.remotesensing.org/geotiff/faq.html</a>):<br />
<blockquote class="tr_bq">
<i>GeoTIFF is a metadata format, which provides geographic information to associate with the image data. But the TIFF file structure allows both the metadata and the image data to be encoded into the same file.<br />GeoTIFF makes use of a public tag structure which is platform interoperable between any and all GeoTIFF-savvy readers. Any GIS, CAD, Image Processing, Desktop Mapping and any other types of systems using geographic images can read any GeoTIFF files created on any system to the GeoTIFF specification.</i> </blockquote>
Before delving into the spec, I need a comparison point to make sure I'm getting the proper values. Thus I'm using <a href="http://www.gdal.org/gdalinfo.html">gdalinfo</a> to provide some information on the raster file. It's usage is really simple:<br />
<pre>gdalinfo altitude.tif
</pre>
This outputs the following information:
<br />
<pre><span style="font-size: x-small;">Size is 21600, 10800
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,90.000000000000000)
Pixel Size = (0.016666666666667,-0.016666666666667)
Metadata:
AREA_OR_POINT=Area
EXIF_ColorSpace=65535
EXIF_DateTime=2005:10:12 22:04:52
EXIF_Orientation=1
EXIF_PixelXDimension=21600
EXIF_PixelYDimension=10800
EXIF_ResolutionUnit=2
EXIF_Software=Adobe Photoshop CS2 Macintosh
EXIF_XResolution=(72)
EXIF_YResolution=(72)
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left (-180.0000000, 90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)
Lower Left (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)
Upper Right ( 180.0000000, 90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)
Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)
Center ( 0.0000000, 0.0000000) ( 0d 0' 0.01"E, 0d 0' 0.01"N)
Band 1 Block=21600x1 Type=Byte, ColorInterp=Gray
Min=0.000 Max=213.000
Minimum=0.000, Maximum=213.000, Mean=22.754, StdDev=25.124
Metadata:
STATISTICS_MAXIMUM=213
STATISTICS_MEAN=22.753594797178
STATISTICS_MINIMUM=0
STATISTICS_STDDEV=25.124203131182</span>
</pre>
I'm mostly interested on these two lines:
<br />
<pre>Origin = (-180.000000000000000,90.000000000000000)
Pixel Size = (0.016666666666667,-0.016666666666667)
</pre>
<div>
This means that the top-left corner of the image corresponds to coordinate -180,90 and that each pixel increments 0.016(7) degrees.</div>
<br />
So, time to check the spec, namely section 2.6.1 on <a href="http://www.remotesensing.org/geotiff/spec/geotiff2.6.html">http://www.remotesensing.org/geotiff/spec/geotiff2.6.html</a>):<br />
<br />
Apparently tags 33922 and 33550 provide exactly the information I need. They're defined as:<br />
<blockquote class="tr_bq">
<i>ModelTiepointTag:<br /> Tag = 33922 (8482.H)<br /> Type = DOUBLE (IEEE Double precision)<br /> N = 6*K, K = number of tiepoints<br /> Alias: GeoreferenceTag<br /> Owner: Intergraph<br />This tag stores raster->model tiepoint pairs in the order<br /><br /> ModelTiepointTag = (...,I,J,K, X,Y,Z...),<br />where (I,J,K) is the point at location (I,J) in raster space with pixel-value K, and (X,Y,Z) is a vector in model space. In most cases the model space is only two-dimensional, in which case both K and Z should be set to zero; this third dimension is provided in anticipation of future support for 3D digital elevation models and vertical coordinate systems.</i></blockquote>
and<br />
<blockquote class="tr_bq">
<i>ModelPixelScaleTag:<br /> Tag = 33550<br /> Type = DOUBLE (IEEE Double precision)<br /> N = 3<br /> Owner: SoftDesk<br />This tag may be used to specify the size of raster pixel spacing in the model space units, when the raster space can be embedded in the model space coordinate system without rotation, and consists of the following 3 values:<br /><br /> ModelPixelScaleTag = (ScaleX, ScaleY, ScaleZ)<br />where ScaleX and ScaleY give the horizontal and vertical spacing of raster pixels. The ScaleZ is primarily used to map the pixel value of a digital elevation model into the correct Z-scale, and so for most other purposes this value should be zero (since most model spaces are 2-D, with Z=0).</i></blockquote>
To read theses tags with the LibTiff.Net lib one needs to use the GetField method. Thus, loading the values from these tags will be as simple as:
<br />
<pre class="prettyprint">FieldValue[] modelPixelScaleTag = tiff.GetField((TiffTag)33550);
FieldValue[] modelTiepointTag = tiff.GetField((TiffTag)33922);
byte[] modelPixelScale = modelPixelScaleTag[1].GetBytes();
double pixelSizeX = BitConverter.ToDouble(modelPixelScale, 0);
double pixelSizeY = BitConverter.ToDouble(modelPixelScale, 8)*-1;
byte[] modelTransformation = modelTiepointTag[1].GetBytes();
double originLon = BitConverter.ToDouble(modelTransformation, 24);
double originLat = BitConverter.ToDouble(modelTransformation, 32);
</pre>
With this information I'm mostly ready to iterate the various raster lines. But, and although conceptually the top-left corner corresponds to coordinate -180, 90 the top-left pixel itself corresponds to coordinate -179.99166, 89.99166. This is obtained through:
<br />
<pre class="prettyprint">double startLat = originLat + (pixelSizeY/2.0);
double startLon = originLon + (pixelSizeX/2.0);</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMBgGzdxd4LGOYR8Mv7M6t9a7s1pm6fI36brx6HQMs2B_yy1c3iRje1IyrQYQb5iy6ziRlcLRAW4OC1l9cJk16u4w-90UGsH7at9vvvYSAtDadcxvlujX_Zdr27eYIE-18YP8tJj1vKb4/s1600/64.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgMBgGzdxd4LGOYR8Mv7M6t9a7s1pm6fI36brx6HQMs2B_yy1c3iRje1IyrQYQb5iy6ziRlcLRAW4OC1l9cJk16u4w-90UGsH7at9vvvYSAtDadcxvlujX_Zdr27eYIE-18YP8tJj1vKb4/s1600/64.png" height="350" width="400" /></a></div>
<br />
So here's the full source-code.<br />
<u>Disclaimer: This is mostly coded for my particular scenario as GeoTiff supports a wider-range of options</u>.
<br />
<pre class="prettyprint">using (Tiff tiff = Tiff.Open(filePath, "r"))
{
int height = tiff.GetField(TiffTag.IMAGELENGTH)[0].ToInt();
FieldValue[] modelPixelScaleTag = tiff.GetField((TiffTag)33550);
FieldValue[] modelTiepointTag = tiff.GetField((TiffTag)33922);
byte[] modelPixelScale = modelPixelScaleTag[1].GetBytes();
double pixelSizeX = BitConverter.ToDouble(modelPixelScale, 0);
double pixelSizeY = BitConverter.ToDouble(modelPixelScale, 8)*-1;
byte[] modelTransformation = modelTiepointTag[1].GetBytes();
double originLon = BitConverter.ToDouble(modelTransformation, 24);
double originLat = BitConverter.ToDouble(modelTransformation, 32);
double startLat = originLat + (pixelSizeY/2.0);
double startLon = originLon + (pixelSizeX / 2.0);
var scanline = new byte[tiff.ScanlineSize()];
//TODO: Check if band is stored in 1 byte or 2 bytes.
//If 2, the following code would be required
//var scanline16Bit = new ushort[tiff.ScanlineSize() / 2];
//Buffer.BlockCopy(scanline, 0, scanline16Bit, 0, scanline.Length);
double currentLat = startLat;
double currentLon = startLon;
for (int i = 0; i < height; i++)
{
tiff.ReadScanline(scanline, i); //Loading ith Line
var latitude = currentLat + (pixelSizeY * i);
for (var j = 0; j < scanline.Length; j++)
{
var longitude = currentLon + (pixelSizeX*j);
geodata.Points[0] = new[] { new PointXY(longitude, latitude) };
object value = scanline[j];
//... process each data item
yield return dataItem;
}
}
}
</pre>
This code is actually working quite well. Eventually could be interesting to put this onto a proper lib and add support for additional GeoTiff features. Adding that to my backlog :)Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com11tag:blogger.com,1999:blog-6029629494508626591.post-88293940063083136082014-12-10T21:12:00.000+00:002014-12-11T08:13:16.214+00:00Splitting vector and raster files in QGIS programmaticallyOn the context of the game I'm developing I have to load gigantic .shp/.tif files and parse them to hexagons.<br />
<br />
The problem is that this operation is executed in memory (because I need to consolidate/merge data) and the process fails with large files. I had two options:<br />
<ul>
<li>Change the process so that it would store the preliminary results on a separate datastore</li>
</ul>
<div>
<ul>
<li>Instead of loading full shapefiles or raster images, split them into smaller chunks</li>
</ul>
<div>
I did try the first option and, although working, became so much slower (like 100x) that I had to discard it (or at least shelve it while I searched for a better alternative).<br />
<br />
I also tried the smaller chunks approach and started by creating "packages" per Country, manually picking data from sources such as <a href="http://download.geofabrik.de/europe.html">Geofabrik</a>.</div>
</div>
<div>
<br /></div>
<div>
But this posed two problems:</div>
<div>
<ul><ul>
<li>Very tedious work, particularly as there are hundreds of countries.</li>
<li>Wouldn't work for larger countries, hitting the memory roadblock as well.</li>
</ul>
</ul>
<div>
So I opted to split the files in a grid like manner. I decided to use <a href="http://www.qgis.org/en/site/">QGIS</a> as it provides all the required tooling. </div>
</div>
<div>
<br /></div>
<div>
Doing this splitting is quite easy to do manually:</div>
<div>
<br /></div>
<div>
1. Open the shapefile (For this example I'm using <a href="http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/cultural/ne_10m_admin_0_countries.zip">Countries</a> data from <a href="http://www.naturalearthdata.com/">NaturalEarth</a>).<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEharAKvxA1vZm7v9FxRCDjMMnocXDahDIfq5Q-N5oqC1viwPmdKUemYrqh2ZWsurXgPDbcGz-kY2P-WPgDG0jsOBwn4i3GErQ-jzuIHbHPoTKxKgXtlGvTGpzPt_zRUkfnen10IV2LCMVA/s1600/60.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEharAKvxA1vZm7v9FxRCDjMMnocXDahDIfq5Q-N5oqC1viwPmdKUemYrqh2ZWsurXgPDbcGz-kY2P-WPgDG0jsOBwn4i3GErQ-jzuIHbHPoTKxKgXtlGvTGpzPt_zRUkfnen10IV2LCMVA/s1600/60.PNG" /></a><br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
</div>
<div>
2. Generate a grid.<br />
<ul>
<li>Open the toolbox</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5KXQXUtrMJOIaCXSb8Rt80V3MaEbf4KSVBq3bmmuLB5kFF32SokvVnahIlxGKXnQjYUwaKVxsSNTlJytaaa5muO1Hhk6bv-AWOha8kYrASy0QArAQWTJSNA9dNg8RlpKLBMqv9MHXbb4/s1600/52.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5KXQXUtrMJOIaCXSb8Rt80V3MaEbf4KSVBq3bmmuLB5kFF32SokvVnahIlxGKXnQjYUwaKVxsSNTlJytaaa5muO1Hhk6bv-AWOha8kYrASy0QArAQWTJSNA9dNg8RlpKLBMqv9MHXbb4/s1600/52.png" /></a><br />
<ul>
<li>Choose Geoalgorithms > Vector > Creation > Create graticule</li>
</ul>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhj5BhvRn10LTFb7NmcxvchCVo6YQG7-fLQYGntgN9djw3jXSwbQGuGROQKTTZofIujwtw10oUXIcqcjq1GZXgsHRH6D9bHsinu9c2E-9Wr32VMmR-LxZ2VX7V2yDiG2b8SsFaMG0VC3os/s1600/53.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhj5BhvRn10LTFb7NmcxvchCVo6YQG7-fLQYGntgN9djw3jXSwbQGuGROQKTTZofIujwtw10oUXIcqcjq1GZXgsHRH6D9bHsinu9c2E-9Wr32VMmR-LxZ2VX7V2yDiG2b8SsFaMG0VC3os/s1600/53.PNG" height="400" width="254" /></a></div>
</div>
<div>
<ul>
<li>Set the desired size (in degrees per rectangle) and set Grid type to <b>Rectangle (polygon)</b></li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSY1q0TFzzdthmATYlKIRyNlisqeK1fLUEXMh1T6SYr-j2QFZfGA99-XafLbR5W5LreIlwyXHa3SIXKVVp4H0pMnFDKinJ3_FwPiO4qs8TJtaqtnq-pRV17ud4drp7jyWn8ipVWKjmrTE/s1600/54.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjSY1q0TFzzdthmATYlKIRyNlisqeK1fLUEXMh1T6SYr-j2QFZfGA99-XafLbR5W5LreIlwyXHa3SIXKVVp4H0pMnFDKinJ3_FwPiO4qs8TJtaqtnq-pRV17ud4drp7jyWn8ipVWKjmrTE/s1600/54.PNG" height="400" width="341" /></a><br />
<ul>
<li>A grid is created over the map (I've changed the opacity of the grid layer to show the map beneath).</li>
</ul>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwippaQgvpkvaiQ_St35_odX2R0HmrlDLu41AVqdQbbf6hx4-HPrHPAJ_ewOX7yBkj-hwrQkswypZLc3G6LIRpJOOH9ZMkq4lYZpI_wf1VkeRVZlDLA9CvP7M1Q9-I7PzE7GwsETt1kms/s1600/55.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhwippaQgvpkvaiQ_St35_odX2R0HmrlDLu41AVqdQbbf6hx4-HPrHPAJ_ewOX7yBkj-hwrQkswypZLc3G6LIRpJOOH9ZMkq4lYZpI_wf1VkeRVZlDLA9CvP7M1Q9-I7PzE7GwsETt1kms/s1600/55.PNG" /></a></div>
<br /></div>
<div>
3. Select one of the rectangles<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVMsLIgvaA1HDnwhkcTxyBTpI9-elimSrgwYE6SKDFznAh0E0bkGVEZ8S5c2lJ7FuYfb6iNecbLbAdLRe3QlaUp1Gwll8t6z_CefWX3-eGOodSVSBTtizfE73qJTSJmmUSORpTAB_MjFo/s1600/56.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVMsLIgvaA1HDnwhkcTxyBTpI9-elimSrgwYE6SKDFznAh0E0bkGVEZ8S5c2lJ7FuYfb6iNecbLbAdLRe3QlaUp1Gwll8t6z_CefWX3-eGOodSVSBTtizfE73qJTSJmmUSORpTAB_MjFo/s1600/56.PNG" height="295" width="400" /></a><br />
<br /></div>
<div>
4. Perform an intersection between both layers<br />
<ul>
<li>Choose Vector > Geoprocessing Tools > Intersect</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqIqmrXiV3lJTgMaAK9g7sQUuIldKTRxwJXt__lJzFsBNXC4nYirVvIf_ogLfPIumQ5zKzBoB-kDZIW0m_VzPgS5CqphsTYiRI9eGDTwNXdgyWsOQt2TQIhbsRL4PpGKoHAIF93Jvdo9o/s1600/57.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqIqmrXiV3lJTgMaAK9g7sQUuIldKTRxwJXt__lJzFsBNXC4nYirVvIf_ogLfPIumQ5zKzBoB-kDZIW0m_VzPgS5CqphsTYiRI9eGDTwNXdgyWsOQt2TQIhbsRL4PpGKoHAIF93Jvdo9o/s1600/57.png" /></a><br />
<ul>
<li>Choose the base layer as input and the rectangle as the intersect layer, using the "Use only selected features" option.</li>
</ul>
<div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjALfQKzMLIRJFhfZ0Y00JxNVTSV3NBFkJ6b9eMu0C2tD3FPwggQv9fp6ECmc3QAbOBGgn3oBiSJKD9HzYIO7xWTf79E6tClkNIebTeqZyPwEFnCgTWDgLDnB21HobTHSM-ciOw99d1dRU/s1600/58.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjALfQKzMLIRJFhfZ0Y00JxNVTSV3NBFkJ6b9eMu0C2tD3FPwggQv9fp6ECmc3QAbOBGgn3oBiSJKD9HzYIO7xWTf79E6tClkNIebTeqZyPwEFnCgTWDgLDnB21HobTHSM-ciOw99d1dRU/s1600/58.PNG" height="226" width="320" /></a></div>
<div>
<ul>
<li>After executing a new layer is created with the intersecting polygons</li>
</ul>
</div>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_sXpvMx8tqS9VxuTrb_OOV5R53Q6N8rGYu5VNRJ_mklBNGh5Hm7QSNINBeOByp1Mec0FuTxLvM1Ps15WgBWqPh4EMmrP2zidAHkqf4wbidGbs-Nt_2D4Mqfn9oxeN0yKdyek9qrBp3E8/s1600/59.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_sXpvMx8tqS9VxuTrb_OOV5R53Q6N8rGYu5VNRJ_mklBNGh5Hm7QSNINBeOByp1Mec0FuTxLvM1Ps15WgBWqPh4EMmrP2zidAHkqf4wbidGbs-Nt_2D4Mqfn9oxeN0yKdyek9qrBp3E8/s1600/59.PNG" height="315" width="400" /></a><br />
<br /></div>
<div>
Now I would just need to repeat this for all the 648 grid items x number of layers to process. I'm assuming this would take about 1 minute per each one and about 10 layers. So, approximately 108 hours non-stop... Not going to happen :). So, what about automating this inside QGIS?<br />
<br /></div>
<div>
QGIS does provide a command-line/editor/plugins to programmatically leverage its functionalities. Unfortunately for me, it's in Python, which I had never used before. Regardless, the optimistic in me jumped at the opportunity to learn something new.<br />
<br />
So, here it is, a sample Python script for QGIS that basically mimics the manual steps I did above:<br />
<ul>
<li>Generates a grid (size hardcoded)</li>
<li>Iterates the various tiles in the grid</li>
<ul>
<li>Iterates all layers currently visible (both raster and vector)</li>
<ul>
<li>Outputs the intersection between the layer and the tile</li>
<li>Creates a subfolder (harcoded) with all the intersected layers for each tile.</li>
</ul>
</ul>
</ul>
<ul>
<ul>
</ul>
</ul>
Complete code (<u>update: there's a newer implementation on the bottom</u>)<br />
<br />
<pre class="prettyprint"><span style="font-size: x-small;">import processing
import os
#Create a GRID
result = processing.runalg("qgis:creategrid", 1, 360, 180, 10, 10, 0, 0, "epsg:4326", None)
#Add it to the canvas
gridLayer = iface.addVectorLayer(result.get("OUTPUT"),"grid","ogr")
#Iterate ever square on the grid
i = 0
for square in gridLayer.getFeatures():
i = i + 1
#Create a new layer
newSquareLayer = iface.addVectorLayer("Polygon?crs=epsg:4326", "temporary_polygon_" + str(i), "memory")
provider = newSquareLayer.dataProvider()
#feature that simply holds one square
newSquare = QgsFeature()
newSquare.setGeometry( QgsGeometry.fromPolygon(square.geometry().asPolygon()))
provider.addFeatures( [ newSquare ] )
#Make sure the target folder exists
folder = "c:\\temp\\grid\\grid_" + str(i)
if not os.path.exists(folder):
os.makedirs(folder)
#iterate the various layers except the grid
for mapLayer in iface.mapCanvas().layers():
layerType = mapLayer.type()
layerName = mapLayer.name()
intersectionName = "intersection_" + layerName + "_" + str(i)
#vector layers and raster layers are processed differently
if layerType == QgsMapLayer.VectorLayer and layerName != "grid":
#Calculate the intersection between the specific grid rectangle and the layer
intersection = processing.runalg("qgis:intersection", mapLayer, newSquareLayer, None)
iface.addVectorLayer(intersection.get("OUTPUT"),intersectionName,"ogr")
#create a shapefile for this new intersection layer on the filesystem.
#A separate folder will be added for each square
intersectionLayer = QgsMapLayerRegistry.instance().mapLayersByName(intersectionName)[0]
QgsVectorFileWriter.writeAsVectorFormat(
intersectionLayer,
folder + "\\" + layerName + ".shp",
"utf-8",
QgsCoordinateReferenceSystem(4326),
"ESRI Shapefile")
#remove the intersection layer from the canvas
QgsMapLayerRegistry.instance().removeMapLayers( [intersectionLayer.id()] )
elif layerType == QgsMapLayer.RasterLayer:
#Calculate the intersection between the specific grid rectangle and the raster layer
intersection = processing.runalg('saga:clipgridwithpolygon', mapLayer, newSquareLayer, None)
#add the intersection to the map
iface.addRasterLayer(intersection.get("OUTPUT"), intersectionName)
#export to file
intersectionLayer = QgsMapLayerRegistry.instance().mapLayersByName(intersectionName)[0]
pipe = QgsRasterPipe()
provider = intersectionLayer.dataProvider()
pipe.set(provider.clone())
rasterWriter = QgsRasterFileWriter(folder + "\\" + layerName + ".tif")
xSize = provider.xSize()
ySize = provider.ySize()
rasterWriter.writeRaster(pipe, xSize, ySize, provider.extent(), provider.crs())
#remove the intersection layer from the canvas
QgsMapLayerRegistry.instance().removeMapLayers( [intersectionLayer.id()] )
else:
print "layer type not supported"
#Now that all the intersections have been calculated remove the new square
print "Removing temporary grid item " + str(i) + "(" + newSquareLayer.id() + ")"
QgsMapLayerRegistry.instance().removeMapLayers( [newSquareLayer.id()] )
#Remove the grid
QgsMapLayerRegistry.instance().removeMapLayers( [gridLayer.id()] )</span></pre>
</div>
To use this script:<br />
<ul>
<li>Open the Python console:</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzdTFO_E0mPEJHN26JLN41oGUfYEybkobnje13iQdWvAA2A6rwE1Cey0hk1QtLhK5H5EdD1SojHNe36cAUCgOHAKb-2O0njAKxts6I_G91pPsMYdJ-TtoeLKSdwOR2Fi2hu3Tl4xTuE48/s1600/61.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhzdTFO_E0mPEJHN26JLN41oGUfYEybkobnje13iQdWvAA2A6rwE1Cey0hk1QtLhK5H5EdD1SojHNe36cAUCgOHAKb-2O0njAKxts6I_G91pPsMYdJ-TtoeLKSdwOR2Fi2hu3Tl4xTuE48/s1600/61.png" /></a></div>
<ol>
<li>Open the editor and copy&paste the script there</li>
<li>Save the script</li>
<li>Execute it</li>
</ol>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcKG73Fygcn-Yju97p_GLV07OJBqkyUHBXpHzeoCCfqmanunV6I1Fl5450clpPUxLMmv7onvLnMZud4iKNgsA4llX4kyp710wNYSs5mZT3EGjHEhgWzXZeRWS-bR0SF_iGuw_6CKyz6z0/s1600/62.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhcKG73Fygcn-Yju97p_GLV07OJBqkyUHBXpHzeoCCfqmanunV6I1Fl5450clpPUxLMmv7onvLnMZud4iKNgsA4llX4kyp710wNYSs5mZT3EGjHEhgWzXZeRWS-bR0SF_iGuw_6CKyz6z0/s1600/62.png" height="236" width="640" /></a></div>
<br />
Could take a couple of minutes for large files, particularly the raster ones, but at least it's automatic.<br />
<br />
<b><br /></b>
<b>Update (11/12/2014)</b><br />
<br />
Although the above script worked as planned, the import process didn't, particularly on the data between the generated tiles. As there's no overlap on the clipped data, after processing small gaps would appear on the end-result. Thus, I've created a brand new implementation that is a little bit more polished and supports a new "buffer" parameter. This allows the tiles to overlap slightly like:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPtlB8wJDYA2i87FAHe7CMC1kJ6nFqElXYYNdo1IiSiL9AlJmbUPEsxjK8CtZOsCptxwkKNzlikXsHj4n4welW20BaDB8oIRWfvHlciPxw-PW3Pqr4BkEdJCQYMunHQ6JCGdWdJPqgnpY/s1600/63.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgPtlB8wJDYA2i87FAHe7CMC1kJ6nFqElXYYNdo1IiSiL9AlJmbUPEsxjK8CtZOsCptxwkKNzlikXsHj4n4welW20BaDB8oIRWfvHlciPxw-PW3Pqr4BkEdJCQYMunHQ6JCGdWdJPqgnpY/s1600/63.png" /></a></div>
Also, the grid is now created programmatically without using "creategrid" function, which also allowed me to use a more logical X,Y naming for the tiles.<br />
<br />
The new code is:
<br />
<pre class="prettyprint"><span style="font-size: x-small;">import processing
import os
####### PARAMS #######
originX = -180
originY = 90
stepX = 10
stepY = 10
width = 360
height = 180
iterationsX = width / stepX
iterationsY = height / stepY
buffer = 1
j = 0
i = 0
targetBaseFolder = "C:\\temp\\grid"
####### MAIN #######
for i in xrange(0,iterationsX):
for j in xrange(0,iterationsY):
tileId = str(i) + "_" + str(j)
folder = targetBaseFolder + "\\" + tileId
if not os.path.exists(folder):
os.makedirs(folder)
print "Processing tile " + tileId
minX = (originX + i * stepX) - buffer
maxY = (originY - j * stepY) + buffer
maxX = (minX + stepX) + buffer
minY = (maxY - stepY) - buffer
wkt = "POLYGON ((" + str(minX) + " " + str(maxY)+ ", " + str(maxX) + " " + str(maxY) + ", " + str(maxX) + " " + str(minY) + ", " + str(minX) + " " + str(minY) + ", " + str(minX) + " " + str(maxY) + "))"
tileLayer = iface.addVectorLayer("Polygon?crs=epsg:4326", "tile", "memory")
provider = tileLayer.dataProvider()
tileFeature = QgsFeature()
tileFeature.setGeometry(QgsGeometry.fromWkt(wkt))
provider.addFeatures( [ tileFeature ] )
for mapLayer in iface.mapCanvas().layers():
layerType = mapLayer.type()
layerName = mapLayer.name()
intersectionName = "intersection_" + layerName + "_" + tileId
#vector layers and raster layers are processed differently
if layerType == QgsMapLayer.VectorLayer and layerName != "tile":
#Calculate the intersection between the specific grid rectangle and the layer
intersection = processing.runalg("qgis:intersection", mapLayer, tileLayer, None)
iface.addVectorLayer(intersection.get("OUTPUT"),intersectionName,"ogr")
#create a shapefile for this new intersection layer on the filesystem.
#A separate folder will be added for each square
intersectionLayer = QgsMapLayerRegistry.instance().mapLayersByName(intersectionName)[0]
QgsVectorFileWriter.writeAsVectorFormat(
intersectionLayer,
folder + "\\" + layerName + ".shp",
"utf-8", QgsCoordinateReferenceSystem(4326),
"ESRI Shapefile")
#remove the intersection layer from the canvas
QgsMapLayerRegistry.instance().removeMapLayers( [intersectionLayer.id()] )
elif layerType == QgsMapLayer.RasterLayer:
#Calculate the intersection between the specific grid rectangle and the raster layer
intersection = processing.runalg('saga:clipgridwithpolygon', mapLayer, tileLayer, None)
#add the intersection to the map
iface.addRasterLayer(intersection.get("OUTPUT"), intersectionName)
#export to file
intersectionLayer = QgsMapLayerRegistry.instance().mapLayersByName(intersectionName)[0]
pipe = QgsRasterPipe()
provider = intersectionLayer.dataProvider()
pipe.set(provider.clone())
rasterWriter = QgsRasterFileWriter(folder + "\\" + layerName + ".tif")
xSize = provider.xSize()
ySize = provider.ySize()
rasterWriter.writeRaster(pipe, xSize, ySize, provider.extent(), provider.crs())
#remove the intersection layer from the canvas
QgsMapLayerRegistry.instance().removeMapLayers( [intersectionLayer.id()] )
else:
print "layer type not supported"
#remove the temporary tile
QgsMapLayerRegistry.instance().removeMapLayers( [tileLayer.id()] )</span>
</pre>
Eventually I'll create a proper QGIS plugin for this that allows setting input parameters without all that hardcoded logic. <br />
<ul>
</ul>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com4tag:blogger.com,1999:blog-6029629494508626591.post-87867396984434359792014-11-11T23:45:00.001+00:002015-03-16T19:20:43.668+00:00Game-development Log (11. Persistence and Authentication) <div style="border: 1px solid gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;">11. Persistency and Authentication</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
On this new iteration the units now have owners. A user can only move his own units (represented in blue) and won't be able to change the other player's units (represented in red).<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA6l42DJsXVoEu5xBvQi6AVmqQ6oBuTxo5o1QkAcd7trR-baVj0nqrG5zryf54RW4g0XezxbfRyybw2_4DQQfywRYc0Ab4Kl2JYjLXugnwi5sOhG3bzmNVYIaMJC8OoS2FHH3l-Uftg7k/s1600/46.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhA6l42DJsXVoEu5xBvQi6AVmqQ6oBuTxo5o1QkAcd7trR-baVj0nqrG5zryf54RW4g0XezxbfRyybw2_4DQQfywRYc0Ab4Kl2JYjLXugnwi5sOhG3bzmNVYIaMJC8OoS2FHH3l-Uftg7k/s1600/46.png" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<b>Authentication</b><br />
<b><br /></b>I've used the default template that comes with ASP.NET MVC 5. It includes most of the logic to create and manage users, integrate with Facebook/Google/Twitter, including persistence support with Entity Framework.<br />
<br />
I did have to tweak it but most was actually pretty simple. Here's the end-result:<br />
<ul>
<li>I've added register/log in buttons at the top.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhj3v8gdF0zD6ku5Agw7rNFtEceGnMlrVM-U2FBZD4TO0ZlzdOiDxMKY9LwSq1j7822NjZwMrElXlr7ufID7RZXW30ytq5cgSJrNvZvIQFl5v0nAg_xj5t4XxFNfmZjgN8fVaJ8rqm1k_g/s1600/47.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhj3v8gdF0zD6ku5Agw7rNFtEceGnMlrVM-U2FBZD4TO0ZlzdOiDxMKY9LwSq1j7822NjZwMrElXlr7ufID7RZXW30ytq5cgSJrNvZvIQFl5v0nAg_xj5t4XxFNfmZjgN8fVaJ8rqm1k_g/s1600/47.PNG" height="40" width="640" /></a></div>
<br />
<ul>
<li>After Log-in/Register the username is displayed, included a dropdown menu with additional options. The "Profile" button opens a new page to link between local and external accounts and to change the password.</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ2Q6H2JG6nl1Yrn9sb3zVCKQbid-Xcr_MghVT79flmEBZVreEwRS5ENbFVvftqu8j7D38nVE3NCEfj0zbNHuQRPLorVobPvnt9xlNf_Smyz9ACxDAmoAp08LdPCXYIe1s7CPEcvTzp_E/s1600/49.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgJ2Q6H2JG6nl1Yrn9sb3zVCKQbid-Xcr_MghVT79flmEBZVreEwRS5ENbFVvftqu8j7D38nVE3NCEfj0zbNHuQRPLorVobPvnt9xlNf_Smyz9ACxDAmoAp08LdPCXYIe1s7CPEcvTzp_E/s1600/49.PNG" height="234" width="320" /></a></div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgraOvTyuhWGxjNpcnRA50P85tIUXKN-GqJvigv__5ppc5dme8tcyR-dywwLB4jCELwY0qhYLev4QrFIGKqCkZefuzVMrhi1Z2M5RU2VKwglUguNzZr75et5POiGXybnezv_-eUT3qou4U/s1600/49.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgraOvTyuhWGxjNpcnRA50P85tIUXKN-GqJvigv__5ppc5dme8tcyR-dywwLB4jCELwY0qhYLev4QrFIGKqCkZefuzVMrhi1Z2M5RU2VKwglUguNzZr75et5POiGXybnezv_-eUT3qou4U/s1600/49.PNG" /></a></div>
The aesthetic still needs tons of work, but at least most of the functionality is there. Currently I only allow local users, Facebook and Google.<br />
<b><br /></b>
<b>Persistence</b><br />
<br />
The unit movement is now persisted on a SQL Server database at Azure, using Entity Framework. To validate it just move an unit and refresh the browser.<br />
<br />
<br />
<b>Add Units</b><br />
<b><br /></b>
I've added a "Developer" menu. Currently it allows units to be created on the map.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwnMPp5_UETBKIUDI585_3fOY_vzSvwwoKQj-Ytq4_sCSKL_NphN_tv2rbzqZnA77lx_oI0wcVDufqkdWYppto0j4uVKuB8zhbYBjgHrkFU8u8y0U9NDudkzV9-DMyt5_xmNSDUJiKglk/s1600/50.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjwnMPp5_UETBKIUDI585_3fOY_vzSvwwoKQj-Ytq4_sCSKL_NphN_tv2rbzqZnA77lx_oI0wcVDufqkdWYppto0j4uVKuB8zhbYBjgHrkFU8u8y0U9NDudkzV9-DMyt5_xmNSDUJiKglk/s1600/50.PNG" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5cEyz9k-WSmQRLgd1tHNLfTU_5-DZViowYdL5k_-JFgMHiW6O9WX5xtXN_eDji6pYYCk_2wc3uo9oAKfBh6hHEbDVlyFnfskkpWQQOW-umD9isws3iyXefMbBxNXhJk8M36BQWaXtgFM/s1600/51.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="212" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5cEyz9k-WSmQRLgd1tHNLfTU_5-DZViowYdL5k_-JFgMHiW6O9WX5xtXN_eDji6pYYCk_2wc3uo9oAKfBh6hHEbDVlyFnfskkpWQQOW-umD9isws3iyXefMbBxNXhJk8M36BQWaXtgFM/s640/51.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<b>HTTPS (and CDN)</b><br />
<b><br /></b>
As I've added authentication to the site I've changed everything from HTTP to HTTPS. Unfortunately, although Azure's CDN supports HTTPS endpoints, it gets much slower than its HTTP counterpart. Also, I was getting random 503 responses.<br />
<br />
So, for now, I've removed my usage of Azure's CDN altogether, either pointing to the blob storage for the static image tiles or to the Tile-Service webAPI for the dynamic ones.<br />
<br />
I really love Azure, but its CDN really sucks, particularly comparing against the likes of Amazon, Cloudfront, Akamai, Level3, etc.Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-67234324743336556282014-10-26T23:52:00.001+00:002015-03-16T19:20:54.793+00:00Game-development Log (10. Real-time with SignalR)<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attacks and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;">10. Real-time with SignalR</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net/ </a><br />
<br />
On this iteration I'm adding real-time notifications to the unit movement using the (awesome) <a href="http://signalr.net/">SignalR</a> library.<br />
<br />
The idea for this will be simple: if a player moves a unit on his browser window all the others players will see that unit move in realtime on their browsers.<br />
<br />
I've created an ultra-simple video demoing it. I've opened three separate browser windows: one with Chrome, Firefox and Safari. The movement of the units should be synchronised in real-time across the various browsers, regardless of the one that triggered the movement.<br />
<br />
<iframe allowfullscreen="" frameborder="0" height="360" src="//www.youtube.com/embed/HugFvmMqM-s" width="640"></iframe>
<br />
So, how was this setup? Easy as pie.<br />
<br />
1. Add SignalR NuGet package
<br />
<pre class="prettyprint"><span style="font-family: Times; white-space: normal;">install-package Microsoft.AspNet.SignalR</span>
</pre>
<br />
2. Initialise SignalR on the Startup routine of ASP.NET MVC 5 (on previous versions it's slightly different)
<br />
<pre class="prettyprint">public partial class Startup
{
public void Configuration(IAppBuilder app)
{
app.MapSignalR();
}
}
</pre>
3. Create a Hub<br />
My hub will be ultra simple. It will simply receive a movement notification and share it with all the other SignalR clients as is.
<br />
<pre class="prettyprint">public class UnitHub : Hub
{
public void SetPosition(dynamic unitdata)
{
Clients.Others.unitUpdated(unitdata);
}
}
</pre>
<br />
4. Create Javascript code both to send a SignalR message when the player moves a unit or when a notification of another player moving a unit has been received.<br />
<pre class="prettyprint">//code to receive notification from server
unitUpdated = function (unit) {
// (...)
};
// Code to submit to server
server.setPosition(unit);
</pre>
5. (Optional) If you're using Azure, you should enable WebSockets on your website.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv8mjB7AU6Lhd_Ym7IMpqiDgvSIRZyAwY3mOpJRhATwhztoDL0vOh_9RrA_5E-uyAf2yQWaafEyoLLPZN80iqkV-DO7zZJmTSqAjGvmgdNNQIKDJKIj9-4Qqbc576BAJtMEMYROSOzMMQ/s1600/45.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv8mjB7AU6Lhd_Ym7IMpqiDgvSIRZyAwY3mOpJRhATwhztoDL0vOh_9RrA_5E-uyAf2yQWaafEyoLLPZN80iqkV-DO7zZJmTSqAjGvmgdNNQIKDJKIj9-4Qqbc576BAJtMEMYROSOzMMQ/s1600/45.png" height="90" width="400" /></a></div>
And that's it.<br />
<br />
Please note that I haven't yet added persistence nor support for different users. Thus, everyone will be moving the same units. Anyway, I don't have that many page views on my blog for it to be a problem :)<br />
<br />
So, what's next?<br />
- Adding a database to persist the units position<br />
- Adding users, each with his own units.<br />
- Authentication with Facebook, Twitter, Google, Windows, EmailPedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com3tag:blogger.com,1999:blog-6029629494508626591.post-68884291606742561562014-10-23T20:13:00.002+01:002015-03-16T19:21:04.968+00:00Game-development Log (9. "Bootstraping")<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance Checkpoint</a></span><br />
<span style="font-size: x-small;">9. Bootstraping</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
This will be a short one. I'm basically picking up my map page and adding a bootstrap template around it.<br />
<br />
I'm not tweaking it too much, just adding a couple of navigation buttons, some modals and some cosmetic details (like a subtle transparency on the navigation bar).<br />
<br />
Without Bootstrap:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXcQ1aKbtDBr_ogdgZFBKiBxiWy3RO1SAIlOdlEE4D_iFkiQv6hc6HzmQBIwciV-ofh__N4qYTbkjDX_r0_NZTyAVLLfaMN4v7oG9sH6zGAvbcfK6mq8QCinapqfxZMEdA22kysU2v12U/s1600/42.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXcQ1aKbtDBr_ogdgZFBKiBxiWy3RO1SAIlOdlEE4D_iFkiQv6hc6HzmQBIwciV-ofh__N4qYTbkjDX_r0_NZTyAVLLfaMN4v7oG9sH6zGAvbcfK6mq8QCinapqfxZMEdA22kysU2v12U/s1600/42.PNG" height="500" width="640" /></a></div>
<br />
<br />
With Bootstrap:<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtrzkMNPWAYrQtrtxj-XyxW_cUOu0_L_RHWNv2Dprdx2qqGCXfADNHONGl1xsadEzxJGw42mp7QQoyfncjd_axLur5wZQ8zxDJGExYxsrZnRvR689HIpUKQULUAFkXJvGQ3EwkKOi7p7s/s1600/41.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgtrzkMNPWAYrQtrtxj-XyxW_cUOu0_L_RHWNv2Dprdx2qqGCXfADNHONGl1xsadEzxJGw42mp7QQoyfncjd_axLur5wZQ8zxDJGExYxsrZnRvR689HIpUKQULUAFkXJvGQ3EwkKOi7p7s/s1600/41.PNG" height="500" width="640" /></a></div>
<br />
Some buttons on the navbar already open some modals but it's mostly a placeholder for the functionality, like the Sign in one.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiLZYtCgLkNBoN8ywEKBEINsh5HCX0w_YCSCtf02kHOqZUSEsFH6yBqQbMaSmC6rdg51z84YSBP0O3VNtR1_CrDjfRmmpXi-VyZmgl4IqLR0Pu1BkymfeFUo4AOriwEWNefzSgM9fQBEQ/s1600/44.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiLZYtCgLkNBoN8ywEKBEINsh5HCX0w_YCSCtf02kHOqZUSEsFH6yBqQbMaSmC6rdg51z84YSBP0O3VNtR1_CrDjfRmmpXi-VyZmgl4IqLR0Pu1BkymfeFUo4AOriwEWNefzSgM9fQBEQ/s1600/44.PNG" height="294" width="400" /></a></div>
<br />
<br />
Also, Boostrap provides, out-of-the-box, responsive design. Thus the mobile-like display looks like:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrtUIFdjpEl52SefJ8kpN1qmeV0kLTu51tFV6qhpr6t3R0VuszeUMLt4kItyayMc8eujHQyQsU1W5tarX8MJ-tv9jDNa1VHiwgntGrkkjbxS3wVk97ByBQB8QZlwqzCAOxGCe9cvHujX0/s1600/43.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrtUIFdjpEl52SefJ8kpN1qmeV0kLTu51tFV6qhpr6t3R0VuszeUMLt4kItyayMc8eujHQyQsU1W5tarX8MJ-tv9jDNa1VHiwgntGrkkjbxS3wVk97ByBQB8QZlwqzCAOxGCe9cvHujX0/s1600/43.PNG" height="400" width="247" /></a></div>
<br />
So, nothing too fancy but I really love the "cleaness" that Bootstrap provides. I still need to tweak the UI a little bit but I'm not expecting it to change that much.Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-63569761441019578352014-10-19T14:12:00.001+01:002015-03-16T19:21:14.917+00:00Game-development Log (8. Performance Checkpoint - I)<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-7-attack-and.html">7. Attack and Explosions</a></span><br />
<span style="font-size: x-small;">8. Performance Checkpoint</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
I don't want to have my full architecture set up only to realise that something isn't really scalable or performs sub-par. Thus, time for a performance checkpoint where I'm reviewing some of my bottlenecks and try to fix them.<br />
<br />
<b><u>Zoomed-out tile images</u></b><br />
<br />
Currently my image tiles are generated dynamically per request and stored on the Azure CDN. Although this is quite interesting from a storage point-of-view/setup time it has a drawback: the time it takes to generate an image on the furthest away zoom levels, particularly for the unlucky users that get cache misses on the CDN.<br />
<br />
The reason is quite-simple: a tile at zoom level 7 includes about 16.000 individual hexagons, requiring some time (like a couple of seconds) to render. For comparison, a tile at zoom level 12 includes about 30 hexagons and is blazing fast to render.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7drtjmaHsEdHEL3ExiVnI5Hw_4jIlAPN4NcMy6yF4OpqCR_2LLuvNMbHF8QLuKZ7hgymb0A3RK1187K7zmseik7oVwE4X53hvT-PtSc1-Pr3eL51OOQalq4sLdu7HEwkhsLgfsksEqes/s1600/40.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7drtjmaHsEdHEL3ExiVnI5Hw_4jIlAPN4NcMy6yF4OpqCR_2LLuvNMbHF8QLuKZ7hgymb0A3RK1187K7zmseik7oVwE4X53hvT-PtSc1-Pr3eL51OOQalq4sLdu7HEwkhsLgfsksEqes/s1600/40.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRu5lgfMHoUc-_LXTuapsmRTdqOUFMxi_F4QE7y8wzy-4W1u3HPYpMNLIVfiFV1JxyEDXvUrk3GI0IUlJ1XHsDKsEOJtweBsz1pYSK3oz2sTwX_9kRRnqBCUs5XtKXrvzI2ROC4sJHXP0/s1600/41.jpeg" imageanchor="1"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRu5lgfMHoUc-_LXTuapsmRTdqOUFMxi_F4QE7y8wzy-4W1u3HPYpMNLIVfiFV1JxyEDXvUrk3GI0IUlJ1XHsDKsEOJtweBsz1pYSK3oz2sTwX_9kRRnqBCUs5XtKXrvzI2ROC4sJHXP0/s1600/41.jpeg" /></a></div>
<br />
Pre-generating everything is not an option so I opted for an hybrid-approach:<br />
<ul>
<li>Between zoom levels 7 and 10 (configurable) I generate the tile images and store them on Azure's blob storage with a CDN fetching from it.</li>
<li>After 11+ they're generated dynamically and stored on the CDN (as they were)</li>
</ul>
So in practice I now have two different urls on my map:<br />
<ul>
<li>Dynamic url: <a href="http://az665126.vo.msecnd.net/?z={z}&x={x}&y={y}&type=jpg">http://az665126.vo.msecnd.net/?z={z}<span class="s2">&</span>x={x}<span class="s2">&</span>y={y}<span class="s2">&</span>type=jpg</a></li>
<li>Static url: <a href="http://az679112.vo.msecnd.net/imagetiles/%7Bquadkey%7D.jpg">http://az679112.vo.msecnd.net/imagetiles/{quadkey}.jpg</a></li>
</ul>
So the question is: how many tiles will I need to pre-generate?<br />
<br />
The math is quite straight-forward:<br />
<br />
<pre class="prettyprint">tiles for zoom level n = 4^n
tiles 7 = 4^7 = 16.384
tiles 8 = 4^8 = 65.536
tiles 9 = 4^9 = 262.144
tiles 10 = 4^10 = 1.048.576
total = 1.392.640
</pre>
<br />
As I only generate tiles with data, and considering that 2/3 of the planet is water, the approximate number of hexagons generated would be:<br />
<br />
approximate number = total * 0.33 ~ 460.000 images<br />
<br />
Not too bad and completely viable, particularly as I don't expect to be re-generating these images very often.<br />
<br />
<b><u>Vector-Load to support unit movement</u></b><br />
<b><br /></b>
Something that I was noticing is that panning the map view on higher zoom levels was a little bit sluggish, particularly as it was loading the vector tiles with the hexagon data in order to calculate the movement options of the units.<br />
<br />
I was loading this info per-tile during panning, which was blocking the UI thread. A colleague of mine suggested me to try something else: <a href="http://www.w3schools.com/html/html5_webworkers.asp">HTML5 Web Workers</a>. You basically spawn a worker thread that is able to do some work (albeit with some limitations, like not being able to access the DOM) and using messaging to communicate with the main UI thread.<br />
<br />
The implementation was really straightforward. Unfortunately I didn't really notice any performance improvement. Anyway, Web Workers could be an incredibly viable alternative if I ever switch to a client-based drawing logic instead of completely server-side. I've added an entry to my backlog for some WebGL experiments with this :)<br />
<br />
I then had a different idea: instead of loading the hexagon-data tile by tile make a single request that would include the various tiles that compose the current viewport. This is triggered on the "viewchangeend" Bing Maps event, particularly using a <a href="http://msdn.microsoft.com/en-us/library/gg427623.aspx">throttled event handler</a>.<br />
<br />
There was a small performance benefit on this approach and it can be further optimised, particularly leveraging local-storage on the client.<br />
<br />
<b><u>Change from PNG to JPEG</u></b><br />
<br />
At a certain point in time my tiles required transparencies but that's no longer the case. Thus, and although being hit with a small quality downgrade, I've changed my image tiles from PNG to JPEG.<br />
<br />
This has 3 advantages:<br />
<br />
<ul>
<li>Storing the images on the CDN/Blob Storage will be cheaper as the JPEG images are considerably smaller</li>
<li>Latency on loading the image tiles</li>
<li>A more subtle advantage happens on the rendering process, as JPEG is faster to load and process, especially when the PNG counterpart has an alpha-channel.</li>
</ul>
<div>
<br /></div>
<div>
The drawback is, as mentioned, image-quality. Here's a PNG image tile and the corresponding JPEG.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3AYY_JE2e3Eb96TfvDImWF4wUpdBqcz8VfKd4XPePvmB4-dZX4XBNN_3yXx1rYnI_30vhsjdmn-sZ82TJ__LVk6KaDQeaEREl7QlhPNWZClOjiFJQJTp5HFRm4NeyBfbnoIWnRXthODA/s1600/39.jpeg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3AYY_JE2e3Eb96TfvDImWF4wUpdBqcz8VfKd4XPePvmB4-dZX4XBNN_3yXx1rYnI_30vhsjdmn-sZ82TJ__LVk6KaDQeaEREl7QlhPNWZClOjiFJQJTp5HFRm4NeyBfbnoIWnRXthODA/s1600/39.jpeg" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0tcTaeHsJX9IdjrC61Kq5BWLy7_vgwrb_knLxuU2QZXT254T4kXWR5RmgukWQrgUFa33SrDrdhOyfXkgUu91jidr1MMPjnfKnN9D-jiy7AhrpYpfnLzIgYHOzNze7wfgi0mMWn3p3NsI/s1600/38.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0tcTaeHsJX9IdjrC61Kq5BWLy7_vgwrb_knLxuU2QZXT254T4kXWR5RmgukWQrgUFa33SrDrdhOyfXkgUu91jidr1MMPjnfKnN9D-jiy7AhrpYpfnLzIgYHOzNze7wfgi0mMWn3p3NsI/s1600/38.png" /></a></div>
<br />
<div>
Highly compressed and noticeable image-loss but on the map as a whole is mostly negligible, particularly if there's no reference to compare. I'm more worried with performance than top-notch image quality.</div>
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-37267371598808644272014-09-27T19:14:00.000+01:002015-03-16T19:21:25.287+00:00Game-development Log (7. Attack and explosions)<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<span style="font-size: x-small;">7. Attack and Explosions</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html">8. Performance checkpoint</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
On my last post I've added interactivity to the units. On this post I'm taking it further by adding an attack option and corresponding explosions :)<br />
<br />
First of all I've included an extra tank on the map that's not controllable by the player. Hence, a nice target to be attacked (although non-destructible).<br />
<br />
The attack is triggered in the same way as a regular movement, by dragging the unit. When on-top of another unit it changes to an attack cursor, showing additional info.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-3-uhVWArTgbAVSAQnXmtaI_9CSbXJ2he_KfeV0kB27KxhTHkrq4p6BipzxIiyszOt5N9bsFj2gDa6R9OwLNlWRmfoMnMQByxAoyQYBunQJRCpSidTE3AEPndVLGAVHw7crojwL43YA8/s1600/35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-3-uhVWArTgbAVSAQnXmtaI_9CSbXJ2he_KfeV0kB27KxhTHkrq4p6BipzxIiyszOt5N9bsFj2gDa6R9OwLNlWRmfoMnMQByxAoyQYBunQJRCpSidTE3AEPndVLGAVHw7crojwL43YA8/s1600/35.png" /></a></div>
<br />
I've implemented the explosion using the particle lib at <a href="http://scurker.com/projects/particles/">http://scurker.com/projects/particles/</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjsrN8i0dtSiPy-hvqu_iS04jxb2YTJCOyVb2-f9uc3dNX6qem4AaO5tluJHc8G7qpT3exNDTlBXYX-4jOTEszTnto2yO8oVo1GOEMx6tEg3gTV81QxpaQQWjMP2aKJoB6LJnFsVUmTI8/s1600/36.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjsrN8i0dtSiPy-hvqu_iS04jxb2YTJCOyVb2-f9uc3dNX6qem4AaO5tluJHc8G7qpT3exNDTlBXYX-4jOTEszTnto2yO8oVo1GOEMx6tEg3gTV81QxpaQQWjMP2aKJoB6LJnFsVUmTI8/s1600/36.png" height="303" width="400" /></a></div>
<br />
To place the explosion on Bing Maps I've used the pushpin's "htmlContent" property to create a new canvas where the explosion is rendered.<br />
<br />
I've also used a custom html pushpin to display the attack details. For now only the range isn't hardcoded.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja6PjKgkLeuE6A93TAVvjE6GK7gIFKEsFAag7hrJnxf4CUH_4mCMNq9tVYatv7BTZ1TlvygppNfJfEIoyl40RzfbjDtOe7ezcKfj1ctjDOhVkKGsT8elqP2RExqKQIwP99Ww9_dtw3OW0/s1600/37.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEja6PjKgkLeuE6A93TAVvjE6GK7gIFKEsFAag7hrJnxf4CUH_4mCMNq9tVYatv7BTZ1TlvygppNfJfEIoyl40RzfbjDtOe7ezcKfj1ctjDOhVkKGsT8elqP2RExqKQIwP99Ww9_dtw3OW0/s1600/37.png" height="232" width="400" /></a></div>
<br />
And that's it for now.<br />
<br />
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com1tag:blogger.com,1999:blog-6029629494508626591.post-82063880051711714532014-09-24T00:30:00.003+01:002015-03-16T19:21:34.708+00:00Game-development Log (6. Adding units and basic interaction)<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-5-cdn-caching-for.html">5. CDN Caching for map tiles</a></span><br />
<span style="font-size: x-small;">6. Adding Units and Basic Interaction</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-7-attack-and.html">7. Attacks and Explosions</a></span><br />
<a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html"><span style="font-size: x-small;">8. Performance checkpoint</span></a><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
Time to make things more interesting by adding some interactive units to the map. Pretty beefy update that, although highly hardcoded on some places, should lay the foundation for what's to come.<br />
<br />
Here's a small video showcasing what's currently deployed on Azure.<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='640' height='390' src='https://www.youtube.com/embed/8BPne67eY0E?feature=player_embedded' frameborder='0'></iframe></div>
<a name='more'></a><ul>
<li>Loading units</li>
</ul>
<div>
I've created an hardcoded WebAPI that returns game units. Currently its code is simply:</div>
<pre class="prettyprint">public IEnumerable<Unit> Get()
{
return new[]
{
new Unit
{
Id = Guid.NewGuid(),
Type = "TANK",
U = 8295,
V = 1643,
UserId = Guid.NewGuid()
},
new Unit
{
Id = Guid.NewGuid(),
Type = "TANK",
U = 8291,
V = 1640,
UserId = Guid.NewGuid()
},
new Unit
{
Id = Guid.NewGuid(),
Type = "HELICOPTER",
U = 8296,
V = 1641,
UserId = Guid.NewGuid()
},
};
}
</pre>
<div>
<br />
It returns three units: two tanks and one helicopter. I've created custom pushpins to represent both concepts.</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGlTa1e7XpVNPSHRkGHrL9-KEnxFJ-1E1P7maWmBublhiz7uR_ikAtMJ7Lo0cWsFCok2FjS0RZDJxVNjO2LDv_bvKfUAA9mOxiV53Jn4b2shrM6eVts1dgrHSHpOXIvjxEkVIGxJbgwgA/s1600/35.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGlTa1e7XpVNPSHRkGHrL9-KEnxFJ-1E1P7maWmBublhiz7uR_ikAtMJ7Lo0cWsFCok2FjS0RZDJxVNjO2LDv_bvKfUAA9mOxiV53Jn4b2shrM6eVts1dgrHSHpOXIvjxEkVIGxJbgwgA/s1600/35.png" /></a></div>
<br />
My objective is to allow the user to drag these units on the map taking into account the surrounding terrain. Thus I'll have to load the hexagon vector data to the Javascript client.<br />
<ul>
<li>Loading Vector Data to the client</li>
</ul>
<div>
I simply need to load the vector tiles that I talked about so much on my previous posts. This way the vector tiles will serve two separate purposes:</div>
<div>
<ul>
<li>Used as the datasource for generating image-tiles in server-side</li>
<li>Used as terrain info on client-side</li>
</ul>
</div>
<div>
This is implemented in a simple way. I created a dummy Bing Maps tile-layer that, instead of returning images, loads vector data from the tile-server and stores it in memory.<br />
<pre class="prettyprint">var tileSource = new MM.TileSource({
uriConstructor: function getTilePath(tile) {
var z = tile.levelOfDetail;
var x = tile.x;
var y = tile.y;
loadVectorTile(z,x,y);
}
});
var vectorData = new MM.TileLayer({ mercator: tileSource});
map.entities.push(vectorData);
</pre>
<br />
As I was requesting tiles from a different server that doesn't support CORS properly (in this case the CDN) the ajax request was failing, leading me to a cross-domain "duh" facepalm moment. So, JSONP for the rescue.</div>
<div>
<ul>
<li>Adding JSONP support</li>
</ul>
<div>
I've added a new option on the vector tiles routes. If a "callback" parameter is passed (and an optional type=jsonp just for verbosity) the response, instead of being a plain JSON, is returned as JSONP. That's basically wrapping the json inside a function call like:</div>
</div>
<div>
<br /></div>
<div>
function_call_name( {{ original json }})</div>
<br />
ex (from the CDN):<br />
<a href="http://az665126.vo.msecnd.net/?z=12&x=1945&y=1565&type=jsonp&callback=vector">http://az665126.vo.msecnd.net/?z=12&x=1945&y=1565&type=jsonp&callback=vector</a><br />
<br />
In this particular case the data is wrapped on a "vector" function:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifrj2zF8iGAkZwSC0Vc0NIw1IB1jhdD5A2j-ZHFLcwx8JO3ZbRM2__l0JOvw-uHOChNLxjUvC5YnDaZ1DfV5HSm9YUEOsqTrZu4-8mKmB5EB-3jJtP3VCKgDm0G-y1VJyvCIFi98rwdwM/s1600/36.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEifrj2zF8iGAkZwSC0Vc0NIw1IB1jhdD5A2j-ZHFLcwx8JO3ZbRM2__l0JOvw-uHOChNLxjUvC5YnDaZ1DfV5HSm9YUEOsqTrZu4-8mKmB5EB-3jJtP3VCKgDm0G-y1VJyvCIFi98rwdwM/s1600/36.PNG" /></a></div>
<br />
<br />
<br />
<ul>
<li>Movement</li>
</ul>
Now that I have the vector information in client-side I simply register the Bing Maps event handlers on the units and add <strike>some</strike> <b>lots of</b> logic on them.<br />
<br />
While dragging the unit I display the various movement alternatives and the path that's used to reach the destination.<br />
<br />
Tanks and helicopters have different movement capabilities:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDdDU9yqHaubZoG-XNylFUjnaU_Mip651R44NHtDdHrb6huWbR6eMBv4wjjY5cry3_ko1v8yS1EeH75a9572p0uOZZzuSs8Wy0E306MprKD-0k8euPunYcWlTwphEOzNKZNc2QDBpBDfg/s1600/33.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDdDU9yqHaubZoG-XNylFUjnaU_Mip651R44NHtDdHrb6huWbR6eMBv4wjjY5cry3_ko1v8yS1EeH75a9572p0uOZZzuSs8Wy0E306MprKD-0k8euPunYcWlTwphEOzNKZNc2QDBpBDfg/s1600/33.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Tanks can move 4 hexagons on a road and 1 hexagon off-road</td></tr>
</tbody></table>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9kcT9BMDoCVPHZIncza8NHnyIJlOUCXOjuEu2KPRSZFIeAxw2BmJ42Aa9KDXyVRoV7qDXdcOs_53pbN4UMhymyluaQxr67kug9r8m0LyRNlRru9buiMd5UD0DE2GagVmotf5tS9f-9eU/s1600/34.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh9kcT9BMDoCVPHZIncza8NHnyIJlOUCXOjuEu2KPRSZFIeAxw2BmJ42Aa9KDXyVRoV7qDXdcOs_53pbN4UMhymyluaQxr67kug9r8m0LyRNlRru9buiMd5UD0DE2GagVmotf5tS9f-9eU/s1600/34.png" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Helicopters have no movement restrictions and can move 6 hexagons on any direction.</td></tr>
</tbody></table>
Eventually the units will have a cooldown period after moving.<br />
<br />
So, that's it for now. Next stop: probably attack-logic :)<br />
<br />Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0tag:blogger.com,1999:blog-6029629494508626591.post-71443829224583829872014-09-13T12:07:00.000+01:002015-03-16T19:21:45.310+00:00Game-development Log (5. CDN Caching for map tiles)<div style="border: solid 1px gray; padding-left: 15px; width: 300px;">
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-iteration-1-webapi.html">1. Introduction + WebAPI for map tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-2-visual-studio.html">2. Visual Studio Online</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/08/game-development-log-3-loading-data-to.html">3. Loading data to Vector Tiles</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/09/game-development-log-4-rendering-image.html">4. Rendering image tiles</a></span><br />
<span style="font-size: x-small;">5. CDN Caching for map tiles</span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2014/09/game-development-log-6-adding-units-and.html">6. Adding Units and Basic Interaction</a></span><br />
<a href="http://build-failed.blogspot.com/2014/09/game-development-log-7-attack-and.html"><span style="font-size: x-small;">7. Attacks and Explosions</span></a><br />
<a href="http://build-failed.blogspot.pt/2014/10/game-development-log-8-performance.html"><span style="font-size: x-small;">8. Performance checkpoint</span></a><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-9-bootstraping.html">9. Bootstraping</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/10/game-development-log-10-real-time-with.html">10. Real-time with SignalR</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.pt/2014/11/game-development-log-11-persistency-and.html">11. Persistence and Authentication</a></span><br />
<span style="font-size: x-small;"><a href="http://build-failed.blogspot.com/2015/01/game-development-log-12-scaling-up.html">12. Scaling up the loader process</a></span><br />
<a href="http://build-failed.blogspot.pt/2015/03/game-development-log-13-polishing.html"><span style="font-size: x-small;">13. Polishing the experience</span></a></div>
<br />
Work-in-progress: <a href="https://site-win.azurewebsites.net/">https://site-win.azurewebsites.net</a><br />
<br />
The map that was displayed on my previous post was using an Azure Web-Site to render the tiles. On this iteration I've included a <a href="http://en.wikipedia.org/wiki/Content_delivery_network">CDN</a> so that the tiles are cached and the latency is minimal when serving them.<br />
<br />
Generically speaking this is my target architecture including a CDN:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg76H-guE_nP8B7YZsbHMxyNsS-VlIQktcUyyV4xKMaPD6C3zTurCW9bLiFzQQdzVv7RMSqtr9YQPGAwWZzK6bRuYO4vlVL_Jq5hgmc7pjQS8eqlIuDr44FN1kVcRvF5YZugmUXzRdanL0/s1600/27.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg76H-guE_nP8B7YZsbHMxyNsS-VlIQktcUyyV4xKMaPD6C3zTurCW9bLiFzQQdzVv7RMSqtr9YQPGAwWZzK6bRuYO4vlVL_Jq5hgmc7pjQS8eqlIuDr44FN1kVcRvF5YZugmUXzRdanL0/s1600/27.png" /></a></div>
<br />
<ol>
<li>The map displayed on the browser requests a tile to the CDN</li>
<li>The CDN checks if it contains the image or not</li>
<li>If not, it queries the tile-server for that particular image</li>
<li>The Tile Server generates it as described on my previous post</li>
<li>The CDN stores the image tile </li>
<li>The CDN returns the image to the browser</li>
</ol>
<div>
<a name='more'></a>So, here's the recipe on how to setup this on Azure:</div>
<div>
<b><br /></b></div>
<div>
<b>1. Create a Cloud-Service instead of a Web-Site for the Tile-Server</b></div>
<div>
<br /></div>
<div>
First, why a cloud-service and not a web-site? Basically a limitation on Azure CDN itself:</div>
<div>
<br /></div>
<div>
<i>"<span style="background-color: white; color: #505050; font-family: wf_segoe-ui_normal, 'Segoe UI', 'Segoe WP', Tahoma, Arial, sans-serif; font-size: 14px; line-height: 21px;">The origin domain is the location from which the CDN caches content. The origin domain can be either a storage account or a cloud service;" </span></i><br />
<span style="color: #505050; font-family: wf_segoe-ui_normal, Segoe UI, Segoe WP, Tahoma, Arial, sans-serif;"><span style="font-size: x-small; line-height: 21px;"><a href="http://azure.microsoft.com/en-us/documentation/articles/cdn-how-to-use/">http://azure.microsoft.com/en-us/documentation/articles/cdn-how-to-use/</a></span></span></div>
<div>
<span style="color: #505050; font-family: wf_segoe-ui_normal, Segoe UI, Segoe WP, Tahoma, Arial, sans-serif;"><span style="font-size: 14px; line-height: 21px;"><br /></span></span></div>
<div>
Anyway, not really fond of Cloud-Services but in my particular case it might be a blessing in disguise, especially as I'm generating tiles with GDI+ and I might want to try using other technologies that require installing stuff on the server, which you simply can't do with Web-Sites.</div>
<div>
<br /></div>
<div>
Regardless, publishing a Web-API to a Cloud-Service is incredibly easy inside Visual Studio 2013. Just follow these steps (taken from <a href="http://msdn.microsoft.com/en-us/library/azure/hh420322.aspx">http://msdn.microsoft.com/en-us/library/azure/hh420322.aspx</a>):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://i.msdn.microsoft.com/dynimg/IC748917.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://i.msdn.microsoft.com/dynimg/IC748917.png" height="520" width="640" /></a></div>
<div>
<br /></div>
<div>
I've published it as "tilewin" and it now appears properly on my Azure Management Portal</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitgXYzKPQaycxHwfXgErjiehtzRl8fysUOV31AOxBBFw01m1IZ5ahiIB1maDtArASCVmwAmt57rXckvOqq-cYxncDgmNdzJl1QU-HLAnRvU1YOKQ29pBg45elJ4F-4mKDwN5P0B1c1Nzs/s1600/28.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEitgXYzKPQaycxHwfXgErjiehtzRl8fysUOV31AOxBBFw01m1IZ5ahiIB1maDtArASCVmwAmt57rXckvOqq-cYxncDgmNdzJl1QU-HLAnRvU1YOKQ29pBg45elJ4F-4mKDwN5P0B1c1Nzs/s1600/28.png" /></a></div>
<div>
<br /></div>
<div>
<br />
To validate that it's working just open the url: <a href="http://tilewin.cloudapp.net/v2/tile/10/490/391.png">http://tilewin.cloudapp.net/v2/tile/10/490/391.png</a><br />
<br />
<br /></div>
<div>
<b>2. Create a new "cdn" route on the tile-server webapi</b></div>
<div>
<br /></div>
<div>
This is an interesting one. According to Azure documentation to have a Cloud-Service be used as pull-origin for CDN it has to provide a "/cdn" route.</div>
<div>
<br /></div>
<b style="background-color: white; box-sizing: border-box; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: inherit;"><i>"Can I use different folder names in the Windows Azure CDN?</i></b><br />
<div>
<div style="background-color: white; box-sizing: border-box; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6; margin-bottom: 0.65rem; padding: 0px; text-rendering: optimizelegibility;">
<i>(...)</i></div>
<div style="background-color: white; box-sizing: border-box; margin-bottom: 0.65rem; padding: 0px; text-rendering: optimizelegibility;">
<i style="color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6;">For hosted-service object delivery as of SDK 1.4, you are restricted to publishing under the /cdn folder root on your service." </i><i style="background-color: transparent; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6;"><span style="color: #505050; font-family: wf_segoe-ui_normal, 'Segoe UI', 'Segoe WP', Tahoma, Arial, sans-serif; line-height: 21px;">(</span></i><span style="background-color: transparent; font-size: 14px; line-height: 21px;"><span style="color: #505050; font-family: wf_segoe-ui_normal, Segoe UI, Segoe WP, Tahoma, Arial, sans-serif;"><i><a href="http://azure.microsoft.com/blog/2011/03/18/best-practices-for-the-windows-azure-content-delivery-network/">http://azure.microsoft.com/blog/2011/03/18/best-practices-for-the-windows-azure-content-delivery-network/</a></i></span></span><i style="background-color: transparent; color: #666666; font-family: 'Segoe UI', 'Helvetica Neue', Arial, sans-serif; font-size: 14px; line-height: 1.6;"><span style="color: #505050; font-family: wf_segoe-ui_normal, Segoe UI, Segoe WP, Tahoma, Arial, sans-serif;"><span style="line-height: 21px;">)</span></span></i></div>
</div>
<br />
<div>
Also, if you recall, a tile-request was something in the likes of:</div>
<div>
<br /></div>
<div>
<a href="http://tilewin.cloudapp.net/v2/tile/10/490/391.png">http://tilewin.cloudapp.net/v2/tile/10/490/391.png</a></div>
<div>
<br /></div>
<div>
Simply adding a "/cdn" at the beginning of the route won't work. Thus, it will need to receive the params through query-string. So, I'm trying to support a route such as:</div>
<div>
<br /></div>
<div>
<a href="http://tilewin.cloudapp.net/cdn?z=10&x=490&y=391&type=png">http://tilewin.cloudapp.net/cdn?z=10&x=490&y=391&type=png</a></div>
<div>
<br /></div>
<div>
Fortunately with WebAPI 2.0 this is a breeze. First I've defined the new action as:</div>
<pre class="prettyprint">[Route("cdn")]
public async Task<VectorTile> GetTile([FromUri]int z,[FromUri]int x,[FromUri]int y)
{
var tile = await _tileLoader.LoadVectorTileAsync(z, x, y);
if (tile == null)
{
throw new HttpResponseException(HttpStatusCode.NotFound);
}
return tile;
}
</pre>
<div>
Also, to support the "type" parameter I had to change my MediaTypeMappings to include <a href="http://msdn.microsoft.com/en-us/library/system.net.http.formatting.querystringmapping(v=vs.118).aspx">QueryStringMapping</a>:<br />
<pre class="prettyprint">var imageFormatter = new TileImageFormatter((ITileGenerator)config
.DependencyResolver
.GetService(typeof(ITileGenerator)));
imageFormatter.MediaTypeMappings
.Add(new UriPathExtensionMapping(
"png", new MediaTypeHeaderValue("image/png")));
imageFormatter.MediaTypeMappings.Add(
new QueryStringMapping(
"type", "png", new MediaTypeHeaderValue("image/png")));
config.Formatters.Add(imageFormatter);
</pre>
</div>
<br />
<div>
Afterwards I re-published the Cloud Service and the following url now works:<br />
<br />
<a href="http://tilewin.cloudapp.net/cdn?z=10&x=490&y=391&type=png">http://tilewin.cloudapp.net/cdn?z=10&x=490&y=391&type=png</a><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVy2EYwJyPgGJakBRO0HV1Ul7j8xTzVP0LVNLxSMxOyAogWKgCxcC1S_pzACeGkPe7JH28nxCLbV6XSszjfNLaOiTj5WggQGtnqQ2h-rdStaAkIHLX76E_jYogtAoHCvvds8VN07ntWYU/s1600/29.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjVy2EYwJyPgGJakBRO0HV1Ul7j8xTzVP0LVNLxSMxOyAogWKgCxcC1S_pzACeGkPe7JH28nxCLbV6XSszjfNLaOiTj5WggQGtnqQ2h-rdStaAkIHLX76E_jYogtAoHCvvds8VN07ntWYU/s1600/29.png" /></a></div>
<br />
<b><br /></b>
<b>3. Create and configure the Azure CDN</b></div>
<div>
<br />
I've created a new CDN entry specifying the Cloud Service as the origin and making sure that Query String is enabled.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKKchqKzJtcpxpDY4BNeM_QsXgmwOu5rWFgoq6Xxm_S5o0baAl22xmpuOy9bxLkxsTdbi56RqgYCG-ZZqasfK53ixTR7U0n7ulfL6-saDyIFWX4x_gw7Yh0ydL5baUWZr2AW2FDbmCEsw/s1600/31.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhKKchqKzJtcpxpDY4BNeM_QsXgmwOu5rWFgoq6Xxm_S5o0baAl22xmpuOy9bxLkxsTdbi56RqgYCG-ZZqasfK53ixTR7U0n7ulfL6-saDyIFWX4x_gw7Yh0ydL5baUWZr2AW2FDbmCEsw/s1600/31.png" /></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJm8QkpDrWfWG-3uEFPdcqwrMlaDo1del4t3oIRrxfbpiAghqwBe6FgXI7r3lSIiREF1yMxXNmRJT5TulCjJuonhwDa-7xkQl4Vlrb2d0YmHaYvho4ZRnv_ei0LBvIGEaubP763OwMDIs/s1600/30.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJm8QkpDrWfWG-3uEFPdcqwrMlaDo1del4t3oIRrxfbpiAghqwBe6FgXI7r3lSIiREF1yMxXNmRJT5TulCjJuonhwDa-7xkQl4Vlrb2d0YmHaYvho4ZRnv_ei0LBvIGEaubP763OwMDIs/s1600/30.png" /></a></div>
<br />
<br />
Now issuing a request for the CDN (without including the "/cdn" on the url) returns an image tile:<br />
<br />
<a href="http://az665126.vo.msecnd.net/?z=10&x=490&y=391&type=png">http://az665126.vo.msecnd.net/?z=10&x=490&y=391&type=png</a><br />
<br />
I've already made a simple performance comparison on obtaining the tile directly from the Cloud-Service vs CDN. The latency on the CDN one is awesome (as expected). Also, I had setup Output cache on the Cloud-Service. Otherwise the difference would be even greater.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX8mxduUOmioGbnLaTTE3vzoOfw14wBr0IMvzcU6axryyS5x78QPwpqgen_sSi_oYY4g9gduYTts1HSKyPqOkTFu37ytIgOEL9zYx7uRRjHKwAI_5wRLR9mnV9HkJ5siwxOVni-z8kJL4/s1600/32.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX8mxduUOmioGbnLaTTE3vzoOfw14wBr0IMvzcU6axryyS5x78QPwpqgen_sSi_oYY4g9gduYTts1HSKyPqOkTFu37ytIgOEL9zYx7uRRjHKwAI_5wRLR9mnV9HkJ5siwxOVni-z8kJL4/s1600/32.png" height="74" width="640" /></a></div>
<br />
So, now ready to setup the map.<br />
<br /></div>
<div>
<br /></div>
<div>
<b>4. Change the Map to use the CDN</b></div>
<div>
<br /></div>
<div>
Now that's the easy one. I've just replaced the URL from<br />
<u><br /></u>
<u>http://tile-win.azurewebsites.net/v2/tile/{z}/{x}/{y}.png</u></div>
<div>
to<br />
<u>http://az665126.vo.msecnd.net/?z={z}<span class="s1">&</span>x={x}<span class="s1">&</span>y={y}<span class="s1">&</span>type=png</u></div>
<div>
<br />
<br />
<br /></div>
<div>
<br /></div>
Pedro Sousahttp://www.blogger.com/profile/17058685497255714267noreply@blogger.com0