Marble/OSMVectorTileCreation: Difference between revisions

From KDE TechBase
No edit summary
Line 4: Line 4:


As an example we'd like to render tiles for tile level 13, 15 and 17 of San Francisco.  
As an example we'd like to render tiles for tile level 13, 15 and 17 of San Francisco.  
First you need the osm.pbf file for that region. In our case we download that from:
First you need the osm.pbf file for that region. In our case we get it from:


http://download.geofabrik.de/north-america/us/california.html
http://download.geofabrik.de/north-america/us/california.html


or to be more specifically: http://download.geofabrik.de/north-america/us/california-latest.osm.pbf
where we locate the link to the pbf file as http://download.geofabrik.de/north-america/us/california-latest.osm.pbf


=== Creating the tools - osmconvert and osmfilter ===
=== Creating the tools - osmconvert and marble-vectorosm-tilecreator ===


Now we need the tools for conversion and filtering the pbf file into tiles. In order to compile the tools you should have zlib installed on your system (e.g. via sudo apt-get install zlib1g-dev). Then run:
Now we need the tools for conversion and filtering the pbf file into tiles. In order to compile the tools you should have zlib installed on your system (e.g. via sudo apt-get install zlib1g-dev). Then run:
Line 19: Line 19:
gcc -O3 -o osmconvert osmconvert.c -lz
gcc -O3 -o osmconvert osmconvert.c -lz
sudo mv osmconvert /usr/local/bin
sudo mv osmconvert /usr/local/bin
and pretty much the same for osmfilter
wget http://m.m.i24.cc/osmfilter.c
gcc -O3 -o osmfilter osmfilter.c
sudo mv osmfilter /usr/local/bin
</source>
</source>


Line 29: Line 25:
<source lang="bash">
<source lang="bash">
osmconvert -h
osmconvert -h
</source>
and also
<source lang="bash">
osmfilter -h
</source>
</source>
should output usage instructions.
should output usage instructions.
Please compile Marble from source code as well as described in https://marble.kde.org/sources.php and make sure to compile the development version (master branch). Pass the option -DBUILD_MARBLE_TOOLS=TRUE to cmake in order to compile all Marble tools.


=== Running the VectorTileCreator ===
=== Running the VectorTileCreator ===


Now our tool is needed. You need python version 3 to be installed. In a shell, go to the marble sources directory and then
Now our tool is needed. In a shell, go to the marble build directory and then
<source lang="bash">
<source lang="bash">
cd tools/vectortilecreator
cd tools/vectorosm-tilecreator
</source>
</source>
now running
now running
<source lang="bash">
<source lang="bash">
./vectortilecreator.py -h
./marble-vectorosm-tilecreator -h
</source>
</source>
should give usage instructions.
should give usage instructions.


The tile cutter tool works as follows: You call it with a region file as parameter (or several). A region file is a comma separated text file that describes the input to process. You cannot pass a pbf file directly. Here is an example of a region file:
The tile cutter tool works as follows: You call it with a link to an osm .pbf file as parameter.  


<tt>http://download.geofabrik.de/europe/germany/baden-wuerttemberg-latest.osm.pbf;Karlsruhe;8.30831;48.93133;8.62496;49.17019</tt>
The tool will generate tiles for zoom levels 11, 13, 15 and 17 by default. You can overwrite that using the -z switch, e.g.
-z 13 15.


The first entry is the pbf file to process. You can use a URL here and vectortilecreator.py will download that file to the cache directory if it does not exist yet.
=== Full Example ===
The second entry is an arbitrary name that helps you remember which region this is.
If you follow the steps above, the final tile creation run after installing the required tools will look like this:
The remaining four entries are a rectangular region (bounding box) where data is extracted from. Make sure that the region you specify here is actually part of the .pbf file! An easy way to get it right is to use openstreetmap.org and its Export feature. Refer to [http://nienhueser.de/marble/marble-vectortilecreator-order.jpg this image] to get the order of the coordinates right.


The script will generate tiles in the current directory unless you use the -d option: For example -d ~/.local/share/marble/maps/earth/vectorosm is what you want for local tiles. By default it will generate tiles for zoom levels 13, 15, 17. You can overwrite that using the -z switch
<source lang="bash">
-z 13 15. This would allow to exclude e.g. level 17 (which can take between < 1 sec for small tiles but > 10 secs for large tiles depending on the input size). Once called, the script does the following
cd ~/marble/build/
* It parses the region file to read which file to work on and which boundaries it has
cd tools/vectorosm-tilecreator
* To speed things up, it then creates another file with only the given boundaries inside (e.g. if you pass california-latest.osm.pbf and work on san francisco it will create a temporary file that has only san francisco inside)
./marble-vectorosm-tilecreator http://download.geofabrik.de/north-america/us/california-latest.osm.pbf
* Afterwards it iterates over the zoom levels passed. For each level it calculates the tiles that are inside the boundaries.  
[========================================>]  100%  0.0/0.0 MB  Downloading california-latest.osm.pbf
Then it calls osmconvert for each tile to generate the tile
[========================================>]  100%  0.0/0.0 MB  Downloading california.poly
* If it also finds a file levels/$ZOOM.level it will call osmfilter afterwards
[========================================>]  100%  osm cache tiles complete.
* And finally the result is zipped
[========================================>]  100%  landmass cache tiles complete.                  
 
[========================================>]  100%  Vector OSM tiles complete.
=== What should I do if this doesn't seem to work for my own data? ===
</source>
Now you might have created your own tiles already. But what are possible pitfalls if your own tile data doesn't show up?
 
* If you get an error something like this "UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9c in position 22: invalid start byte" from the vectortilecreator.py script, then this means that you are passing it a .pbf file directly. That is wrong: It wants a .csv file which itself refers to one or more pbf files and which rectangular regions and zoom levels it should create.
* Check the directory ~/.local/share/marble/maps/earth/vectorosm . It should have subdirectories of the levels that you created. Since the vector tiles are created as .osm.zip files you can zgrep their content in order to look for features that should be inside the data you wanted to create
* Check the bounding rect definition inside the text file. It should specify the bottom left and top right corner of your area.

Revision as of 13:25, 19 November 2016

OSM Vector Tiles

Creating tiles from a pbf file

As an example we'd like to render tiles for tile level 13, 15 and 17 of San Francisco. First you need the osm.pbf file for that region. In our case we get it from:

http://download.geofabrik.de/north-america/us/california.html

where we locate the link to the pbf file as http://download.geofabrik.de/north-america/us/california-latest.osm.pbf

Creating the tools - osmconvert and marble-vectorosm-tilecreator

Now we need the tools for conversion and filtering the pbf file into tiles. In order to compile the tools you should have zlib installed on your system (e.g. via sudo apt-get install zlib1g-dev). Then run: 

cd /tmp
wget http://m.m.i24.cc/osmconvert.c
gcc -O3 -o osmconvert osmconvert.c -lz
sudo mv osmconvert /usr/local/bin

if that works, running 

osmconvert -h

should output usage instructions.

Please compile Marble from source code as well as described in https://marble.kde.org/sources.php and make sure to compile the development version (master branch). Pass the option -DBUILD_MARBLE_TOOLS=TRUE to cmake in order to compile all Marble tools.

Running the VectorTileCreator

Now our tool is needed. In a shell, go to the marble build directory and then 

cd tools/vectorosm-tilecreator

now running 

./marble-vectorosm-tilecreator -h

should give usage instructions.

The tile cutter tool works as follows: You call it with a link to an osm .pbf file as parameter.

The tool will generate tiles for zoom levels 11, 13, 15 and 17 by default. You can overwrite that using the -z switch, e.g. -z 13 15.

Full Example

If you follow the steps above, the final tile creation run after installing the required tools will look like this: 

cd ~/marble/build/
cd tools/vectorosm-tilecreator
./marble-vectorosm-tilecreator http://download.geofabrik.de/north-america/us/california-latest.osm.pbf
 [========================================>]  100%  0.0/0.0 MB  Downloading california-latest.osm.pbf
 [========================================>]  100%  0.0/0.0 MB  Downloading california.poly
 [========================================>]  100%  osm cache tiles complete.
 [========================================>]  100%  landmass cache tiles complete.                    
 [========================================>]  100%  Vector OSM tiles complete.
Retrieved from "https://techbase.kde.org/index.php?title=Marble/OSMVectorTileCreation&oldid=98329"