Documentation on using OpenLR.
apt install postgresql postgis postgresql-10-pgrouting osmosisPostGIS is built on top of PostgreSQL, which is considered to be the most versatile open source database available. For clarity, all example SQL statements in this document will have delimited identifiers. This makes it immediately clear what is SQL syntax and what is an identifier.
First we have to create the database.
create database "autograph";Then connect to the new database, either by typing ”\c autograph” in the psql prompt, or from the shell:
psql -d autographNow, from the psql prompt we can install the necessary extensions:
create extension postgis;
create extension pgrouting;Prepare the PostGIS database to accept OSM data. The schema is shipped with the Osmosis installation:
psql -d autograph -f /usr/share/doc/osmosis/examples/pgsimple_schema_0.6.sqlMore on PostGIS schemas can be found at the OSM Wiki: https://wiki.openstreetmap.org/wiki/Osm2pgsql/schema
Use Osmosis to load OSM data into the database
osmosis --rb file=netherlands-latest.osm.pbf --ws host="localhost" database="autograph"Take a cup of coffee. In fact, go on a hiking trip through the Alps to arrive in northern Italy, find a nice espresso bar, have some good coffee. Doing this with the Luxembourg dataset takes about one minute on a laptop, all of the Netherlands may take more than two hours.
Download a osm.bz2 file. Run the following command:
osm2pgsql -d luxembourg -U postgres -P 5432 luxembourg-latest.osm.bz2This loads the data in a schema that QGis also understands, and is faster than using osmosis.
Here are some tips on setting cache sizes: https://postgis.net/docs/performance_tips.html#database_tuning_configuration
PGRouting is a routing framework on top of PostGIS. For PGRouting to work, the road network needs to have a suitable network topology defined. One way to achieve this, is to import the OSM data using the osm2pgrouting command. This command is not as flexible with data formats as osm2pgsql or osmosis, and reads the data as unpacked XML osm file. This means the size of the data that can be imported in one go is limited.
For any real use, it is probably more practical to define the routing topology on top of an OSM map imported with osm2pgsql.
Suppose we have created a database before, then we can add the topology as follows:
-- Add "source" and "target" column
ALTER TABLE planet_osm_roads ADD COLUMN "source" integer;
ALTER TABLE planet_osm_roads ADD COLUMN "target" integer;
-- Run topology function
SELECT pgr_createTopology('planet_osm_roads', 0.00001, 'way', 'osm_id');Take a cup of coffee, doctor’s orders. This operation takes a lot of memory too. Using the OSM data for the Netherlands, this took everything out of my laptop with 16GB memory and 32GB swap, but it made it!
NOTICE: -------------> TOPOLOGY CREATED FOR 163889 edges
Unfortunately, these are the wrong ‘roads’. We need to work with planet_osm_lines, which is a much bigger dataset. First create a table that contains only the roads.
create table "ways_topology" as
select "osm_id" from "planet_osm_line"
where "highway" is not null;From this we can create a view containing only the necessary information for displaying the new table.
Add a new layer. Set layer source to:
crs=EPSG:3857&format=&type=xyz&url=http://tile.openstreetmap.org/%7Bz%7D/%7Bx%7D/%7By%7D.png
The CRS will be EPSG:3857, WGS 84 / Pseudo Mercator.
Connecting to PostGIS is one of the better ways to provide QGIS with large amounts of data. The elephant logo button hides the action of adding a new source from PostGIS to the current rendering.
OSM Data consists of points, lines and polygons. There is a fourth class of objects called roads. These are not roads. They are the major set of lines needed to depict an overview map, containing rivers, administrative boundaries and, yes, also some major roads.
To get to the road network we need to filter the lines for a valid highway entry. The list of values for the highway column is given on the OSM Wiki.
Not all OSM editors adhere to these standards. If we query for all unique values in the Netherlands,
one entry in the Netherlands says “Luchtfoto suggereert hier een pad” (Dutch for: “Areal photo suggests a trail here”).
| key | value | description 0 |
| <25> | ||
|---|---|---|
| highway | motorway | A restricted access major divided highway, normally with 2 or more running lanes plus emergency hard shoulder. Equivalent to the Freeway, Autobahn, etc.. |
| highway | trunk | The most important roads in a country’s system that aren’t motorways. (Need not necessarily be a divided highway.) |
| highway | primary | The next most important roads in a country’s system. (Often link larger towns.) |
| highway | secondary | The next most important roads in a country’s system. (Often link towns.) |
| highway | tertiary | The next most important roads in a country’s system. (Often link smaller towns and villages) |
| highway | unclassified | The least most important through roads in a country’s system – i.e. minor roads of a lower classification than tertiary, but which serve a purpose other than access to properties. Often link villages and hamlets. (The word ‘unclassified’ is a historical artefact of the UK road system and does not mean that the classification is unknown; you can use highway=road for that.) |
| highway | residential | Roads which serve as an access to housing, without function of connecting settlements. Often lined with housing. |
| highway | service | For access roads to, or within an industrial estate, camp site, business park, car park etc. Can be used in conjunction with service=* to indicate the type of usage and with access=* to indicate who can use it and in what circumstances. |
| highway | motorway_link | The link roads (sliproads/ramps) leading to/from a motorway from/to a motorway or lower class highway. Normally with the same motorway restrictions. |
| highway | trunk_link | The link roads (sliproads/ramps) leading to/from a trunk road from/to a trunk road or lower class highway. |
| highway | primary_link | The link roads (sliproads/ramps) leading to/from a primary road from/to a primary road or lower class highway. |
| highway | secondary_link | The link roads (sliproads/ramps) leading to/from a secondary road from/to a secondary road or lower class highway. |
| highway | tertiary_link | The link roads (sliproads/ramps) leading to/from a tertiary road from/to a tertiary road or lower class highway. |
| highway | living_street | For living streets, which are residential streets where pedestrians have legal priority over cars, speeds are kept very low and where children are allowed to play on the street. |
| highway | pedestrian | For roads used mainly/exclusively for pedestrians in shopping and some residential areas which may allow access by motorised vehicles only for very limited periods of the day. To create a ‘square’ or ‘plaza’ create a closed way and tag as pedestrian and also with area=yes. |
| highway | track | Roads for mostly agricultural or forestry uses. To describe the quality of a track, see tracktype=*. Note: Although tracks are often rough with unpaved surfaces, this tag is not describing the quality of a road but its use. Consequently, if you want to tag a general use road, use one of the general highway values instead of track. |
| highway | bus_guideway | A busway where the vehicle guided by the way (though not a railway) and is not suitable for other traffic. Please note: this is not a normal bus lane, use access=no, psv=yes instead! |
| highway | escape | For runaway truck ramps, runaway truck lanes, emergency escape ramps, or truck arrester beds. It enables vehicles with braking failure to safely stop. |
| highway | raceway | A course or track for (motor) racing |
| highway | road | A road/way/street/motorway/etc. of unknown type. It can stand for anything ranging from a footpath to a motorway. This tag should only be used temporarily until the road/way/etc. has been properly surveyed. If you do know the road type, do not use this value, instead use one of the more specific highway=* values. |
| highway | footway | For designated footpaths; i.e., mainly/exclusively for pedestrians. This includes walking tracks and gravel paths. If bicycles are allowed as well, you can indicate this by adding a bicycle=yes tag. Should not be used for paths where the primary or intended usage is unknown. Use highway=pedestrian for pedestrianised roads in shopping or residential areas and highway=track if it is usable by agricultural or similar vehicles. |
| highway | bridleway | For horses. Equivalent to highway=path + horse=designated. |
| highway | steps | For flights of steps (stairs) on footways. Use with step_count=* to indicate the number of steps |
| highway | path | A non-specific path. Use highway=footway for paths mainly for walkers, highway=cycleway for one also usable by cyclists, highway=bridleway for ones available to horses as well as walkers and highway=track for ones which is passable by agriculture or similar vehicles. |
| highway | cycleway | For designated cycleways. Add foot=* only if default-access-restrictions do not apply. |
We can put these values into a table in our database and use the following query to create a new table with only the desired roads.
create table "ways" as
select "osm_id", "highway", "way"
from "planet_osm_line"
where "highway" in
(select "value" from "highway_values");
Osmosis is a command-line tool to manipulate/filter/extract OSM data.
How to build an OSM database that is up to date. Rather than doing a quarterly manual job of downloading a 14Gb+ file, decompressing it (250Gb+) and inserting it into a database it will be a lot easier to download daily (50Mb) OSM change files (.osc) and apply the changes to the existing database.
The instructions below is a modified version of Martin van Exel’s tutorial here - https://docs.google.com/document/pub?id=1paaYsOakgJEYP380R70s4SGYq8ME3ASl-mweVi1DlQ4
adduser osm passwd osm
su - postgres psql createdb osm createlang plpgsql osm CREATE USER osm WITH PASSWORD ‘osm’; #CREATE DATABASE osm; GRANT ALL PRIVILEGES ON DATABASE osm to osm; GRANT ALL PRIVILEGES ON DATABASE osm to postgres; \q psql -d osm -U osm -f /usr/pgsql-9.1/share/contrib/postgis-2.0/postgis.sql psql -d osm -U osm f /usr/pgsql-9.1/share/contrib/postgis-2.0/spatial_ref_sys.sql psql -U osm -d osm CREATE EXTENSION hstore; \q psql -U osm -d osm -f /home/<user name>/osm/osmosis-0.41/script/pgsimple_schema_0.6.sql
su mkdir /tmp/osm cd tmp/osm mkdir planet mkdir planet/replication
cd tmp/osm/planet wget http://planet.openstreetmap.org/planet-latest.osm.bz2
osmosis –rrii workingDirectory=/tmp/osm/planet/replication
bunzip2 -c planet-latest.osm.bz2 | head
Now at this point I’ve been running osmosis to grab the change file and osm2pgsql to throw it at the database but you can apparently run osmosis on it’s own;
/opt/osmosis-0.41/bin/osmosis –rri workingDirectory=/tmp/osm/planet/replication –sc –wpc user=”osm” database=”osm” password=”osm”
/opt/osmosis-0.41/bin/osmosis –rri workingDirectory=/tmp/osm/planet/replication –simplify-change –write-xml-change /tmp/osm/planet/replication/changes.osc.gz
/opt/osm2pgsql/osm2pgsql –append -S /opt/osm2pgsql/default.style -d osm -U postgres –slim /tmp/osm/planet/replication/changes.osc.gz
#!/bin/sh n=`ps -ef | grep -v grep | grep opt/osmosis-0.41 | wc -l` m=`ps -ef | grep -v grep | grep opt/osm2pgsql | wc -l` let i=n+m if [ $i -gt 0 ]; then echo osmosis or osm2pgsql running else echo not running /opt/osmosis-0.41/bin/osmosis –rri workingDirectory=/tmp/osm/planet/replication –simplify-change –write-xml-change /tmp/osm/planet/replication/changes.osc.gz /opt/osm2pgsql/osm2pgsql –append -S /opt/osm2pgsql/default.style -d osm -U postgres –slim /tmp/osm/planet/replication/changes.osc.gz fi
/opt/osm2pgsql/osm2pgsql –append -S /opt/osm2pgsql/default.style -d osm -U postgres –slim /tmp/osm/planet/replication/changes.osc.gz -e15 -o expire.list
cat expire.list | /opt/mapnik/mod_tile/render_expired –map=osm –min-zoom=6 –touch-from=7 >/dev/null
There is a Python package to access the PGRouting database: =psycopgr=. It has a tutorial.
With any Python installation, just type:
pip install psycopgrThe Itinero/OpenLR library is written in C#. To build software using it, you’ll need a C# SDK, which can be downloaded from Microsoft: Get started with .NET
C# is a language developed by Microsoft, much like Java. It compiles to the virtual platform known as the CLR (Common Language Runtime). Together with a library of standard classes (FCL) this environment is .NET. The .NET environment runs on all major platforms (Windows, MacOS, Linux, Android…) making applications written in a .NET language (C#, F#, Visual Basic) portable.
If you have successfully installed the .NET SDK you should have the dotnet command-line tool.
dotnet --version2.1.4
A new project is created using dotnet new
dotnet new console --name my-openlr-tool --language C#Then the new project is run using dotnet run
cd my-openlr-tool
dotnet runHello World!
Our console application was initialized with a friendly “Hello World!”. We will be extending the “Hello World!” example with the first OpenLR example.
using System;
<<first-example-imports>>
namespace my_openlr_tool
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
<<first-example-load-data>>
<<first-example-encode-line>>
}
}
}The .NET environment has two ways of adding external libraries to a project: packages and references. Adding a package will obtain a pre-compiled library from NuGet.
Documentation for OpenLR is a bit scarce: docs.itinero.tech.
dotnet add package Itinero
dotnet add package Itinero.IO.Osm
dotnet add package OpenLR// import File
using System.IO;
// import RouterDb
using Itinero;
// import method RouterDb.LoadOsmData
using Itinero.IO.Osm;
// import Vehicle
using Itinero.Osm.Vehicles;
// import Coder
using OpenLR;
// import OsmCoderProfile
using OpenLR.Osm;
// import ReferencedLine
using OpenLR.Referenced.Locations;The following example encodes and decodes a line in Luxembourg. This loads OSM data from a PBF, which can be found on GeoFabrik.de.
First we need to load the PBF file into the RouterDB. Loading the Luxembourg data takes a few seconds.
Console.Write("Loading Luxembourg ... ");
var routerDb = new RouterDb();
using (var sourceStream =
File.OpenRead(
Path.Combine(AppDomain.CurrentDomain.BaseDirectory,
"../../../luxembourg-latest.osm.pbf")))
{
routerDb.LoadOsmData(sourceStream, Vehicle.Car);
}
Console.WriteLine("done");Next we encode a line between two coordinates and then decode it again.
// create coder.
var coder = new Coder(routerDb, new OsmCoderProfile());
Console.WriteLine("Building a line location, and encoding it.");
// build a line location from a shortest path.
var line = coder.BuildLine(
new Itinero.LocalGeo.Coordinate(
49.67218282319583f, 6.142280101776122f),
new Itinero.LocalGeo.Coordinate(
49.67776489459803f, 6.1342549324035645f));
Console.WriteLine("original line: {0} --> {1}",
line.StartLocation,
line.EndLocation);
// encode this location.
var encoded = coder.Encode(line);
Console.WriteLine("encoded line: {0}", encoded);
// decode this location.
var decodedLine = coder.Decode(encoded) as ReferencedLine;
Console.WriteLine("decoded line: {0} --> {1}",
decodedLine.StartLocation,
decodedLine.EndLocation);Running this example should give the following output:
Hello World! Loading Luxembourg ... done Building a line location, and encoding it. original line: 17060@0% [49.67221,6.14243] --> 10712@0% [49.67784,6.133887] encoded line: CwReMiNSjSOYEvyqAjIjaAIG decoded line: 17060@0% [49.67221,6.14243] --> 10712@0% [49.67784,6.133887]