4. Preprocessing functions

Download planet OSM

gmtra.preprocessing.planet_osm()

This function will download the planet file from the OSM servers.

Create osm.pbf files for a single country

gmtra.preprocessing.single_country(country, regionalized=False, create_poly_files=False)

Clip a country from the planet osm file and save to individual osm.pbf files

This function has the option to extract individual regions

Arguments:

country : The country for which we want extract the data.

Keyword Arguments:

regionalized : Default is False. Set to True will parallelize the extraction over all regions within a country.

create_poly_files : Default is False. Set to True will create new .poly files.

Create osm.pbf files for all specified countries

gmtra.preprocessing.all_countries(subset=[], regionalized=False, reversed_order=False)

Clip all countries from the planet osm file and save them to individual osm.pbf files

Optional Arguments:

subset : allow for a pre-defined subset of countries. REquires ISO3 codes. Will run all countries if left empty.

regionalized : Default is False. Set to True if you want to have the regions of a country as well.

reversed_order : Default is False. Set to True to work backwards for a second process of the same country set to prevent overlapping calculations.

Returns:

clipped osm.pbf files for the defined set of countries (either the whole world by default or the specified subset)

Create global shapefiles

gmtra.preprocessing.global_shapefiles(regionalized=False)

This function will simplify shapes and add necessary columns, to make further processing more quickly

For now, we will make use of the latest GADM data: https://gadm.org/download_world.html

Optional Arguments:

regionalized : Default is False. Set to True will also create the global_regions.shp file.

Remove tiny shapes from large multipolygons

gmtra.preprocessing.remove_tiny_shapes(x, regionalized=False)

This function will remove the small shapes of multipolygons. Will reduce the size of the file.

Arguments:

x : a geometry feature (Polygon) to simplify. Countries which are very large will see larger (unhabitated) islands being removed.

Optional Arguments:

regionalized : Default is False. Set to True will use lower threshold settings (default: False).

Returns:

MultiPolygon : a shapely geometry MultiPolygon without tiny shapes.

Create .poly files

gmtra.preprocessing.poly_files(data_path, global_shape, save_shapefile=False, regionalized=False)

This function will create the .poly files from the world shapefile. These .poly files are used to extract data from the openstreetmap files.

This function is adapted from the OSMPoly function in QGIS.

Arguments:

data_path : base path to location of all files.

global_shape: exact path to the global shapefile used to create the poly files.

Optional Arguments:

save_shape_file : Default is False. Set to True will the new shapefile with the countries that we include in this analysis will be saved.

regionalized : Default is False. Set to True will perform the analysis on a regional level.

Returns:

.poly file for each country in a new dir in the working directory.

Clip a region from a larger .osm.pbf file

gmtra.preprocessing.clip_osm(data_path, planet_path, area_poly, area_pbf)

Clip the an area osm file from the larger continent (or planet) file and save to a new osm.pbf file. This is much faster compared to clipping the osm.pbf file while extracting through ogr2ogr.

This function uses the osmconvert tool, which can be found at http://wiki.openstreetmap.org/wiki/Osmconvert.

Either add the directory where this executable is located to your environmental variables or just put it in the ‘scripts’ directory.

Arguments:

continent_osm: path string to the osm.pbf file of the continent associated with the country.

area_poly: path string to the .poly file, made through the ‘create_poly_files’ function.

area_pbf: path string indicating the final output dir and output name of the new .osm.pbf file.

Returns:

a clipped .osm.pbf file.

Merge SSBN maps within a country

gmtra.preprocessing.merge_SSBN_maps(country)

Function to merge SSBN maps to a country level.

Arguments:

country : ISO3 code of the country for which we want to merge the river and surface flood maps to country level.

Extract bridges from OpenStreetMap

gmtra.preprocessing.region_bridges(n)

This function will extract all bridges from OpenStreetMap for the specified region.

Arguments:

n : the index ID of a region in the specified shapefile with all the regions.

Returns:

GeoDataFrame : A geopandas GeoDataFrame with all bridges in a region. Will also save this to a .csv file.