mapping module

Module for mapping tile-based library.

CreateFolders(outFolder, scratchFolderName='scratch', outMapFolderName='maps', removeExist=True)

Create folders for storing temporary files and output maps.

Parameters:

Name	Type	Description	Default
outFolder	str	output folder	required
scratchFolderName	str	name of the folder for storing temporary files	'scratch'
outMapFolderName	str	name of the folder for storing output maps, default is 'maps'	'maps'
removeExist	str	bool whether to remove existing folders, default is True	True

DownloadTiledLibrary(libUrl, libName, localLibFolder)

Download and unzip tiled libraries.

Parameters:

Name	Type	Description	Default
libUrl	str	the url of the library	required
libName	str	the name of the library	required
localLibFolder	str	the folder where the library will be saved	required

Returns:

Type	Description
None	no return.

EstimateFspDofFromGauge(libFolder, libName, gaugeFspDf, minGaugeDof=0.0328084, weightingType='V')

Estimate/interpolate FSP DOF (Depth of Flow, i.e., FSP stage) from observed gauge DOFs using distance-(horizontal) or elevation-based (vertical) linear interpolation.

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are located	required
libName	str	the name of the library that the gauges will be snapped to	required
gaugeFspDf	data frame	a data frame of gauge FSPs (i.e., FSPs to which gauges are snapped). It should have at least 4 columns ['stage_elevation','lib_name','FspX','FspY'].	required
minGaugeDof	float	min DOF a gauge should have, by default is 1 cm = 0.0328083989501312 foot	0.0328084
weightingType	str	'V' for vertical distance-based or 'H' for horizontal distance-based, default is 'V'	'V'

Returns:

Type	Description
data frame	a data frame with interpolated FSP DOFs.

EstimateFspDofFromGaugeBlob(libBlobSerClient, libName, gaugeDf, gaugeElevField, minGaugeDof=0.0328084)

Estimate FSP Depth of Flow (DoF), i.e., FSP stage from gauges on Microsoft Planetary Computer using Azure Blob Storage.

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a BlobServiceClient object	required
libName	str	the name of the library that the gauges will be snapped to	required
gaugeDf	data frame	a data frame of gauges. It should have at least 4 columns ['FspX','FspY','FspFilledElev','Dist']	required
gaugeElevField	str	the field in gaugeDf that stores gauge's water surface elevation	required
minGaugeDof	float	min DOF a gauge should have, default is 1 cm = 0.0328083989501312 foot. Negative DOF may occur as incorrect gauge datum	0.0328084

Returns:

Type	Description
data frame	a data frame with interpolated FSP DOFs.

GetTileTifs(tifFiles)

Get tile Geotif files

Parameters:

Name	Type	Description	Default
tifFiles	list	a list of tile Geotif files	required

Returns:

Type	Description
list	a list of tile Geotif files

InterpBetweenTwoGauges(fx, fe, gx1, ge1, gy1, gx2, ge2, gy2, weightingType='V')

Interpolate FSP DOF (Depth of Flow, i.e., FSP stage) based on FSP's elevation (fe) or distance (fx) between two gauges.

Parameters:

Name	Type	Description	Default
fx	float or vector of float	FSP's distance from downstream outlet.	required
fe	float or vector of float	FSP's elevation. Can be a vector	required
gx1	float	gauge1's distance from downstream outlet	required
ge1	float	gauge1's elevation	required
gy1	float	gauge1's DOF	required
gx2	float	gauge2's distance from downstream outlet	required
ge2	float	gauge2's elevation	required
gy2	float	gauge2's DOF	required
weightingType	str	'V' for vertical distance-based or 'H' for horizontal distance-based, default is 'V'	'V'

Returns:

Type	Description
float or vector of float	interpolated DOF at fx

InterpDofWithGauges(fx, fe, gx, ge, gy, weightingType='V')

Interpolate FSP DOF (Depth of Flow, i.e., FSP stage) using the DOFs observed at a list of gauges.

Parameters:

Name	Type	Description	Default
fx	vector of float	FSP's distance from downstream outlet.	required
fe	vector of float	FSP's elevation.	required
gx	vector of float	gauge's distance from downstream outlet.	required
ge	vector of float	gauge's elevation.	required
gy	vector of float	gauge's DOF.	required
weightingType	str	'V' for vertical distance-based or 'H' for horizontal distance-based, default is 'V'	'V'

Returns:

Type	Description
vector of float	interpolated DOF at fx

InterpolateFspDofFromGauge(libFolder, libName, gaugeFspDf, minGaugeDof=0.0328084, weightingType='V')

Interpolate FSP DOF (Depth of Flow, i.e., FSP stage) from observed gauge DOFs using distance-(horizontal) or elevation-based (vertical) linear interpolation. Different from EstimateFspDofFromGauge(), this function assumes gauge FSPs already have their DOF calculated!

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are located	required
libName	str	the name of the library that the gauges will be snapped to	required
gaugeFspDf	data frame	a data frame of gauge FSPs (i.e., FSPs to which gauges are snapped). It should have at least 4 columns ['lib_name','FspX','FspY','Dof'].	required
minGaugeDof	float	min DOF a gauge should have, by default is 1 cm = 0.0328083989501312 foot	0.0328084
weightingType	str	'V' for vertical distance-based or 'H' for horizontal distance-based, default is 'V'	'V'

Returns:

Type	Description
data frame	a data frame with interpolated FSP DOFs.

MapFloodDepthWithTiles(libFolder, libName, fileFormat, outMapFolder, fspDof='MinDtf', aoiExtent=None)

Map flood depth with tiled library based on FSP DOF and AOI extent

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are stored	required
libName	str	the name of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
outMapFolder	str	the folder where the mapped tiles will be saved	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None

Returns:

Type	Description
list	a list of mapped tile names as GeoTif files.

MapFloodDepthWithTilesAsDag(libFolder, libName, fileFormat, outMapFolder, fspDof='MinDtf', aoiExtent=None)

Map flood depth with tiled library based on FSP DOF and AOI extent as a Directed Acyclic Graph (DAG)

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are stored	required
libName	str	the name of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
outMapFolder	str	the folder where the mapped tiles will be saved	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None

Returns:

Type	Description
tuple	a Directed Acyclic Graph (DAG) and the root node name.

MapFloodDepthWithTilesBlob(libBlobSerClient, libName, fileFormat, mapContainerClient, fspDof='MinDtf', aoiExtent=None)

Map flood depth with tiled library based on FSP DOF and AOI extent on Microsoft Planetary Computer using data in Azure Blob Storage.

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a BlobServiceClient object	required
libName	str	the name of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
mapContainerClient	ContainerClient	a ContainerClient object for the container to store the mapped tiles	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None

Returns:

Type	Description
list	a list of mapped tile names as GeoTif files

MapFloodDepthWithTilesBlobAsDag(libBlobSerClient, libName, fileFormat, mapContainerClient, fspDof='MinDtf', aoiExtent=None)

Map flood depth with tiled library based on FSP DOF and AOI extent on Microsoft Planetary Computer using data in Azure Blob Storage as a Directed Acyclic Graph (DAG).

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a BlobServiceClient object	required
libName	str	the name of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
mapContainerClient	ContainerClient	a ContainerClient object for the container to store the mapped tiles	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None

Returns:

Type	Description
tuple	a Directed Acyclic Graph (DAG) and the root node name

MapOneTile(libFolder, libName, tid, fppExtent, cellSize, libSr, fileFormat, outMapFolder, fspDof='MinDtf', aoiExtent=None)

Map one tile as a GeoTif file based on FSP DOF and AOI extent

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are stored	required
libName	str	the name of the library	required
tid	int	the tile ID	required
fppExtent	list	the extent of the FPPs in the tile [minX,maxX,minY,maxY]	required
cellSize	float	the cell size of the raster	required
libSr	str	the spatial reference of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
outMapFolder	str	the folder where the mapped tiles will be saved	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None

Returns:

Type	Description
str	the name of the mapped tile as a GeoTif file

MapOneTileBlob(libBlobSerClient, libName, tid, fppExtent, cellSize, libSr, fileFormat, mapContainerClient, fspDof='MinDtf', aoiExtent=None)

Map one tile as a GeoTif file based on FSP DOF and AOI extent on Microsoft Planetary Computer using Azure Blob Storage.

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a BlobServiceClient object	required
libName	str	the name of the library	required
tid	int	the tile ID	required
fppExtent	list	the extent of the FPPs in the tile [minX,maxX,minY,maxY]	required
cellSize	float	the cell size of the raster	required
libSr	str	the spatial reference of the library	required
fileFormat	str	the file format of the tile, 'snappy' or 'mat'	required
mapContainerClient	ContainerClient	a ContainerClient object for the container to store the mapped tiles	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY], default is None	None

Returns:

Type	Description
str	the name of the mapped tile as a GeoTif file

MosaicGtifs(outMapFolder, gtifs, mosaicTifName, keepTifs=False)

Mosaic a list of GeoTifs into one GeoTif file using rasterio.merge module. See https://medium.com/spatial-data-science/how-to-mosaic-merge-raster-data-in-python-fb18e44f3c8. This func may cause memory overflow as the merge() first creates the mosaiced array in memory!

Parameters:

Name	Type	Description	Default
outMapFolder	str	the folder where the mosaiced tif will be saved	required
gtifs	list	a list of tile GeoTifs to be mosaiced	required
mosaicTifName	str	the name of the mosaiced GeoTif file	required
keepTifs	bool	whether to keep the tile GeoTifs, default is False	False

Returns:

Type	Description
str	the name of the mosaiced GeoTif.

MosaicGtifsBlob(mapContClient, gtifs, outGtif, keepTifs=False)

Mosaic a list of GeoTifs into one GeoTif file using rasterio.merge module on Microsoft Planetary Computer using data in Azure Blob Storage. See https://medium.com/spatial-data-science/how-to-mosaic-merge-raster-data-in-python-fb18e44f3c8. This func may cause memory overflow as the merge() first creates the mosaiced array in memory!

Parameters:

Name	Type	Description	Default
mapContClient	ContainerClient	a ContainerClient object for the container to store the mosaiced tif	required
gtifs	list	a list of tile GeoTifs to be mosaiced	required
outGtif	str	the name of the mosaiced GeoTif file	required
keepTifs	bool	whether to keep the tile GeoTifs, default is False	False

Returns:

Type	Description
None	no return.

MosaicGtifsUsingVirtualRaster(gtifs, outGtif)

Mosaic a list of GeoTifs into one GeoTif file using GDAL virtual raster. Easiest way of mosaic very large Gtif. Based on the video at https://www.youtube.com/watch?v=sBBMKbAj8XE

Parameters:

Name	Type	Description	Default
gtifs	list	a list of tile GeoTifs to be mosaiced	required
outGtif	str	the name of the mosaiced GeoTif file	required

Returns:

Type	Description
None	no return

NearestPoint(p1df, x1FieldName, y1FieldName, p2df, x2FieldName, y2FieldName, distFieldName='dist', otherColumns=None)

Join two sets of points by nearest distance. The returned data frame will have, in addition to p1df fields, a new distance field (i.e., distFieldName), plus other fields (i.e., otherColumns) copied from p2df.

Parameters:

Name	Type	Description	Default
p1df	data frame	the first set of points as a pandas DataFrame	required
x1FieldName	str	the field name of y coordinates in p1df	required
y1FieldName	str	the field name of y coordinates in p1df	required
p2df	data frame	the second set of points as a pandas DataFrame	required
x2FieldName	str	the field name of x coordinates in p2df	required
y2FieldName	str	the field name y coordinates in p2df	required
distFieldName	str	the name of the distance field in the returned data frame, default is 'dist'	'dist'
otherColumns	list	the names of other fields to be copied from p2df to the returned data frame, default is None.	None

NearestPointInPlace(p1df, x1FieldName, y1FieldName, p2df, x2FieldName, y2FieldName, distFieldName='dist', otherColumns=None)

Join two sets of points by nearest distance. p1df will have a new distance field (i.e., distFieldName), plus other fields (i.e., otherColumns) copied from p2df. Note that this function changes p1df. This is the only difference between this function and NearestPoint()! This is the only difference between this function and NearestPoint()!

Parameters:

Name	Type	Description	Default
p1df	data frame	the first set of points as a pandas DataFrame	required
x1FieldName	str	the field name of y coordinates in p1df	required
y1FieldName	str	the field name of y coordinates in p1df	required
p2df	data frame	the second set of points as a pandas DataFrame	required
x2FieldName	str	the field name of x coordinates in p2df	required
y2FieldName	str	the field name y coordinates in p2df	required
distFieldName	str	the name of the distance field in the returned data frame, default is 'dist'	'dist'
otherColumns	list	the names of other fields to be copied from p2df to the returned data frame, default is None	None

Returns:

Type	Description
data frame	p1df with the nearest points from p2df.

SnapGauges2Fsps(libFolder, libNames, gauges, snapDist=350, gaugeXField='X', gaugeYField='Y', fspColumns=['FspId', 'FspX', 'FspY', 'FilledElev'])

Snap gauges to library FSPs. The function will return a data frame with the nearest FSPs for each gauge in each library. Note that multiple FSPs from different libraries might be snapped to the same gauge!

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are located	required
libNames	list	a list of library names that the gauges will be snapped to	required
gauges	str or data frame	a text file or a pandas DF of gauges. It must have the columns of 'X' and 'Y' in FSP's coordinate system	required
snapDist	float	the distance to snap gauges to FSPs, default is 350	350
gaugeXField	str	the field name of x coordinates in gauges, default is 'X'	'X'
gaugeYField	str	the field name of y coordinates in gauges, default is 'Y'	'Y'
fspColumns	list	the names of FSP columns to be returned, default is ['FspId','FspX','FspY','FilledElev']	['FspId', 'FspX', 'FspY', 'FilledElev']

Returns:

Type	Description
data frame	a data frame with the nearest FSPs for each gauge in each library.

SnapGauges2FspsBlob(libBlobSerClient, libName, gaugesDf, snapDist=350, gaugeIdField='GaugeLID', gaugeXField='X', gaugeYField='Y')

Snap gauges to library FSPs on Microsoft Planetary Computer (MPC) using Azure Blob Storage. The function has NOT been checked yet!

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a blob service client	required
libName	str	the name of the library that the gauges will be snapped to	required
gaugesDf	data frame	a pandas DF of gauges. It must have the columns of 'X' and 'Y' in FSP's coordinate system	required
snapDist	float	the distance to snap gauges to FSPs, default is 350	350
gaugeIdField	str	the field name of gauge IDs in gauges, default is 'GaugeLID'	'GaugeLID'
gaugeXField	str	the field name of x coordinates in gauges, default is 'X'	'X'
gaugeYField	str	the field name of y coordinates in gauges, default is 'Y'	'Y'

Returns:

Type	Description
data frame	a data frame with the nearest FSPs for each gauge in the library.

SnapGaugesToFsps(libFolder, libName, gauges, snapDist=250, gaugeIdField='GaugeLID', gaugeXField='X', gaugeYField='Y')

Snap gauges to library FSPs. The function will return a data frame with the snapped gauges only.

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are located	required
libName	str	the name of the library that the gauges will be snapped to	required
gauges	str or data frame	a text file or a pandas data frame of gauges. It must have the columns of 'X' and 'Y' in FSP's coordinate system	required
snapDist	float	the distance to snap gauges to FSPs, default is 250	250
gaugeIdField	str	the field name of gauge IDs in gauges, default is 'GaugeLID'	'GaugeLID'
gaugeXField	str	the field name of x coordinates in gauges, default is 'X'	'X'
gaugeYField	str	the field name of y coordinates in gauges, default is 'Y'	'Y'

Returns:

Type	Description
data frame	a data frame with the snapped gauges only.

SnapGaugesToFspsBlob(libBlobSerClient, libName, gaugesDf, snapDist=250, gaugeIdField='GaugeLID', gaugeXField='X', gaugeYField='Y')

Snap gauges to library FSPs on Microsoft Planetary Computer (MPC) using Azure Blob Storage. The function is has NOT been checked yet!

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a blob service client	required
libName	str	the name of the library that the gauges will be snapped to	required
gaugesDf	data frame	a pandas DF of gauges. It must have the columns of 'X' and 'Y' in FSP's coordinate system	required
snapDist	float	the distance to snap gauges to FSPs, default is 250	250
gaugeIdField	str	the field name of gauge IDs in gauges, default is 'GaugeLID'	'GaugeLID'
gaugeXField	str	the field name of x coordinates in gauges, default is 'X'	'X'
gaugeYField	str	the field name of y coordinates in gauges, default is 'Y'	'Y'

Returns:

Type	Description
data frame	a data frame with the snapped gauges only.

TileFspFppRelations2Array(fspFppRels, fppExtent, cellSize, fspDof='MinDtf', aoiExtent=None, noData=-9999)

Turn a dataframe of FSP-FPP relations to a 2D array of flood depth. The minimum bounding extent of the FPPs in the relations is always used when create the map for the tile!

Parameters:

Name	Type	Description	Default
fspFppRels	data frame	a dataframe of FSP-FPP relations which have the columns of ["FspId", "FppCol", "FppRow", "Dtf", "FilledDepth"] from a tile	required
fppExtent	list	a list of [minX, maxX, minY, maxY], FPP's external extent of the tile and is also used to locate FPP's columns and rows in map coordinate	required
cellSize	float	the cell size of the raster	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None	None
noData	int	the no data value, default is -9999	-9999

Returns:

Type	Description
tuple	a tuple of the np array as the map, the no data value, the minimum X value, and the minimum Y value

Tiles2Map(libFolder, libName, fspDof='MinDtf', aoiExtent=None)

Decide the tiles need to be mapped for the library.

Parameters:

Name	Type	Description	Default
libFolder	str	the folder where the libraries are stored	required
libName	str	the name of the library	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None.	None

Returns:

Type	Description
tuple	a list of tile IDs, a list of tile FPP extents

Tiles2MapBlob(libBlobSerClient, libName, fspDof='MinDtf', aoiExtent=None)

Decide the tiles need to be mapped for the library on Microsoft Planetary Computer using Azure Blob Storage.

Parameters:

Name	Type	Description	Default
libBlobSerClient	BlobServiceClient	a BlobServiceClient object	required
libName	str	the name of the library	required
fspDof	str, float, or data frame	the FSP DOF for mapping flood depth. default is 'MinDtf'. If it's a string, it can be 'MinDtf', 'NumOfFsps', or 'Depression'. If it's a float, it's a constant stage for all the FSPs. If it's a data frame, it's a data frame of FSPs with DOF.	'MinDtf'
aoiExtent	list	the extent of the area of interest [minX,maxX,minY,maxY]. default is None.	None

Returns:

Type	Description
tuple	a list of tile IDs, a list of tile FPP extents