`moad_tools` API

`moad_tools.geo_tools` Module

UBC MOAD group tools regarding gridding and geography.

moad_tools.geo_tools.estimate_closest_point(dataset, lons, lats)[source]

Estimate the closest grid point to an array of lat/lons using a index file created by make_mapping_file above

Parameters:

dataset (xarray) – dataset
list (numpy array or) – lons
list – lats

return: numpy array of indexes: iis, jjs

moad_tools.geo_tools.make_mapping_file(coordinate_file, mapping_file, grid_description, lon_var='nav_lon', lat_var='nav_lat')[source]

Make two arrays that index from longitude and latitude to grid index The indices and information to use them are written to a netCDF file and the same values are returned by the function.

Parameters:

coordinate_file (str) – netCDF file to read the coordinates from
mapping_file (str) – netCDF file written with the index arrays
grid_description (str) – name of the grid that is being mapped, to be written as a comment into the netcdf file
lon_var (str) – name of longitude variable in the coordinate file
lat_var (str) – name of latitude variable in the coordinate file

Returns:

lonmin, latmin: start of the indexes dlon, dlat: step size for the indexes indexi, indexj: index arrays

`moad_tools.observations` Module

Functions for downloading observations data from web services.

moad_tools.observations.get_ndbc_buoy(buoy_id)[source]

Retrieve a collection of time series of the last 45 days of real-time observed buoy data for an ECCC or NOAA data buoy from the NOAA National Data Buoy Center (NDBC) https://www.ndbc.noaa.gov/data/realtime2/ web service.

The time series is returned as a pandas.DataFrame object.

The returned time series date/time index is UTC.

Parameters:: buoy_id (int or str) – Buoy number or name. Names use moad_tools.places.PLACES to look up the buoy number.
Returns:: Buoy data time series.
Return type:: pandas.DataFrame

`moad_tools.places` Module

UBC MOAD group geographic places information.

It is recommended that library code that uses the PLACES data structure from this module should use try...except to catch KeyError exceptions and produce an error message that is more informative than the default, for example:

try:
    stn_number = f"{PLACES[stn_id]['stn number']}"
except KeyError as e:
    raise KeyError(f"place name or info key not found in moad_tools.places.PLACES: {e}")

moad_tools.places.PLACES = {'2nd Narrows Rail Bridge': {'lon lat': (-123.0247222, 49.2938889), 'stn number': 3160171}, 'Halibut Bank': {'EC buoy number': 46146, 'GEM2.5 grid ji': (149, 141), 'NEMO grid ji': (503, 261), 'lon lat': (-123.72, 49.34)}, 'Sentry Shoal': {'EC buoy number': 46131, 'GEM2.5 grid ji': (183, 107), 'NEMO grid ji': (707, 145), 'lon lat': (-125.0, 49.92)}}: Information about geographic places used in the analysis and presentation of UBC EOAS MOAD group model results.

`moad_tools.midoss` Sub-package

geotiff-watermask Tool

Functions and command-line tool to calculate and store a Numpy array file containing a SalishSeaCast domain water mask for the AIS ship track density GeoTIFF files used to generate oil spill parameters for Monte Carlo runs of MOHID.

moad_tools.midoss.geotiff_watermask.calc_watermask(ais_density, ssc_mesh)[source]

Calculate a Numpy boolean array containing a SalishSeaCast domain water mask for the AIS ship track density GeoTIFF files used to generate oil spill parameters for Monte Carlo runs of MOHID.

Parameters:

ais_density (rasterio.io.DatasetReader) – AIS ship tracks GeoTIFF object to use the pixel lons/lats from to calculate water mask.
ssc_mesh (xarray.Dataset) – SalishSeaCast NEMO mesh mask dataset to use the NEMO grid lons/lats and T-grid water/land maks from to calculate the water mask.

Returns:

Boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.

Return type:

numpy.ndarray

moad_tools.midoss.geotiff_watermask.geotiff_watermask(geotiff_file, meshmask_file)[source]

Calculate a Numpy boolean array containing a SalishSeaCast domain water mask for the AIS ship track density GeoTIFF files used to generate oil spill parameters for Monte Carlo runs of MOHID.

Parameters:

geotiff_file (str) – File path and name of an AIS ship tracks GeoTIFF file to use the pixel lons/lats from to calculate water mask; typically a ShipTrackDensityGeoTIFFs/all_yyyy_mm.tif file.
meshmask_file (str) – File path and name of a SalishSeaCast NEMO mesh mask file to use the NEMO grid lons/lats and T-grid water/land maks from to calculate the water mask.

Returns:

Boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.

Return type:

numpy.ndarray

moad_tools.midoss.geotiff_watermask.write_numpy_file(watermask, numpy_file)[source]

Store a Numpy array file containing a boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.

Parameters:

watermask (numpy.ndarray) – Boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.
numpy_file (str) – File path and name of Numpy array file to write the water mask to.

hdf5-to-netCDF4 Tool

Functions to transform an MOHID HDF5 output file into a netCDF4 file.

moad_tools.midoss.hdf5_to_netcdf4.hdf5_to_netcdf4(hdf5_file, netcdf4_file)[source]

Transform selected contents of a MOHID HDF5 results file HDF5_FILE into a netCDF4 file stored as NETCDF4_FILE.

Parameters:

hdf5_file (pathlib.Path or str) – File path and name of MOHID HDF5 results file to read from.
netcdf4_file (pathlib.Path or str) – File path and name of netCDF4 file to write to.

random-oil-spills Script

The random-oil-spills script produces a CSV file of randomly selected oil spill parameters. Such a file is one of the items required to launch a collection of MOHID runs via the monte-carlo Sub-command of the MOHID command processor.

Information about the random-oil-spills command-line arguments and options is available via:

random-oil-spills --help

Usage: random-oil-spills [OPTIONS] N_SPILLS CONFIG_FILE CSV_FILE

  Calculate and store a CSV file containing parameters of a set of random
  oil spills to drive Monte Carlo runs of MOHID.

  Please see

  https://ubc-moad-tools.readthedocs.io/en/latest/moad_tools.html#moad_tools.midoss.random_oil_spills.cli

  for more information about arguments and options, and

  https://ubc-moad-tools.readthedocs.io/en/latest/moad_tools.html#processing-configuration-yaml-file

  for details of the contents of the config file.

Options:
  --version                       Show the version and exit.
  -v, --verbosity [debug|info|warning|error|critical]
                                  Choose how much information you want to see
                                  about the progress of the calculation;
                                  warning, error, and critical should be
                                  silent unless something bad goes wrong.
                                  [default: warning]

  --help                          Show this message and exit.

The Functions and Command-line Interface section below provides API details for the public functions that make up the random-oil-spills script.

Processing Configuration YAML File

A collection of values for the processing that the random-oil-spills performs are stored in a YAML file that is passed in as the command-line argument CONFIG_FILE.

In contrast to the random-oil-spills command-line arguments N_SPILLS and CSV_FILE that may be different every time the script is run, the values in the YAML file are likely to change only rarely. Storing them in a (version controlled) YAML file avoids the tedium of typing them for every invocation of random-oil-spills.

Here is an example YAML file:

# Config items for random-oil-spills script

# Starting and ending dates of period from which
# spill dates and hours are to be selected
start date: 2015-01-01
end date: 2018-12-31

# Directory path to read AIS GeoTIFF files from
geotiffs dir: /media/doug/warehouse/MIDOSS/ShipTrackDensityGeoTIFFs/

# Directory path to read AIS vessel track shapefiles from
shapefiles dir: /media/doug/warehouse/MIDOSS/ShipTrackShapefiles/

# File to read SalishSeaCast NEMO domain water mask for AIS GeoTIFF files from
geotiff watermask: /media/doug/warehouse/MIDOSS/ShipTrackDensityGeoTIFFs/geotiff-watermask.npy

# File to read SalishSeaCast NEMO mesh mask from
nemo meshmask: /media/doug/warehouse/MEOPAR/grid/mesh_mask201702.nc

# List of vessel types
vessel types:
  - tanker
  - atb
  - barge
  - cargo
  - cruise
  - ferry
  - fishing
  - smallpass
  - other

# File to read oil attribution data from
oil attribution: /media/doug/warehouse/MIDOSS/marine_transport_data/oil_attribution.yaml

Functions and Command-line Interface

Functions and command-line tool to calculate a CSV file containing parameters of a set of random oil spills to drive Monte Carlo runs of MOHID.

moad_tools.midoss.random_oil_spills.adjust_tug_tank_barge_length(vessel_type, vessel_len, random_generator)[source]

Standardize ATB and tug lengths to represent length of tug and tank barge. See AIS data attribute table for more information.

Parameters:

vessel_type (str) – Vessel type from which spill occurs.
vessel_len (int) – Length of vessel from which spill occurs [m].
random_generator (numpy.random.Generator) – PCG-64 random number generator.

Returns:

Randomly selected tug and tank barge length [m].

Return type:

int

moad_tools.midoss.random_oil_spills.calc_vte_probability(geotiffs_dir, geotiff_watermask)[source]

Calculate monthly spill probability weights from vessel traffic exposure (VTE) in AIS GeoTIFF files, masked to include only cells that are within the SalishSeaCast NEMO domain.

Parameters:

geotiffs_dir (pathlib.Path) – Directory path to read AIS GeoTIFF files from.
geotiff_watermask (numpy.ndarray) – Boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.

Returns:

12 elements array of monthly spill probability weights

Return type:

numpy.ndarray

moad_tools.midoss.random_oil_spills.choose_fraction_spilled(random_generator)[source]

Randomly choose a fraction spilled based on the _cumulative_spill_fraction() fit.

Parameters:: random_generator (numpy.random.Generator) – PCG-64 random number generator.
Returns:: Fraction of oil volume spilled.
Return type:: float

moad_tools.midoss.random_oil_spills.fuel_or_cargo_spill(oil_attrs, vessel_type, random_generator)[source]

Randomly choose whether the spill is from fuel or cargo volume for vessel types that transport oil as cargo.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
vessel_type (str) – Vessel type from which spill occurs.
random_generator (numpy.random.Generator) – PCG-64 random number generator.

Returns:

Fuel or cargo spill flag.

Return type:

boolean

moad_tools.midoss.random_oil_spills.get_date(start_date, end_date, vte_probability, random_generator)[source]

Randomly select a spill date and hour, with the month weighted by vessel traffic exposure (VTE) probability.

Parameters:

start_date (datetime.datetime) – Starting date of period from which spill dates and hours are to be selected.
end_date (datetime.datetime) – Ending date of period from which spill dates and hours are to be selected.
vte_probability (numpyt.ndarray) – 12 elements array of monthly spill probability weights
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

Randomly selected spill date and hour

Return type:

datetime.datetime

moad_tools.midoss.random_oil_spills.get_lat_lon_indices(geotiffs_dir, spill_month, geotiff_watermask, ssc_mesh, random_generator)[source]

Randomly select a spill lat/lon based on vessel traffic exposure (VTE) in a particular month’s AIS GeoTIFF file. The VTE data are masked to include only cells that have water cells within the SalishSeaCast NEMO domain. The lat/lon from VTE is used to select a SalishSeaCast NEMO surface grid cell from which one of 9 uniformly distributed sub-grid points within the cell is randomly selected as the spill location.

Parameters:

geotiffs_dir (pathlib.Path) – Directory path to read AIS GeoTIFF files from.
spill_month (int) – Month number for which to choose a spill location.
geotiff_watermask (numpy.ndarray) – Boolean water mask to apply to AIS ship track density GeoTIFF files to restrict them to the SalishSeaCast NEMO domain.
ssc_mesh (xarray.Dataset) – SalishSeaCast NEMO mesh mask dataset to use the NEMO grid lons/lats and T-grid water/land maks from to calculate the water mask.
random_generator (numpy.random.Generator) – PCG-64 random number generator.

Returns:

6-tuple composed of:

spill latitude [°N in [-90°, 90°] range]
spill longitude [°E in [-180°, 180°] range]
x-index of GeoTIFF cell in which spill is located
y-index of GeoTIFF cell in which spill is located
GeoTIFF cell bounding box
value of GeoTIFF cell in which spill is located (for QA/QC)

Return type:

tuple

moad_tools.midoss.random_oil_spills.get_length_origin_destination(shapefiles_dir, vessel_type, spill_month, geotiff_bbox, random_generator)[source]

Randomly choose an AIS vessel track from which the spill occurs, with the choice weighted by the vessel traffic exposure (VTE) for the specified vessel type, month, and GeoTIFF cell. Return the length of the vessel, and voyage origin & destination, and vessel MMSI from the chosen AIS track.

Parameters:

shapefiles_dir (pathlib.Path) – Directory path to read shapefiles from
vessel_type (str) – Vessel type from which spill occurs.
spill_month (int) – Month number in which spill occurs.
geotiff_bbox (shapely.geometry.Polygon) – Bounding box of GeoTIFF cell containing spill location.
random_generator (numpy.random.Generator) – PCG-64 random number generator.

Returns:

4-tuple composed of:

length of vessel from which spill occurs [m] (int)
origin of AIS track from which spill occurs (str or None)
destination of AIS track from which spill occurs (str or None)
vessel MMSI (str)

Return type:

tuple

moad_tools.midoss.random_oil_spills.get_oil_capacity(oil_attrs, vessel_length, vessel_type, random_generator)[source]

Calculate fuel_capacity [liters] and cargo_capacity [liters] based on vessel length and type, with the exception of ATBs and barges. Tank_capacity is estimated by length for ATBs > 50 m only. For all other ATB and barge traffic, as well as fuel capacity for ATBs > 50 m, both fuel and cargo capacities are estimated by weighted values based on AIS data for ATB ship traffic for which tugs and barges were married by a combination of web research and AIS analysis. We assume that the fuel and cargo capacities for non-ATB tug and tank barges is well represented by the ATB data.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
vessel_length (int or float) – Length of vessel from which spill occurs [m].
vessel_type (str) – Vessel type from which spill occurs.
random_generator (numpy.random.Generator) – PCG-64 random number generator.

moad_tools.midoss.random_oil_spills.get_oil_type(oil_attrs, vessel_type, vessel_origin, vessel_dest, fuel_spill, vessel_fuel_types, marine_transport_data_dir, random_generator)[source]

Randomly choose a type of oil spilled based on vessel type, AIS origin & destination, and whether fuel or cargo is spilled.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
vessel_type (str) – Vessel type from which spill occurs.
vessel_origin (str or None) – Origin of AIS track from which spill occurs.
vessel_dest (str or None) – Destination of AIS track from which spill occurs.
fuel_spill (boolean) – Fuel or cargo spill flag.
vessel_fuel_types (dict) – Mapping of fuel types and probabilities for vessel types.
marine_transport_data_dir (pathlib.Path) – Directory path to marine_transport_data files repository cloned from https://github.com/MIDOSS/marine_transport_data.
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

2-tuple composed of:

Type of oil spilled (str).
Flag to indicate that the spill is a fuel spill from a barge that is carrying something other than oil as cargo (boolean).

Return type:

tuple

moad_tools.midoss.random_oil_spills.get_oil_type_atb(oil_attrs, origin, destination, transport_data_dir, random_generator)[source]

Randomly choose type of cargo oil spilled from an ATB (articulated tug and barge) based on AIS track origin & destination, and oil cargo attribution analysis.

Unlike traditional tank barges, the vessels with ‘atb’ designation are known oil-cargo vessels. We used three different data sources to verify: AIS, Dept of Ecology’s fuel transfer records and Charlie Costanzo’s ATB list. Details of traffic can be seen in this google spreadsheet: https://docs.google.com/spreadsheets/d/1dlT0JydkFG43LorqgtHle5IN6caRYjf_3qLrUYqANDY/edit

Because of this pre-identification and selection method, we can assume that all ATBs are oil-cargo atbs and that the absence of origin-destination information is due to issues in linking ship tracks and not ambiguity about whether traffic is oil-cargo traffic.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
origin (str or None) – Origin of AIS track from which spill occurs.
destination (str or None) – Destination of AIS track from which spill occurs.
transport_data_dir (pathlib.Path) – Directory path to marine_transport_data files repository cloned from https://github.com/MIDOSS/marine_transport_data.
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

Type of oil spilled.

Return type:

str

moad_tools.midoss.random_oil_spills.get_oil_type_barge(oil_attrs, origin, destination, transport_data_dir, random_generator)[source]

Randomly choose type of cargo oil spilled from abarge based on AIS track origin & destination, and oil cargo attribution analysis.

Decision tree for allocating oil type to barge traffic see Google drawing [Barge_Oil_Attribution](https://docs.google.com/drawings/d/10PM53-UnnILYCAPKU9MxiR-Y4OW0tIMhVzSjaHr-iSc/edit) for a visual representation.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
origin (str or None) – Origin of AIS track from which spill occurs.
destination (str or None) – Destination of AIS track from which spill occurs.
transport_data_dir (pathlib.Path) – Directory path to marine_transport_data files repository cloned from https://github.com/MIDOSS/marine_transport_data.
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

2-tuple composed of:

Type of oil spilled (str or None)
Fuel or cargo spill flag (boolean)

Return type:

tuple

moad_tools.midoss.random_oil_spills.get_oil_type_cargo(cargo_info, facility, vessel_type, random_generator)[source]

Randomly choose cargo oil type based on facility and vessel type by querying information in input yaml_file.

Parameters:

cargo_info (dict) – Cargo oil type attribution information from the output of a make_cargo_*.ipynb notebooks.
facility (str or None) – Vessel origin from AIS.
vessel_type (str) – Vessel type from which spill occurs.
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

Cargo oil type.

Return type:

str

moad_tools.midoss.random_oil_spills.get_oil_type_tanker(oil_attrs, origin, destination, transport_data_dir, random_generator)[source]

Randomly choose type of cargo oil spilled from a tanker based on AIS track origin & destination, and oil cargo attribution analysis.

Decision tree for allocating oil type to tanker traffic see Google drawing [Tanker_Oil_Attribution](https://docs.google.com/drawings/d/1-4gl2yNNWxqXK-IOr4KNZxO-awBC-bNrjRNrt86fykU/edit) for a visual representation.

Parameters:

oil_attrs (dict) – Oil attribution information from the output of make_oil_attrs.py.
origin (str or None) – Origin of AIS track from which spill occurs.
destination (str or None) – Destination of AIS track from which spill occurs.
transport_data_dir (pathlib.Path) – Directory path to marine_transport_data files repository cloned from https://github.com/MIDOSS/marine_transport_data.
random_generator (numpy.random.Generator) – PCG-64 random number generator

Returns:

Type of oil spilled.

Return type:

str

moad_tools.midoss.random_oil_spills.get_vessel_type(geotiffs_dir, vessel_types, spill_month, geotiff_x_index, geotiff_y_index, random_generator)[source]

Randomly choose a vessel type from which the spill occurs, with the choice weighted by the vessel traffic exposure (VTE) for the specified month and GeoTIFF cell.

Parameters:

geotiffs_dir (pathlib.Path) – Directory path to read AIS GeoTIFF files from.
vessel_types (list) – Vessel types from which spill can occur, and for which there are monthly 2018 VTE GeoTIFFs.
spill_month (int) – Month number for which to choose a spill location.
geotiff_x_index (int) – x-index of GeoTIFF cell in which spill is located
geotiff_y_index (int) – y-index of GeoTIFF cell in which spill is located
random_generator (numpy.random.Generator) – PCG-64 random number generator.

Returns:

Randomly selected vessel type from which spill occurs.

Return type:

str

moad_tools.midoss.random_oil_spills.random_oil_spills(n_spills, config_file, random_seed=None)[source]

Calculate a dataframe containing parameters of a set of random oil spills to drive Monte Carlo runs of MOHID.

Parameters:

n_spills (int) – Number of spills to calculate parameters for.
config_file (str) – File path and name of the YAML file to read processing configuration dictionary from.
random_seed (None or int) – Seed to initialize random number generator with. This is facilitates a reproducible stream of random number that is useful for unit testing.

Returns:

Dataframe of random oil spill parameters with n_spills rows.

Return type:

pandas.DataFrame

moad_tools.midoss.random_oil_spills.write_csv_file(df, csv_file)[source]

Parameters:

df (pandas.DataFrame) – Dataframe to write to CSV file.
csv_file (str) – File path and name of CSV file to write to.

moad_tools API

moad_tools.geo_tools Module

moad_tools.observations Module

moad_tools.places Module

moad_tools.midoss Sub-package

geotiff-watermask Tool

hdf5-to-netCDF4 Tool

random-oil-spills Script

Processing Configuration YAML File

Functions and Command-line Interface

`moad_tools` API

`moad_tools.geo_tools` Module

`moad_tools.observations` Module

`moad_tools.places` Module

`moad_tools.midoss` Sub-package