API Documentation#
Core Classes#
Model#
A model run.
It is intented to be model agnostic. It deals primarily with how the model is to be run, i.e. the period of the run and where the output is going. The actual configuration of the run is provided by the config object.
Further explanation is given in the rompy.core.Baseconfig docstring.
Config#
- pydantic model rompy.core.config.BaseConfig[source]
Base class for model templates.
The template class provides the object that is used to set up the model configuration. When implemented for a given model, can move along a scale of complexity to suit the application.
In its most basic form, as implemented in this base object, it consists of path to a cookiecutter template with the class providing the context for the {{config}} values in that template. Note that any {{runtime}} values are filled from the ModelRun object.
If the template is a git repo, the checkout parameter can be used to specify a branch or tag and it will be cloned and used.
If the object is callable, it will be colled prior to rendering the template. This mechanism can be used to perform tasks such as fetching exteral data, or providing additional context to the template beyond the arguments provided by the user..
Show JSON schema
{ "title": "BaseConfig", "description": "Base class for model templates.\n\nThe template class provides the object that is used to set up the model configuration.\nWhen implemented for a given model, can move along a scale of complexity\nto suit the application.\n\nIn its most basic form, as implemented in this base object, it consists of path to a cookiecutter template\nwith the class providing the context for the {{config}} values in that template. Note that any\n{{runtime}} values are filled from the ModelRun object.\n\nIf the template is a git repo, the checkout parameter can be used to specify a branch or tag and it\nwill be cloned and used.\n\nIf the object is callable, it will be colled prior to rendering the template. This mechanism can be\nused to perform tasks such as fetching exteral data, or providing additional context to the template\nbeyond the arguments provided by the user..", "type": "object", "properties": { "model_type": { "const": "base", "default": "base", "title": "Model Type", "type": "string" }, "template": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "/opt/hostedtoolcache/Python/3.10.18/x64/lib/python3.10/site-packages/rompy/templates/base", "description": "The path to the model template", "title": "Template" }, "checkout": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "main", "description": "The git branch to use if the template is a git repo", "title": "Checkout" } }, "additionalProperties": true }
- field checkout: str | None = 'main'
The git branch to use if the template is a git repo
- field model_type: Literal['base'] = 'base'
- field template: str | None = '/opt/hostedtoolcache/Python/3.10.18/x64/lib/python3.10/site-packages/rompy/templates/base'
The path to the model template
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
Time#
- pydantic model rompy.core.time.TimeRange[source]
A time range object
Examples
>>> from rompy import TimeRange >>> tr = TimeRange(start="2020-01-01", end="2020-01-02") >>> tr TimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=datetime.datetime(2020, 1, 2, 0, 0), duration=None, interval=None, include_end=True) >>> tr = TimeRange(start="2020-01-01", duration="1d") >>> tr TimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=datetime.datetime(2020, 1, 2, 0, 0), duration=timedelta(days=1), interval=None, include_end=True) >>> tr = TimeRange(start="2020-01-01", duration="1d", interval="1h") >>> tr TimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=None, duration=timedelta(days=1), interval=timedelta(hours=1), include_end=True)
Show JSON schema
{ "title": "TimeRange", "description": "A time range object\n\nExamples\n--------\n>>> from rompy import TimeRange\n>>> tr = TimeRange(start=\"2020-01-01\", end=\"2020-01-02\")\n>>> tr\nTimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=datetime.datetime(2020, 1, 2, 0, 0), duration=None, interval=None, include_end=True)\n>>> tr = TimeRange(start=\"2020-01-01\", duration=\"1d\")\n>>> tr\nTimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=datetime.datetime(2020, 1, 2, 0, 0), duration=timedelta(days=1), interval=None, include_end=True)\n>>> tr = TimeRange(start=\"2020-01-01\", duration=\"1d\", interval=\"1h\")\n>>> tr\nTimeRange(start=datetime.datetime(2020, 1, 1, 0, 0), end=None, duration=timedelta(days=1), interval=timedelta(hours=1), include_end=True)", "type": "object", "properties": { "start": { "anyOf": [ { "format": "date-time", "type": "string" }, { "type": "null" } ], "default": null, "description": "The start date of the time range", "examples": [ "2020-01-01" ], "title": "Start" }, "end": { "anyOf": [ { "format": "date-time", "type": "string" }, { "type": "null" } ], "default": null, "description": "The end date of the time range", "examples": [ "2020-01-02" ], "title": "End" }, "duration": { "anyOf": [ { "type": "string" }, { "format": "duration", "type": "string" }, { "type": "null" } ], "default": null, "description": "The duration of the time range", "examples": [ "1d" ], "title": "Duration" }, "interval": { "anyOf": [ { "type": "string" }, { "format": "duration", "type": "string" }, { "type": "null" } ], "default": "1h", "description": "The frequency of the time range", "examples": [ "1h", "'1h'" ], "title": "Interval" }, "include_end": { "default": true, "description": "Determines if the end date should be included in the range", "title": "Include End", "type": "boolean" } } }
- Fields:
duration (str | datetime.timedelta | None)
end (datetime.datetime | None)
include_end (bool)
interval (str | datetime.timedelta | None)
start (datetime.datetime | None)
- Validators:
parse_start_end_duration
»all fields
valid_duration_interval
»duration
valid_duration_interval
»interval
validate_start_end
»end
validate_start_end
»start
validate_start_end_duration
»all fields
- field duration: str | timedelta | None = None
The duration of the time range
- Validated by:
parse_start_end_duration
valid_duration_interval
validate_start_end_duration
- field end: datetime | None = None
The end date of the time range
- Validated by:
parse_start_end_duration
validate_start_end
validate_start_end_duration
- field include_end: bool = True
Determines if the end date should be included in the range
- Validated by:
parse_start_end_duration
validate_start_end_duration
- field interval: str | timedelta | None = '1h'
The frequency of the time range
- Validated by:
parse_start_end_duration
valid_duration_interval
validate_start_end_duration
- field start: datetime | None = None
The start date of the time range
- Validated by:
parse_start_end_duration
validate_start_end
validate_start_end_duration
- common_times(date_range: TimeRange) list[datetime] [source]
- contains(date: datetime) bool [source]
- contains_range(date_range: TimeRange) bool [source]
- format_duration(duration: timedelta) str [source]
Format a timedelta object as a human-readable string.
This method formats a timedelta in a way that’s suitable for display in logs and other output.
- Parameters:
duration – The timedelta object to format
- Returns:
A formatted string representation of the duration
- validator parse_start_end_duration » all fields[source]
- serialize_model() dict [source]
- validator valid_duration_interval » duration, interval[source]
- validator validate_start_end » start, end[source]
- validator validate_start_end_duration » all fields[source]
- property date_range: list[datetime]
Grid#
- pydantic model rompy.core.grid.BaseGrid[source]
Representation of a grid in geographic space.
This is the base class for all Grid objects. The minimum representation of a grid are two NumPy array’s representing the vertices or nodes of some structured or unstructured grid, its bounding box and a boundary polygon. No knowledge of the grid connectivity is expected.
Show JSON schema
{ "title": "BaseGrid", "description": "Representation of a grid in geographic space.\n\nThis is the base class for all Grid objects. The minimum representation of a grid\nare two NumPy array's representing the vertices or nodes of some structured or\nunstructured grid, its bounding box and a boundary polygon. No knowledge of the\ngrid connectivity is expected.", "type": "object", "properties": { "grid_type": { "const": "base", "default": "base", "title": "Grid Type", "type": "string" } }, "additionalProperties": false }
- Fields:
- field grid_type: Literal['base'] = 'base'
- bbox(buffer=0.0) Bbox [source]
Returns a bounding box for the spatial grid
This function returns a list [ll_x, ll_y, ur_x, ur_y] where ll_x, ll_y (ur_x, ur_y) are the lower left (upper right) x and y coordinates bounding box of the model domain
- boundary(tolerance=0.2) Polygon [source]
Returns the convex hull boundary polygon from the grid.
- Parameters:
tolerance (float) – Simplify polygon shape based on maximum distance from original geometry, see https://shapely.readthedocs.io/en/stable/manual.html#object.simplify.
- Returns:
polygon – See https://shapely.readthedocs.io/en/stable/manual.html#Polygon
- Return type:
shapely.Polygon
- boundary_points(spacing=None, tolerance=0.2) tuple [source]
Returns array of coordinates from boundary polygon.
- Parameters:
tolerance (float) – Simplify polygon shape based on maximum distance from original geometry, see https://shapely.readthedocs.io/en/stable/manual.html#object.simplify.
spacing (float) – If specified, points are returned evenly spaced along the boundary at the specified spacing, otherwise all points are returned.
Returns
--------
points (tuple) – Tuple of x and y coordinates of the boundary points.
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- plot(ax=None, figsize=None, fscale=10, buffer=0.1, borders=True, land=True, coastline=True)[source]
Plot the grid
- property maxx: float
- property maxy: float
- property minx: float
- property miny: float
- property x: ndarray
- property y: ndarray
- pydantic model rompy.core.grid.RegularGrid[source]
Regular grid in geographic space.
This object provides an abstract representation of a regular grid in some geographic space.
Show JSON schema
{ "title": "RegularGrid", "description": "Regular grid in geographic space.\n\nThis object provides an abstract representation of a regular grid in some\ngeographic space.", "type": "object", "properties": { "grid_type": { "const": "regular", "default": "regular", "description": "Type of grid, must be 'regular'", "title": "Grid Type", "type": "string" }, "x0": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "description": "X coordinate of the grid origin", "title": "X0" }, "y0": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "description": "Y coordinate of the grid origin", "title": "Y0" }, "rot": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": 0.0, "description": "Rotation angle of the grid in degrees", "title": "Rot" }, "dx": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "description": "Spacing between grid points in the x direction", "title": "Dx" }, "dy": { "anyOf": [ { "type": "number" }, { "type": "null" } ], "default": null, "description": "Spacing between grid points in the y direction", "title": "Dy" }, "nx": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "description": "Number of grid points in the x direction", "title": "Nx" }, "ny": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "description": "Number of grid points in the y direction", "title": "Ny" } }, "additionalProperties": false }
- Fields:
- Validators:
generate
»all fields
- field dx: float | None = None
Spacing between grid points in the x direction
- Validated by:
- field dy: float | None = None
Spacing between grid points in the y direction
- Validated by:
- field grid_type: Literal['regular'] = 'regular'
Type of grid, must be ‘regular’
- Validated by:
- field nx: int | None = None
Number of grid points in the x direction
- Validated by:
- field ny: int | None = None
Number of grid points in the y direction
- Validated by:
- field rot: float | None = 0.0
Rotation angle of the grid in degrees
- Validated by:
- field x0: float | None = None
X coordinate of the grid origin
- Validated by:
- field y0: float | None = None
Y coordinate of the grid origin
- Validated by:
- bbox(buffer=0.0) Bbox
Returns a bounding box for the spatial grid
This function returns a list [ll_x, ll_y, ur_x, ur_y] where ll_x, ll_y (ur_x, ur_y) are the lower left (upper right) x and y coordinates bounding box of the model domain
- boundary(tolerance=0.2) Polygon
Returns the convex hull boundary polygon from the grid.
- Parameters:
tolerance (float) – Simplify polygon shape based on maximum distance from original geometry, see https://shapely.readthedocs.io/en/stable/manual.html#object.simplify.
- Returns:
polygon – See https://shapely.readthedocs.io/en/stable/manual.html#Polygon
- Return type:
shapely.Polygon
- boundary_points(spacing=None, tolerance=0.2) tuple
Returns array of coordinates from boundary polygon.
- Parameters:
tolerance (float) – Simplify polygon shape based on maximum distance from original geometry, see https://shapely.readthedocs.io/en/stable/manual.html#object.simplify.
spacing (float) – If specified, points are returned evenly spaced along the boundary at the specified spacing, otherwise all points are returned.
Returns
--------
points (tuple) – Tuple of x and y coordinates of the boundary points.
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- validator generate » all fields[source]
Generate the grid from the provided parameters.
- plot(ax=None, figsize=None, fscale=10, buffer=0.1, borders=True, land=True, coastline=True)
Plot the grid
- property maxx: float
- property maxy: float
- property minx: float
- property miny: float
- property x: ndarray
- property xlen
- property y: ndarray
- property ylen
Data#
Rompy core data objects.
- pydantic model rompy.core.data.DataBase[source]
Base class for data objects.
Show JSON schema
{ "title": "DataBase", "description": "Base class for data objects.", "type": "object", "properties": { "model_type": { "const": "base", "default": "base", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "id": { "default": "data", "description": "Unique identifier for this data source", "title": "Id", "type": "string" } }, "additionalProperties": false }
- Fields:
id (str)
model_type (Literal['base'])
- field id: str = 'data'
Unique identifier for this data source
- field model_type: Literal['base'] = 'base'
Model type discriminator
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- abstract get(destdir: str | Path, *args, **kwargs) Path [source]
Abstract method to get the data.
- pydantic model rompy.core.data.DataBlob[source]
Data source for model ingestion.
Generic data source for files that either need to be copied to the model directory or linked if link is set to True.
Show JSON schema
{ "title": "DataBlob", "description": "Data source for model ingestion.\n\nGeneric data source for files that either need to be copied to the model directory\nor linked if `link` is set to True.", "type": "object", "properties": { "model_type": { "default": "data_blob", "description": "Model type discriminator", "enum": [ "data_blob", "data_link" ], "title": "Model Type", "type": "string" }, "id": { "default": "data", "description": "Unique identifier for this data source", "title": "Id", "type": "string" }, "source": { "description": "URI of the data source, either a local file path or a remote uri", "title": "Source" }, "link": { "default": false, "description": "Whether to create a symbolic link instead of copying the file", "title": "Link", "type": "boolean" } }, "additionalProperties": false, "required": [ "source" ] }
- Fields:
- field id: str = 'data'
Unique identifier for this data source
- field link: bool = False
Whether to create a symbolic link instead of copying the file
- field model_type: Literal['data_blob', 'data_link'] = 'data_blob'
Model type discriminator
- field source: AnyPath [Required]
URI of the data source, either a local file path or a remote uri
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- get(destdir: str | Path, name: str | None = None, *args, **kwargs) Path [source]
Copy or link the data source to a new directory.
- Parameters:
destdir (str | Path) – The destination directory to copy or link the data source to.
- Returns:
The path to the copied file or created symlink.
- Return type:
Path
- pydantic model rompy.core.data.DataGrid[source]
Data object for gridded source data.
Generic data object for xarray datasets that with gridded spatial dimensions
Note
The fields filter_grid and filter_time trigger updates to the crop filter from the grid and time range objects passed to the get method. This is useful for data sources that are not defined on the same grid as the model grid or the same time range as the model run.
Show JSON schema
{ "title": "DataGrid", "description": "Data object for gridded source data.\n\nGeneric data object for xarray datasets that with gridded spatial dimensions\n\nNote\n----\nThe fields `filter_grid` and `filter_time` trigger updates to the crop filter from\nthe grid and time range objects passed to the get method. This is useful for data\nsources that are not defined on the same grid as the model grid or the same time\nrange as the model run.", "type": "object", "properties": { "model_type": { "const": "grid", "default": "grid", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "id": { "default": "data", "description": "Unique identifier for this data source", "title": "Id", "type": "string" }, "source": { "description": "Source reader, must return an xarray gridded dataset in the open method", "discriminator": { "mapping": { "csv": "#/$defs/SourceTimeseriesCSV", "datamesh": "#/$defs/SourceDatamesh", "file": "#/$defs/SourceFile", "intake": "#/$defs/SourceIntake", "wavespectra": "#/$defs/SourceWavespectra" }, "propertyName": "model_type" }, "oneOf": [ { "$ref": "#/$defs/SourceTimeseriesCSV" }, { "$ref": "#/$defs/SourceDatamesh" }, { "$ref": "#/$defs/SourceFile" }, { "$ref": "#/$defs/SourceIntake" }, { "$ref": "#/$defs/SourceWavespectra" } ], "title": "Source" }, "filter": { "anyOf": [ { "$ref": "#/$defs/Filter" }, { "type": "null" } ], "description": "Optional filter specification to apply to the dataset" }, "variables": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": [], "description": "Subset of variables to extract from the dataset", "title": "Variables" }, "coords": { "anyOf": [ { "$ref": "#/$defs/DatasetCoords" }, { "type": "null" } ], "default": { "t": "time", "x": "longitude", "y": "latitude", "z": null, "s": null }, "description": "Names of the coordinates in the dataset" }, "crop_data": { "default": true, "description": "Update crop filters from Grid and Time objects if passed to get method", "title": "Crop Data", "type": "boolean" }, "buffer": { "default": 0.0, "description": "Space to buffer the grid bounding box if `filter_grid` is True", "title": "Buffer", "type": "number" }, "time_buffer": { "default": [ 0, 0 ], "description": "Number of source data timesteps to buffer the time range if `filter_time` is True", "items": { "type": "integer" }, "title": "Time Buffer", "type": "array" } }, "$defs": { "DatasetCoords": { "additionalProperties": false, "description": "Coordinates representation.", "properties": { "t": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "time", "description": "Name of the time coordinate", "title": "T" }, "x": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "longitude", "description": "Name of the x coordinate", "title": "X" }, "y": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "latitude", "description": "Name of the y coordinate", "title": "Y" }, "z": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the z coordinate", "title": "Z" }, "s": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the site coordinate", "title": "S" } }, "title": "DatasetCoords", "type": "object" }, "Filter": { "additionalProperties": false, "properties": { "sort": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Sort" }, "subset": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Subset" }, "crop": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Crop" }, "timenorm": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Timenorm" }, "rename": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Rename" }, "derived": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Derived" } }, "title": "Filter", "type": "object" }, "SourceDatamesh": { "additionalProperties": false, "description": "Source dataset from Datamesh.\n\nDatamesh documentation: https://docs.oceanum.io/datamesh/index.html", "properties": { "model_type": { "const": "datamesh", "default": "datamesh", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "datasource": { "description": "The id of the datasource on Datamesh", "title": "Datasource", "type": "string" }, "token": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "description": "Datamesh API token, taken from the environment if not provided", "title": "Token" }, "kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to pass to `oceanum.datamesh.Connector`", "title": "Kwargs", "type": "object" } }, "required": [ "datasource", "token" ], "title": "SourceDatamesh", "type": "object" }, "SourceFile": { "additionalProperties": false, "description": "Source dataset from file to open with xarray.open_dataset.", "properties": { "model_type": { "const": "file", "default": "file", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "uri": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "description": "Path to the dataset", "title": "Uri" }, "kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to pass to xarray.open_dataset", "title": "Kwargs", "type": "object" }, "variable": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Variable to select from the dataset", "title": "Variable" } }, "required": [ "uri" ], "title": "SourceFile", "type": "object" }, "SourceIntake": { "additionalProperties": false, "description": "Source dataset from intake catalog.\n\nnote\n----\nThe intake catalog can be prescribed either by the URI of an existing catalog file\nor by a YAML string defining the catalog. The YAML string can be obtained from\ncalling the `yaml()` method on an intake dataset instance.", "properties": { "model_type": { "const": "intake", "default": "intake", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "dataset_id": { "description": "The id of the dataset to read in the catalog", "title": "Dataset Id", "type": "string" }, "catalog_uri": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "The URI of the catalog to read from", "title": "Catalog Uri" }, "catalog_yaml": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "The YAML string of the catalog to read from", "title": "Catalog Yaml" }, "kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to define intake dataset parameters", "title": "Kwargs", "type": "object" } }, "required": [ "dataset_id" ], "title": "SourceIntake", "type": "object" }, "SourceTimeseriesCSV": { "additionalProperties": false, "description": "Timeseries source class from CSV file.\n\nThis class should return a timeseries from a CSV file. The dataset variables are\ndefined from the column headers, therefore the appropriate read_csv kwargs must be\npassed to allow defining the columns. The time index is defined from column name\nidentified by the tcol field.", "properties": { "model_type": { "const": "csv", "default": "csv", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "filename": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "description": "Path to the csv file", "title": "Filename" }, "tcol": { "default": "time", "description": "Name of the column containing the time data", "title": "Tcol", "type": "string" }, "read_csv_kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to pass to pandas.read_csv", "title": "Read Csv Kwargs", "type": "object" } }, "required": [ "filename" ], "title": "SourceTimeseriesCSV", "type": "object" }, "SourceWavespectra": { "additionalProperties": false, "description": "Wavespectra dataset from wavespectra reader.", "properties": { "model_type": { "const": "wavespectra", "default": "wavespectra", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "uri": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "description": "Path to the dataset", "title": "Uri" }, "reader": { "description": "Name of the wavespectra reader to use, e.g., read_swan", "title": "Reader", "type": "string" }, "kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to pass to the wavespectra reader", "title": "Kwargs", "type": "object" } }, "required": [ "uri", "reader" ], "title": "SourceWavespectra", "type": "object" } }, "additionalProperties": false, "required": [ "source" ] }
- Fields:
buffer (float)
coords (Optional[DatasetCoords])
crop_data (bool)
filter (Optional[Filter])
id (str)
time_buffer (list[int])
variables (Optional[list[str]])
- field buffer: float = 0.0
Space to buffer the grid bounding box if filter_grid is True
- field coords: DatasetCoords | None = DatasetCoords(t='time', x='longitude', y='latitude', z=None, s=None)
Names of the coordinates in the dataset
- field crop_data: bool = True
Update crop filters from Grid and Time objects if passed to get method
- field filter: Filter | None [Optional]
Optional filter specification to apply to the dataset
- field id: str = 'data'
Unique identifier for this data source
- field model_type: Literal['grid'] = 'grid'
Model type discriminator
- field source: SourceTimeseriesCSV | SourceDatamesh | SourceFile | SourceIntake | SourceWavespectra [Required]
Source reader, must return an xarray gridded dataset in the open method
- field time_buffer: list[int] = [0, 0]
Number of source data timesteps to buffer the time range if filter_time is True
- field variables: list[str] | None = []
Subset of variables to extract from the dataset
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- get(destdir: str | Path, grid: BaseGrid | RegularGrid | None = None, time: TimeRange | None = None) Path
Write the data source to a new location.
- Parameters:
destdir (str | Path) – The destination directory to write the netcdf data to.
grid (GRID_TYPES, optional) – The grid to filter the data to, only used if self.crop_data is True.
time (TimeRange, optional) – The times to filter the data to, only used if self.crop_data is True.
- Returns:
outfile – The path to the written file.
- Return type:
Path
- plot(param, isel={}, model_grid=None, cmap='turbo', figsize=None, fscale=10, borders=True, land=True, coastline=True, **kwargs)[source]
Plot the grid.
- property ds
Return the xarray dataset for this data source.
- property outfile: str
- pydantic model rompy.core.data.DataPoint[source]
Data object for timeseries source data.
Generic data object for xarray datasets that only have time as a dimension and do not need to be cropped to a specific grid.
Show JSON schema
{ "title": "DataPoint", "description": "Data object for timeseries source data.\n\nGeneric data object for xarray datasets that only have time as a dimension and do\nnot need to be cropped to a specific grid.", "type": "object", "properties": { "model_type": { "const": "point", "default": "point", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "id": { "default": "data", "description": "Unique identifier for this data source", "title": "Id", "type": "string" }, "source": { "description": "Source reader, must return an xarray timeseries point dataset in the open method", "discriminator": { "mapping": { "csv": "#/$defs/SourceTimeseriesCSV" }, "propertyName": "model_type" }, "oneOf": [ { "$ref": "#/$defs/SourceTimeseriesCSV" } ], "title": "Source" }, "filter": { "anyOf": [ { "$ref": "#/$defs/Filter" }, { "type": "null" } ], "description": "Optional filter specification to apply to the dataset" }, "variables": { "anyOf": [ { "items": { "type": "string" }, "type": "array" }, { "type": "null" } ], "default": [], "description": "Subset of variables to extract from the dataset", "title": "Variables" }, "coords": { "anyOf": [ { "$ref": "#/$defs/DatasetCoords" }, { "type": "null" } ], "default": { "t": "time", "x": "longitude", "y": "latitude", "z": null, "s": null }, "description": "Names of the coordinates in the dataset" }, "crop_data": { "default": true, "description": "Update crop filters from Grid and Time objects if passed to get method", "title": "Crop Data", "type": "boolean" }, "buffer": { "default": 0.0, "description": "Space to buffer the grid bounding box if `filter_grid` is True", "title": "Buffer", "type": "number" }, "time_buffer": { "default": [ 0, 0 ], "description": "Number of source data timesteps to buffer the time range if `filter_time` is True", "items": { "type": "integer" }, "title": "Time Buffer", "type": "array" } }, "$defs": { "DatasetCoords": { "additionalProperties": false, "description": "Coordinates representation.", "properties": { "t": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "time", "description": "Name of the time coordinate", "title": "T" }, "x": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "longitude", "description": "Name of the x coordinate", "title": "X" }, "y": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "latitude", "description": "Name of the y coordinate", "title": "Y" }, "z": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the z coordinate", "title": "Z" }, "s": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the site coordinate", "title": "S" } }, "title": "DatasetCoords", "type": "object" }, "Filter": { "additionalProperties": false, "properties": { "sort": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Sort" }, "subset": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Subset" }, "crop": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Crop" }, "timenorm": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Timenorm" }, "rename": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Rename" }, "derived": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Derived" } }, "title": "Filter", "type": "object" }, "SourceTimeseriesCSV": { "additionalProperties": false, "description": "Timeseries source class from CSV file.\n\nThis class should return a timeseries from a CSV file. The dataset variables are\ndefined from the column headers, therefore the appropriate read_csv kwargs must be\npassed to allow defining the columns. The time index is defined from column name\nidentified by the tcol field.", "properties": { "model_type": { "const": "csv", "default": "csv", "description": "Model type discriminator", "title": "Model Type", "type": "string" }, "filename": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "description": "Path to the csv file", "title": "Filename" }, "tcol": { "default": "time", "description": "Name of the column containing the time data", "title": "Tcol", "type": "string" }, "read_csv_kwargs": { "additionalProperties": true, "default": {}, "description": "Keyword arguments to pass to pandas.read_csv", "title": "Read Csv Kwargs", "type": "object" } }, "required": [ "filename" ], "title": "SourceTimeseriesCSV", "type": "object" } }, "additionalProperties": false, "required": [ "source" ] }
- Fields:
buffer (float)
coords (rompy.core.types.DatasetCoords | None)
crop_data (bool)
filter (rompy.core.filters.Filter | None)
id ()
model_type (Literal['point'])
source (rompy.core.source.SourceTimeseriesCSV)
time_buffer (list[int])
variables (list[str] | None)
- field buffer: float = 0.0
Space to buffer the grid bounding box if filter_grid is True
- field coords: DatasetCoords | None = DatasetCoords(t='time', x='longitude', y='latitude', z=None, s=None)
Names of the coordinates in the dataset
- field crop_data: bool = True
Update crop filters from Grid and Time objects if passed to get method
- field filter: Filter | None [Optional]
Optional filter specification to apply to the dataset
- field id: str = 'data'
Unique identifier for this data source
- field model_type: Literal['point'] = 'point'
Model type discriminator
- field source: SourceTimeseriesCSV [Required]
Source reader, must return an xarray timeseries point dataset in the open method
- field time_buffer: list[int] = [0, 0]
Number of source data timesteps to buffer the time range if filter_time is True
- field variables: list[str] | None = []
Subset of variables to extract from the dataset
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- get(destdir: str | Path, grid: BaseGrid | RegularGrid | None = None, time: TimeRange | None = None) Path [source]
Write the data source to a new location.
- Parameters:
destdir (str | Path) – The destination directory to write the netcdf data to.
grid (GRID_TYPES, optional) – The grid to filter the data to, only used if self.crop_data is True.
time (TimeRange, optional) – The times to filter the data to, only used if self.crop_data is True.
- Returns:
outfile – The path to the written file.
- Return type:
Path
- property ds
Return the xarray dataset for this data source.
- property outfile: str
Filters#
- pydantic model rompy.core.filters.Filter[source]
Show JSON schema
{ "title": "Filter", "type": "object", "properties": { "sort": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Sort" }, "subset": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Subset" }, "crop": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Crop" }, "timenorm": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Timenorm" }, "rename": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Rename" }, "derived": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": {}, "title": "Derived" } }, "additionalProperties": false }
- Fields:
crop (dict | None)
derived (dict | None)
rename (dict | None)
sort (dict | None)
subset (dict | None)
timenorm (dict | None)
- Validators:
convert_slices
»crop
- field crop: dict | None = {}
- Validated by:
convert_slices
- field derived: dict | None = {}
- field rename: dict | None = {}
- field sort: dict | None = {}
- field subset: dict | None = {}
- field timenorm: dict | None = {}
- validator convert_slices » crop[source]
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- rompy.core.filters.crop_filter(ds, **data_slice) Dataset [source]
Crop dataset.
- Parameters:
ds (xr.Dataset) – Input dataset to transform.
data_slice (Iterable) – Data slice to crop
- Returns:
ds
- Return type:
xr.Dataset
- rompy.core.filters.derived_filter(ds, derived_variables)[source]
Add derived variable to Dataset.
- Parameters:
ds (xarray.Dataset) – Input dataset to add derived variables to.
derived_variables (dict) – Mapping {derived_variable_name: derived_variable_definition} where derived_variable_definition is a string to be evaluated defining some transformation based on existing variables in the input dataset ds.
- Returns:
ds – Input dataset with extra derived variables.
- Return type:
xarray.Dataset
Example
>>> import xarray as xr >>> ds = xr.DataArray([-10, -11], coords={"x": [0, 1]}).to_dataset(name="elevation") >>> ds = derived_filter(ds, {"depth": "ds.elevation * -1"})
- rompy.core.filters.get_filter_fns() dict [source]
Get dictionary of filter functions
- rompy.core.filters.rename_filter(ds, **varmap) Dataset [source]
Rename variables in dataset
- Parameters:
ds (xr.Dataset) – Input dataset to transform.
varmap (dict) – Dictionary of variable names to rename
- Returns:
ds
- Return type:
xr.Dataset
- rompy.core.filters.subset_filter(ds, data_vars=None) Dataset [source]
Subset data variables from dataset.
- Parameters:
ds (xr.Dataset) – Input dataset to transform.
data_vars (Iterable) – Variables to subset from ds.
- Returns:
ds
- Return type:
xr.Dataset
- rompy.core.filters.timenorm_filter(ds, interval='hour', reftime=None) Dataset [source]
Normalize time to lead time in hours
- Parameters:
ds (xr.Dataset) – Input dataset to transform.
interval (str, optional) – Time interval to normalize to, by default “hour”
reftime (str, optional) – Reference time variable, by default None
- Returns:
ds
- Return type:
xr.Dataset
Boundary#
- pydantic model rompy.core.render.TemplateRenderer[source]
Template renderer class that provides enhanced logging and formatting.
This class wraps the cookiecutter template rendering process and provides detailed formatting through the _format_value method.
Show JSON schema
{ "title": "TemplateRenderer", "description": "Template renderer class that provides enhanced logging and formatting.\n\nThis class wraps the cookiecutter template rendering process and provides\ndetailed formatting through the _format_value method.", "type": "object", "properties": { "template": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Template" }, "output_dir": { "anyOf": [ { "type": "string" }, { "format": "path", "type": "string" } ], "title": "Output Dir" }, "context": { "additionalProperties": true, "title": "Context", "type": "object" }, "checkout": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Checkout" } }, "additionalProperties": false, "required": [ "template", "output_dir", "context" ] }
- Fields:
checkout (str | None)
context (Dict[str, Any])
output_dir (str | pathlib.Path)
template (str | pathlib.Path)
- field checkout: str | None = None
- field context: Dict[str, Any] [Required]
- field output_dir: str | Path [Required]
- field template: str | Path [Required]
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- rompy.core.render.find_template(repo_dir, env)[source]
Determine which child directory of repo_dir is the project template.
- Parameters:
repo_dir – Local directory of newly cloned repo.
- Returns project_template:
Relative path to project template.
- rompy.core.render.render(context, template, output_dir, checkout=None)[source]
Render the template with the given context.
This function handles the rendering process and provides detailed progress information during the rendering.
- Parameters:
context (dict) – The context to use for rendering
template (str) – The template directory or URL
output_dir (str) – The output directory
checkout (str, optional) – The branch, tag or commit to checkout
- Returns:
The path to the rendered template
- Return type:
str
- rompy.core.render.repository_has_cookiecutter_json(repo_directory)[source]
Determine if repo_directory contains a cookiecutter.json file.
- Parameters:
repo_directory – The candidate repository directory.
- Returns:
True if the repo_directory is valid, else False.
Types#
Rompy types.
- pydantic model rompy.core.types.Bbox[source]
Bounding box
Show JSON schema
{ "title": "Bbox", "description": "Bounding box", "type": "object", "properties": { "minlon": { "$ref": "#/$defs/Longitude", "description": "Minimum longitude" }, "minlat": { "$ref": "#/$defs/Latitude", "description": "Minimum latitude" }, "maxlon": { "$ref": "#/$defs/Longitude", "description": "Maximum longitude" }, "maxlat": { "$ref": "#/$defs/Latitude", "description": "Maximum latitude" } }, "$defs": { "Latitude": { "description": "Latitude", "properties": { "lat": { "description": "Latitude", "maximum": 90, "minimum": -90, "title": "Lat", "type": "number" } }, "required": [ "lat" ], "title": "Latitude", "type": "object" }, "Longitude": { "description": "Longitude", "properties": { "lon": { "description": "Longitude", "maximum": 180, "minimum": -180, "title": "Lon", "type": "number" } }, "required": [ "lon" ], "title": "Longitude", "type": "object" } }, "required": [ "minlon", "minlat", "maxlon", "maxlat" ] }
- Fields:
maxlat (rompy.core.types.Latitude)
maxlon (rompy.core.types.Longitude)
minlat (rompy.core.types.Latitude)
minlon (rompy.core.types.Longitude)
- Validators:
validate_coords
»all fields
- field maxlat: Latitude [Required]
Maximum latitude
- Validated by:
validate_coords
- field maxlon: Longitude [Required]
Maximum longitude
- Validated by:
validate_coords
- field minlat: Latitude [Required]
Minimum latitude
- Validated by:
validate_coords
- field minlon: Longitude [Required]
Minimum longitude
- Validated by:
validate_coords
- contains(other)[source]
- expand(amount)[source]
- classmethod from_geojson(geojson)[source]
- classmethod from_polygon(polygon)[source]
- classmethod from_wkb(wkb)[source]
- classmethod from_wkt(wkt)[source]
- intersection(other)[source]
- intersects(other)[source]
- scale(amount)[source]
- shrink(amount)[source]
- to_geojson()[source]
- to_wkb()[source]
- to_wkt()[source]
- translate(amount)[source]
- union(other)[source]
- validator validate_coords » all fields[source]
- property area
- property center
- property height
- property width
- pydantic model rompy.core.types.Coordinate[source]
Coordinate
Show JSON schema
{ "title": "Coordinate", "description": "Coordinate", "type": "object", "properties": { "lon": { "description": "Longitude", "title": "Lon", "type": "number" }, "lat": { "description": "Latitude", "title": "Lat", "type": "number" } }, "required": [ "lon", "lat" ] }
- Fields:
lat (float)
lon (float)
- Validators:
validate_lat
»lat
validate_lon
»lon
- field lat: float [Required]
Latitude
- Validated by:
validate_lat
- field lon: float [Required]
Longitude
- Validated by:
validate_lon
- validator validate_lat » lat[source]
- validator validate_lon » lon[source]
- pydantic model rompy.core.types.DatasetCoords[source]
Coordinates representation.
Show JSON schema
{ "title": "DatasetCoords", "description": "Coordinates representation.", "type": "object", "properties": { "t": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "time", "description": "Name of the time coordinate", "title": "T" }, "x": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "longitude", "description": "Name of the x coordinate", "title": "X" }, "y": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": "latitude", "description": "Name of the y coordinate", "title": "Y" }, "z": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the z coordinate", "title": "Z" }, "s": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Name of the site coordinate", "title": "S" } }, "additionalProperties": false }
- Fields:
s (str | None)
t (str | None)
x (str | None)
y (str | None)
z (str | None)
- field s: str | None = None
Name of the site coordinate
- field t: str | None = 'time'
Name of the time coordinate
- field x: str | None = 'longitude'
Name of the x coordinate
- field y: str | None = 'latitude'
Name of the y coordinate
- field z: str | None = None
Name of the z coordinate
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
- pydantic model rompy.core.types.Latitude[source]
Latitude
Show JSON schema
{ "title": "Latitude", "description": "Latitude", "type": "object", "properties": { "lat": { "description": "Latitude", "maximum": 90, "minimum": -90, "title": "Lat", "type": "number" } }, "required": [ "lat" ] }
- Fields:
lat (float)
- Validators:
validate_lat
»lat
- field lat: float [Required]
Latitude
- Constraints:
ge = -90
le = 90
- Validated by:
validate_lat
- validator validate_lat » lat[source]
- pydantic model rompy.core.types.Longitude[source]
Longitude
Show JSON schema
{ "title": "Longitude", "description": "Longitude", "type": "object", "properties": { "lon": { "description": "Longitude", "maximum": 180, "minimum": -180, "title": "Lon", "type": "number" } }, "required": [ "lon" ] }
- Fields:
lon (float)
- Validators:
validate_lon
»lon
- field lon: float [Required]
Longitude
- Constraints:
ge = -180
le = 180
- Validated by:
validate_lon
- validator validate_lon » lon[source]
- pydantic model rompy.core.types.RompyBaseModel[source]
Show JSON schema
{ "title": "RompyBaseModel", "type": "object", "properties": {}, "additionalProperties": false }
- dump_inputs_dict() dict [source]
Return the original inputs as a dictionary.
- dump_inputs_json() str [source]
Return the original inputs as a JSON string.
- pydantic model rompy.core.types.Slice[source]
Basic float or datetime slice representation
Show JSON schema
{ "title": "Slice", "description": "Basic float or datetime slice representation", "type": "object", "properties": { "start": { "anyOf": [ { "type": "number" }, { "format": "date-time", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "description": "Slice start", "title": "Start" }, "stop": { "anyOf": [ { "type": "number" }, { "format": "date-time", "type": "string" }, { "type": "string" }, { "type": "null" } ], "default": null, "description": "Slice stop", "title": "Stop" } } }
- Fields:
start (float | datetime.datetime | str | None)
stop (float | datetime.datetime | str | None)
- field start: float | datetime | str | None = None
Slice start
- field stop: float | datetime | str | None = None
Slice stop
- classmethod from_dict(d: dict)[source]
- classmethod from_slice(s: slice)[source]
- to_slice() slice [source]
- pydantic model rompy.core.types.Spectrum[source]
A class to represent a wave spectrum.
Show JSON schema
{ "title": "Spectrum", "description": "A class to represent a wave spectrum.", "type": "object", "properties": { "fmin": { "default": 0.0464, "description": "Minimum frequency in Hz", "title": "Fmin", "type": "number" }, "fmax": { "default": 1.0, "description": "Maximum frequency in Hz", "title": "Fmax", "type": "number" }, "nfreqs": { "default": 31, "description": "Number of frequency components", "title": "Nfreqs", "type": "integer" }, "ndirs": { "default": 36, "description": "Number of directional components", "title": "Ndirs", "type": "integer" } }, "additionalProperties": false }
- Fields:
fmax (float)
fmin (float)
ndirs (int)
nfreqs (int)
- field fmax: float = 1.0
Maximum frequency in Hz
- field fmin: float = 0.0464
Minimum frequency in Hz
- field ndirs: int = 36
Number of directional components
- field nfreqs: int = 31
Number of frequency components
- dump_inputs_dict() dict
Return the original inputs as a dictionary.
- dump_inputs_json() str
Return the original inputs as a JSON string.
Backend System#
Backend Configurations#
Backend configuration classes for ROMPY.
This module provides Pydantic-based configuration classes for different execution backends. These configurations handle transient execution parameters while maintaining type safety and validation.
- pydantic model rompy.backends.config.BaseBackendConfig[source]
Base class for all backend configurations.
This class defines common configuration parameters that apply to all backend types, such as timeouts and environment variables.
Show JSON schema
{ "title": "BaseBackendConfig", "description": "Base class for all backend configurations.\n\nThis class defines common configuration parameters that apply to all\nbackend types, such as timeouts and environment variables.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" } }, "additionalProperties": false }
- Fields:
env_vars (Dict[str, str])
timeout (int)
working_dir (pathlib.Path | None)
- Validators:
validate_env_vars
»env_vars
validate_working_dir
»working_dir
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- abstract get_backend_class()[source]
Return the backend class that should handle this configuration.
- Returns:
The backend class to use for execution
- validator validate_env_vars » env_vars[source]
Validate environment variables.
- validator validate_working_dir » working_dir[source]
Validate working directory exists if specified.
- pydantic model rompy.backends.config.DockerConfig[source]
Configuration for Docker execution backend.
This configuration is used when running models inside Docker containers with appropriate resource limits and volume mounts.
Show JSON schema
{ "title": "DockerConfig", "description": "Configuration for Docker execution backend.\n\nThis configuration is used when running models inside Docker containers\nwith appropriate resource limits and volume mounts.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" }, "image": { "anyOf": [ { "pattern": "^[a-zA-Z0-9._:/-]+$", "type": "string" }, { "type": "null" } ], "default": null, "description": "Docker image to use (required if not building from dockerfile)", "title": "Image" }, "dockerfile": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Path to Dockerfile to build (alternative to image)", "title": "Dockerfile" }, "executable": { "default": "/usr/local/bin/run.sh", "description": "Path to executable inside the container", "title": "Executable", "type": "string" }, "cpu": { "default": 1, "description": "Number of CPU cores to allocate", "maximum": 128, "minimum": 1, "title": "Cpu", "type": "integer" }, "memory": { "anyOf": [ { "pattern": "^\\d+[kmgKMG]?$", "type": "string" }, { "type": "null" } ], "default": null, "description": "Memory limit (e.g., '2g', '512m')", "title": "Memory" }, "mpiexec": { "default": "", "description": "MPI execution command (if using MPI)", "title": "Mpiexec", "type": "string" }, "build_args": { "additionalProperties": { "type": "string" }, "description": "Arguments to pass to docker build", "title": "Build Args", "type": "object" }, "build_context": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Docker build context directory (defaults to dockerfile parent directory)", "title": "Build Context" }, "volumes": { "description": "Additional volumes to mount (format: 'host_path:container_path')", "items": { "type": "string" }, "title": "Volumes", "type": "array" }, "remove_container": { "default": true, "description": "Whether to remove container after execution", "title": "Remove Container", "type": "boolean" }, "user": { "default": "root", "description": "User to run as inside the container", "title": "User", "type": "string" } }, "additionalProperties": false, "examples": [ { "cpu": 4, "image": "swan:latest", "memory": "2g", "timeout": 7200, "volumes": [ "/data:/app/data" ] }, { "build_args": { "VERSION": "1.0" }, "cpu": 2, "dockerfile": "./docker/Dockerfile", "mpiexec": "mpirun" } ] }
- Fields:
build_args (Dict[str, str])
build_context (pathlib.Path | None)
cpu (int)
dockerfile (pathlib.Path | None)
env_vars ()
executable (str)
image (str | None)
memory (str | None)
mpiexec (str)
remove_container (bool)
timeout ()
user (str)
volumes (List[str])
working_dir ()
- Validators:
validate_build_context_exists
»build_context
validate_dockerfile_exists
»dockerfile
validate_env_vars
»env_vars
validate_image_or_dockerfile
»dockerfile
validate_image_or_dockerfile
»image
validate_memory_format
»memory
validate_volumes
»volumes
validate_working_dir
»working_dir
- field build_args: Dict[str, str] [Optional]
Arguments to pass to docker build
- field build_context: Path | None = None
Docker build context directory (defaults to dockerfile parent directory)
- Validated by:
validate_build_context_exists
- field cpu: int = 1
Number of CPU cores to allocate
- Constraints:
ge = 1
le = 128
- field dockerfile: Path | None = None
Path to Dockerfile to build (alternative to image)
- Validated by:
validate_dockerfile_exists
validate_image_or_dockerfile
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field executable: str = '/usr/local/bin/run.sh'
Path to executable inside the container
- field image: str | None = None
Docker image to use (required if not building from dockerfile)
- Constraints:
pattern = ^[a-zA-Z0-9._:/-]+$
- Validated by:
validate_image_or_dockerfile
- field memory: str | None = None
Memory limit (e.g., ‘2g’, ‘512m’)
- Constraints:
pattern = ^d+[kmgKMG]?$
- Validated by:
validate_memory_format
- field mpiexec: str = ''
MPI execution command (if using MPI)
- field remove_container: bool = True
Whether to remove container after execution
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field user: str = 'root'
User to run as inside the container
- field volumes: List[str] [Optional]
Additional volumes to mount (format: ‘host_path:container_path’)
- Validated by:
validate_volumes
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- get_backend_class()[source]
Return the DockerRunBackend class.
- validator validate_build_context_exists » build_context[source]
Validate build context directory exists if specified.
- validator validate_dockerfile_exists » dockerfile[source]
Validate dockerfile path format (should be relative).
- validator validate_env_vars » env_vars
Validate environment variables.
- validator validate_image_or_dockerfile » dockerfile, image[source]
Validate that either image or dockerfile is provided, but not both.
- validator validate_memory_format » memory[source]
Validate memory format.
- validator validate_volumes » volumes[source]
Validate volume mount specifications.
- validator validate_working_dir » working_dir
Validate working directory exists if specified.
- pydantic model rompy.backends.config.LocalConfig[source]
Configuration for local execution backend.
This configuration is used when running models directly on the local system using the system’s Python interpreter or shell commands.
Show JSON schema
{ "title": "LocalConfig", "description": "Configuration for local execution backend.\n\nThis configuration is used when running models directly on the local system\nusing the system's Python interpreter or shell commands.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" }, "command": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Optional shell command to run instead of config.run()", "title": "Command" }, "shell": { "default": true, "description": "Whether to execute commands through the shell", "title": "Shell", "type": "boolean" }, "capture_output": { "default": true, "description": "Whether to capture stdout and stderr", "title": "Capture Output", "type": "boolean" } }, "additionalProperties": false, "examples": [ { "command": "python run_model.py", "env_vars": { "OMP_NUM_THREADS": "4" }, "timeout": 7200 }, { "timeout": 3600, "working_dir": "/path/to/model/dir" } ] }
- Fields:
capture_output (bool)
command (str | None)
env_vars ()
shell (bool)
timeout ()
working_dir ()
- Validators:
validate_env_vars
»env_vars
validate_working_dir
»working_dir
- field capture_output: bool = True
Whether to capture stdout and stderr
- field command: str | None = None
Optional shell command to run instead of config.run()
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field shell: bool = True
Whether to execute commands through the shell
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- get_backend_class()[source]
Return the LocalRunBackend class.
- validator validate_env_vars » env_vars
Validate environment variables.
- validator validate_working_dir » working_dir
Validate working directory exists if specified.
Run Backends#
Local execution backend for model runs.
This module provides the local run backend implementation.
- class rompy.run.LocalRunBackend[source]
Execute models locally using the system’s Python interpreter.
This is the simplest backend that just runs the model directly on the local system.
- run(model_run, config: LocalConfig, workspace_dir: str | None = None) bool [source]
Run the model locally.
- Parameters:
model_run – The ModelRun instance to execute
config – LocalConfig instance with execution parameters
workspace_dir – Path to the generated workspace directory (if None, will generate)
- Returns:
True if execution was successful, False otherwise
- Raises:
ValueError – If model_run is invalid
TimeoutError – If execution exceeds timeout
Postprocessors#
Backend Registry#
Backend configuration system for ROMPY.
This module provides Pydantic-based configuration classes for different execution backends, enabling type-safe and validated backend configurations.
- pydantic model rompy.backends.BaseBackendConfig[source]
Base class for all backend configurations.
This class defines common configuration parameters that apply to all backend types, such as timeouts and environment variables.
Show JSON schema
{ "title": "BaseBackendConfig", "description": "Base class for all backend configurations.\n\nThis class defines common configuration parameters that apply to all\nbackend types, such as timeouts and environment variables.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" } }, "additionalProperties": false }
- Fields:
env_vars (Dict[str, str])
timeout (int)
working_dir (pathlib.Path | None)
- Validators:
validate_env_vars
»env_vars
validate_working_dir
»working_dir
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- abstract get_backend_class()[source]
Return the backend class that should handle this configuration.
- Returns:
The backend class to use for execution
- validator validate_env_vars » env_vars[source]
Validate environment variables.
- validator validate_working_dir » working_dir[source]
Validate working directory exists if specified.
- pydantic model rompy.backends.DockerConfig[source]
Configuration for Docker execution backend.
This configuration is used when running models inside Docker containers with appropriate resource limits and volume mounts.
Show JSON schema
{ "title": "DockerConfig", "description": "Configuration for Docker execution backend.\n\nThis configuration is used when running models inside Docker containers\nwith appropriate resource limits and volume mounts.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" }, "image": { "anyOf": [ { "pattern": "^[a-zA-Z0-9._:/-]+$", "type": "string" }, { "type": "null" } ], "default": null, "description": "Docker image to use (required if not building from dockerfile)", "title": "Image" }, "dockerfile": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Path to Dockerfile to build (alternative to image)", "title": "Dockerfile" }, "executable": { "default": "/usr/local/bin/run.sh", "description": "Path to executable inside the container", "title": "Executable", "type": "string" }, "cpu": { "default": 1, "description": "Number of CPU cores to allocate", "maximum": 128, "minimum": 1, "title": "Cpu", "type": "integer" }, "memory": { "anyOf": [ { "pattern": "^\\d+[kmgKMG]?$", "type": "string" }, { "type": "null" } ], "default": null, "description": "Memory limit (e.g., '2g', '512m')", "title": "Memory" }, "mpiexec": { "default": "", "description": "MPI execution command (if using MPI)", "title": "Mpiexec", "type": "string" }, "build_args": { "additionalProperties": { "type": "string" }, "description": "Arguments to pass to docker build", "title": "Build Args", "type": "object" }, "build_context": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Docker build context directory (defaults to dockerfile parent directory)", "title": "Build Context" }, "volumes": { "description": "Additional volumes to mount (format: 'host_path:container_path')", "items": { "type": "string" }, "title": "Volumes", "type": "array" }, "remove_container": { "default": true, "description": "Whether to remove container after execution", "title": "Remove Container", "type": "boolean" }, "user": { "default": "root", "description": "User to run as inside the container", "title": "User", "type": "string" } }, "additionalProperties": false, "examples": [ { "cpu": 4, "image": "swan:latest", "memory": "2g", "timeout": 7200, "volumes": [ "/data:/app/data" ] }, { "build_args": { "VERSION": "1.0" }, "cpu": 2, "dockerfile": "./docker/Dockerfile", "mpiexec": "mpirun" } ] }
- Fields:
build_args (Dict[str, str])
build_context (pathlib.Path | None)
cpu (int)
dockerfile (pathlib.Path | None)
env_vars (Dict[str, str])
executable (str)
image (str | None)
memory (str | None)
mpiexec (str)
remove_container (bool)
timeout (int)
user (str)
volumes (List[str])
working_dir (Optional[Path])
- Validators:
validate_build_context_exists
»build_context
validate_dockerfile_exists
»dockerfile
validate_env_vars
»env_vars
validate_image_or_dockerfile
»dockerfile
validate_image_or_dockerfile
»image
validate_memory_format
»memory
validate_volumes
»volumes
validate_working_dir
»working_dir
- field build_args: Dict[str, str] [Optional]
Arguments to pass to docker build
- field build_context: Path | None = None
Docker build context directory (defaults to dockerfile parent directory)
- Validated by:
validate_build_context_exists
- field cpu: int = 1
Number of CPU cores to allocate
- Constraints:
ge = 1
le = 128
- field dockerfile: Path | None = None
Path to Dockerfile to build (alternative to image)
- Validated by:
validate_dockerfile_exists
validate_image_or_dockerfile
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field executable: str = '/usr/local/bin/run.sh'
Path to executable inside the container
- field image: str | None = None
Docker image to use (required if not building from dockerfile)
- Constraints:
pattern = ^[a-zA-Z0-9._:/-]+$
- Validated by:
validate_image_or_dockerfile
- field memory: str | None = None
Memory limit (e.g., ‘2g’, ‘512m’)
- Constraints:
pattern = ^d+[kmgKMG]?$
- Validated by:
validate_memory_format
- field mpiexec: str = ''
MPI execution command (if using MPI)
- field remove_container: bool = True
Whether to remove container after execution
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field user: str = 'root'
User to run as inside the container
- field volumes: List[str] [Optional]
Additional volumes to mount (format: ‘host_path:container_path’)
- Validated by:
validate_volumes
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- get_backend_class()[source]
Return the DockerRunBackend class.
- model_post_init(_DockerConfig__context) None [source]
Validate that either image or dockerfile is provided, but not both.
- validator validate_build_context_exists » build_context[source]
Validate build context directory exists if specified.
- validator validate_dockerfile_exists » dockerfile[source]
Validate dockerfile path format (should be relative).
- validator validate_image_or_dockerfile » dockerfile, image[source]
Validate that either image or dockerfile is provided, but not both.
- validator validate_memory_format » memory[source]
Validate memory format.
- validator validate_volumes » volumes[source]
Validate volume mount specifications.
- pydantic model rompy.backends.LocalConfig[source]
Configuration for local execution backend.
This configuration is used when running models directly on the local system using the system’s Python interpreter or shell commands.
Show JSON schema
{ "title": "LocalConfig", "description": "Configuration for local execution backend.\n\nThis configuration is used when running models directly on the local system\nusing the system's Python interpreter or shell commands.", "type": "object", "properties": { "timeout": { "default": 3600, "description": "Maximum execution time in seconds (1 minute to 24 hours)", "maximum": 86400, "minimum": 60, "title": "Timeout", "type": "integer" }, "env_vars": { "additionalProperties": { "type": "string" }, "description": "Additional environment variables to set during execution", "title": "Env Vars", "type": "object" }, "working_dir": { "anyOf": [ { "format": "path", "type": "string" }, { "type": "null" } ], "default": null, "description": "Working directory for execution (defaults to model output directory)", "title": "Working Dir" }, "command": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "description": "Optional shell command to run instead of config.run()", "title": "Command" }, "shell": { "default": true, "description": "Whether to execute commands through the shell", "title": "Shell", "type": "boolean" }, "capture_output": { "default": true, "description": "Whether to capture stdout and stderr", "title": "Capture Output", "type": "boolean" } }, "additionalProperties": false, "examples": [ { "command": "python run_model.py", "env_vars": { "OMP_NUM_THREADS": "4" }, "timeout": 7200 }, { "timeout": 3600, "working_dir": "/path/to/model/dir" } ] }
- Fields:
capture_output (bool)
command (str | None)
env_vars (Dict[str, str])
shell (bool)
timeout (int)
working_dir (Optional[Path])
- Validators:
validate_env_vars
»env_vars
validate_working_dir
»working_dir
- field capture_output: bool = True
Whether to capture stdout and stderr
- field command: str | None = None
Optional shell command to run instead of config.run()
- field env_vars: Dict[str, str] [Optional]
Additional environment variables to set during execution
- Validated by:
validate_env_vars
- field shell: bool = True
Whether to execute commands through the shell
- field timeout: int = 3600
Maximum execution time in seconds (1 minute to 24 hours)
- Constraints:
ge = 60
le = 86400
- field working_dir: Path | None = None
Working directory for execution (defaults to model output directory)
- Validated by:
validate_working_dir
- get_backend_class()[source]
Return the LocalRunBackend class.