7. Configuration as YAML and the rompy CLI¶

What this shows: the hindcast from Tutorial 6 written as a YAML file, loaded in Python, and generated and run from the command line.

Prerequisites: 6. A nonstationary hindcast.

You will learn:

• how Python objects map to YAML, and what model_type is for
• how to load, check and generate a YAML configuration in Python
• how to validate, generate and run it with the rompy command-line tool

Data used: the same files as Tutorial 6, referenced from 07_yaml_and_cli.yml.

Why YAML?¶

A YAML file holds the whole model in one place. It can be version-controlled, shared, edited without Python, and run on another machine or in a pipeline. It is checked against the same classes as the Python objects, so mistakes are caught before any file is written.

Setup¶

In [1]:

import shutil
from pathlib import Path

import yaml
from pydantic import ValidationError
from rompy.logging import config as logging_config
from rompy.model import ModelRun

logging_config.update(level="WARNING")

CONFIG_FILE = Path("07_yaml_and_cli.yml")
shutil.rmtree("_output/07_yaml_and_cli", ignore_errors=True)

import shutil from pathlib import Path import yaml from pydantic import ValidationError from rompy.logging import config as logging_config from rompy.model import ModelRun logging_config.update(level="WARNING") CONFIG_FILE = Path("07_yaml_and_cli.yml") shutil.rmtree("_output/07_yaml_and_cli", ignore_errors=True)

1. The YAML file¶

The file has the fields of a ModelRun: run_id, output_dir and period, plus config holding the SWAN model. Each nested object has the same fields as its Python class. The grid appears as the values of SWAN's CGRID command, which SwanGrid.component computed in Python.

In [2]:

print(CONFIG_FILE.read_text())

print(CONFIG_FILE.read_text())

# The nonstationary hindcast of Tutorial 6 as a rompy ModelRun.
# Paths are relative to this file's folder.
run_id: hindcast_yaml
output_dir: _output/07_yaml_and_cli
period:
  start: 2023-01-01T00:00
  end: 2023-01-02T00:00
  interval: 10m
config:
  model_type: swan
  startup:
    project:
      name: Perth hindcast
      nr: t07
    set:
      direction_convention: nautical
    mode:
      kind: nonstationary
    coordinates:
      kind:
        model_type: spherical
  cgrid:
    model_type: regular
    grid:
      xp: 114.5
      yp: -32.8
      alp: 0.0
      xlen: 1.4
      ylen: 1.3
      mx: 70
      my: 65
    spectrum:
      mdc: 36
      flow: 0.04
      fhigh: 1.0
  inpgrid:
    model_type: data_interface
    bottom:
      var: bottom
      source:
        model_type: file
        uri: ../data/etopo15s_perth.nc
      z1: z
      fac: -1.0
      coords: {x: longitude, y: latitude}
      buffer: 0.1
    input:
      - var: wind
        source:
          model_type: file
          uri: ../data/era5-20230101.nc
        z1: u10
        z2: v10
        coords: {x: longitude, y: latitude}
        filter:
          sort: {coords: [latitude]}
        buffer: 0.25
  boundary:
    model_type: boundary_interface
    kind:
      model_type: boundnest1
      id: ww3
      source:
        model_type: wavespectra
        uri: ../data/ww3-spectra-20230101-short.nc
        reader: read_ww3
      sel_method: idw
      sel_method_kwargs: {tolerance: 1.5}
      spacing: 0.1
  physics:
    gen:
      model_type: gen3
    breaking:
      model_type: constant
      alpha: 1.0
      gamma: 0.73
    friction:
      model_type: jonswap
      cfjon: 0.038
    triad:
      model_type: triad
  output:
    points:
      model_type: points
      sname: sites
      xp: [114.8, 115.4, 115.71, 115.66]
      yp: [-32.1, -31.9, -31.98, -32.5]
    block:
      model_type: block
      sname: COMPGRID
      fname: swangrid.nc
      output: [depth, hsign, tps, dir, wind]
      times: {delt: PT1H}
    table:
      sname: sites
      format: header
      fname: sites.txt
      output: [time, depth, hsign, tps, dir, wind]
      times: {delt: PT1H}
    specout:
      sname: sites
      fname: spectra.nc
      times: {delt: PT1H}
  lockup:
    compute:
      model_type: nonstat
      initstat: true

Where a field accepts several classes, model_type names the one to use: for example model_type: boundnest1 selects Boundnest1, and model_type: jonswap selects FRICTION_JONSWAP for the friction. Many SWAN fields have several options, so a SWAN YAML file names more classes than most.

2. Load and check in Python¶

Loading the YAML into a ModelRun checks everything and builds the same objects as in Tutorial 6.

In [3]:

conf = yaml.safe_load(CONFIG_FILE.read_text())
modelrun = ModelRun(**conf)

print(type(modelrun.config.boundary.kind).__name__)
print(modelrun.config.physics.render())

conf = yaml.safe_load(CONFIG_FILE.read_text()) modelrun = ModelRun(**conf) print(type(modelrun.config.boundary.kind).__name__) print(modelrun.config.physics.render())

Boundnest1
GEN3 WESTHUYSEN DRAG WU

BREAKING CONSTANT alpha=1.0 gamma=0.73

FRICTION JONSWAP CONSTANT cfjon=0.038

TRIAD

Invalid values are reported with the path to the field. Here the spectral grid has its frequencies the wrong way round:

In [4]:

conf["config"]["cgrid"]["spectrum"]["flow"] = 1.5
try:
    ModelRun(**conf)
except ValidationError as error:
    for err in error.errors():
        print(".".join(str(loc) for loc in err["loc"]), "->", err["msg"])

conf["config"]["cgrid"]["spectrum"]["flow"] = 1.5 try: ModelRun(**conf) except ValidationError as error: for err in error.errors(): print(".".join(str(loc) for loc in err["loc"]), "->", err["msg"])

config.swan.cgrid.regular.spectrum -> Value error, flow must be less than fhigh

3. Generate from Python¶

Calling the ModelRun writes the workspace to output_dir/run_id.

In [5]:

workspace = Path(ModelRun(**yaml.safe_load(CONFIG_FILE.read_text()))())
sorted(p.name for p in workspace.iterdir())

workspace = Path(ModelRun(**yaml.safe_load(CONFIG_FILE.read_text()))()) sorted(p.name for p in workspace.iterdir())

Out[5]:

['INPUT', 'bottom.grd', 'wind.grd', 'ww3.bnd']

4. The rompy command-line tool¶

rompy installs a rompy command that works directly on the YAML file, so no Python code is needed to produce a workspace or run the model. In a notebook, the ! prefix runs a shell command. In a terminal, type the same command without the !.

rompy validate checks the file and exits with an error if anything is wrong:

In [6]:

!rompy validate 07_yaml_and_cli.yml && echo "Configuration is valid"

!rompy validate 07_yaml_and_cli.yml && echo "Configuration is valid"

Configuration is valid

rompy generate writes the workspace:

In [7]:

!rompy generate 07_yaml_and_cli.yml && ls _output/07_yaml_and_cli/hindcast_yaml

!rompy generate 07_yaml_and_cli.yml && ls _output/07_yaml_and_cli/hindcast_yaml

bottom.grd  INPUT  wind.grd  ww3.bnd

rompy run generates the workspace and runs the model with a backend, described in its own small YAML file. 07_docker_backend.yml runs SWAN in the public Docker image:

In [8]:

print(Path("07_docker_backend.yml").read_text())

print(Path("07_docker_backend.yml").read_text())

# Run SWAN in the public Docker image with rompy's Docker backend.
type: docker
image: ghcr.io/rom-py/swan:41.51
executable: swan.exe

The run takes a few minutes, and needs Docker:

In [9]:

!rompy run 07_yaml_and_cli.yml --backend-config 07_docker_backend.yml && echo "Done"

!rompy run 07_yaml_and_cli.yml --backend-config 07_docker_backend.yml && echo "Done"

Done

In [10]:

!grep -c "Error" _output/07_yaml_and_cli/hindcast_yaml/PRINT; ls _output/07_yaml_and_cli/hindcast_yaml

!grep -c "Error" _output/07_yaml_and_cli/hindcast_yaml/PRINT; ls _output/07_yaml_and_cli/hindcast_yaml

0
bottom.grd  norm_end  sites.txt   swangrid.nc	   swaninit  ww3.bnd
INPUT	    PRINT     spectra.nc  swangrid.nc.dum  wind.grd

As in Tutorial 6, a count of 0 errors in PRINT confirms that SWAN completed the computation. Other useful commands are rompy schema (print the JSON schema of the configuration) and rompy --help. Running SWAN covers other backends.

Summary¶

• A YAML file mirrors the Python objects. model_type selects between alternatives.
• ModelRun(**yaml.safe_load(...)) loads and checks it in Python.
• rompy validate, rompy generate and rompy run do the same from the command line; run takes the backend from a second YAML file.

This completes the tutorial. The examples cover each feature in more depth.

---

Previous lesson: notebooks-swan-tutorial-06-nonstationary-hindcast