puerto_rico Conversion to Zarr

Checklist for Workflow associated with dataset conversion:

Related to #47 (closed)

Dataset Name: JMA Non-Hydrostatic Model (NHM): Puerto Rico and US Virgin Islands Dynamical Downscaled Climate Change Projections

https://cida.usgs.gov/thredds/catalog/demo/thredds/PuertoRico/FSU/catalog.html

8 OPeNDAP endpoints -- hourly, daily, and monthly

A few dozen variables.

    Float64 lat[lat = 100];
    Float64 lon[lon = 250];
    Int32 time[time = 350592];

Try OPeNDAP directly.


  • Identify Source Data location and access (check the dataset spreadsheet)
  • Collect ownership information (Who do we ask questions of if we have problems?)
  • Create new workflow notebook from template; stash in the ./workflows folder tree in an appropriate spot.
    • Identify landing spot on S3 (currently somewhere in: https://s3.console.aws.amazon.com/s3/buckets/nhgf-development?prefix=workspace/&region=us-west-2)
    • Calculate chunking, layout, compression, etc
    • Run notebook
    • Read test (pattern to be determined by the dataset)
  • Create STAC catalog entry;
    • Verify all metadata
    • Create entry
  • Reportage
    • add notebook and the dask performance report to the repo
    • Calculate summary statistics on output (compression ratio, total size)
    • Save STAC JSON snippet to repo
  • Merge and close the issue.
Edited Oct 23, 2023 by Blodgett, David L.
Assignee Loading
Time tracking Loading