Virtual Icechunk · IOOS Hackathon 2026
What to check before anyone writes build code
Nothing is rewritten, so the files’ layout is the store’s layout.
CoastWatch Ocean Heat Content: three regions on three different lat/lon grids, so three separate stores.
NOAA NDVI: NDVI is chunked 1 × 1200 × 2400, but QA in the same file is unchunked, so any read of QA pulls all 3600 × 7200 values.
PacIOOS: the servers implied HDF5, but the files were NetCDF-3. About 3,200 “replacement” files were compressed HDF5 under the same names.
CEFI daily files: 365 or 366 days each, chunked 100 days. In a test with that layout, appending put 430 of 730 values in the wrong place, with no error.
HYCOM GOFS 3.1: a few analyses are missing from the archive, so the snow crab notebook takes the nearest time that exists.
This is the one thing a virtual store can fix, without touching the data.
units, standard_name, long_name, or a vertical positiveNaN fill value on a coordinateFix it in the store, write down every change, and make the result follow CF conventions.
...
NDVI int16 (1, 3600, 7200) chunks: (1, 1200, 2400) gzip
TIMEOFDAY int16 (1, 3600, 7200) chunks: (1, 1200, 2400) gzip
QA int16 (1, 3600, 7200) chunks: None None
First, tell it how to find the files. It cannot guess:
Then ask:
“Audit the NOAA CDR NDVI files in
s3://noaa-cdr-ndvi-pds/data/(public, us-east-1, one NetCDF per day) for a virtual Icechunk store. Check grids, chunking, encoding, whether they can be appended, missing days and metadata. Read headers only. Write the findings toaudit/report.md. No build code.”
⌂ Home · Auditing your files