Data Updates wamsi-csiem(v1.7.0)

[santi727G]

This thread will record all updates and changes to the v1.7.0 dataset.
Please add a new comment below whenever data files are modified, replaced, or reprocessed.

[santi727G]

Data sync csiem-data v1.6.0 ->

code

Updated:

• \import\wamsi_theme3\CTD\import_theme3ctd_data.m
• \import\wamsi_theme3\SEDDEPO\IMPORTSEDDEPO.m

Added:

• \import_py\WAMSI\importWAMSIWWMSP4_chla.py

data-governance

Updated: site_key and variable_key

data-lake updates

• \CSIRO\Dalseno was added

• WAMSI\WWMSP3\WWMSP3.1_SEDDEPO → WWMSP3.1.3_SEDDEPO: updated to latest available data.

• WAMSI\WWMSP3\WWMSP3.1_CTD → WWMSP3.1.4_CTD: updated to latest available data.

• WAMSI\WWMSP4\WWMSP4_zoop\Chl_a_Westport_process.xlsx: added.

data-warehouse

Updated: csv\wamsi\wwmsp3\ctd, csv\wamsi\wwmsp3\seddepo
Added: csv\wamsi\wwmsp4\chla, csv\csiro\dalseno

Aggregations

• csiem_WAMSI_public.mat was updated with the new data

[santi727G]

Sentinel Data

Code

Created: \code\import_py\ESA\importESASENTINEL2.py

New Sentinel‑2 importer. Recognizes site names (keeps CS_CENTER, Point_6, etc.) and writes matching DATA/HEADER files. No path or date‑format changes.

Updated: \code\import_py\ESA\importESASENTINEL3.py

Previose Sentinel import was change to Santinel 3. Previoes export now land under the SEN3/OLCI warehouse path. No date‑format changes.

data-governance

Updated: site_key and variable_key

• site_key: validated/added coordinates for SEN2 C2RCC‑NN points (id, lat, lon).

• variable_key: confirmed mapping from chl_value to Chlorophyll‑a (IDs and conversion).

data-warehouse

Sentinel‑2 (C2RCC‑NN)

• Exported named sites: 16 DATA files.

• Exported polygons: 11 DATA files.

• Exported numeric points: 3 DATA files (Point_6, Point_8, Point_9).

• Skipped points with no valid values (Point_1–5, 7, 10–13).

Sentinel‑3 (OLCI)

• Exported points: 32 DATA files (one per Point_*).

• Polygon aggregates also present in the OLCI folder.

notes

• All exported DATA use the existing format: Date,Depth,Data,QC with Date as yyyy‑mm‑dd HH:MM:SS (kept consistent with prior loads)

• All SMD were updated

[santi727G]

BOM IDO Data

Code

Created: \code\import_py\BOM\importBOMIDO.py, a BOM IDO importer that consumes the refreshed site/variable keys and writes DATA/HEADER pairs without altering legacy folders or date formats.

data-governance

• site_key: added/validated the BOM IDO station IDs plus lat/lon so each point resolves during the import.

• variable_key: mapped BOM IDO measurements (e.g., temperature, rainfall, wind) to their Chlorophyll‑a-style counterparts with confirmed IDs/conversions.

data-warehouse | BOM IDO

Generated DATA/HEADER outputs for every 1 station was included in the refreshed site_key

[santi727G]

CSIEM mat files update

data-warehouse

csiem_ESA_public and csiem_WAMSI_public have been updated including lasted data

[santi727G]

#WCMA data update

Code:

• Updated: \code\import\WCWA\calc_dates.m

Import was updated in order to solve a date misalignment of oxygen data

• Updated:\code\import\WCWA\import_PhyWQ_1334_09.m

New functions were included to import the dissolved oxigen concentration into data-warehouse

data-warehouse

• Dissolved oxigen concentration has been add to the current available data

• csiem_WCWA_public has been updated to solve a date misalignment of WC dataset and the inclusion of dissolved oxigen concentration