Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,19 +23,19 @@ Contents
:caption: Methods for Instruments

Data Acquisition <methods/acquisition>
Standardisation <methods/standardisation>
Trim to Deployment <methods/trimming>
Automatic QC <methods/auto_qc>
Apply Calibration <methods/calibration>
Standardisation (Stage 1) <methods/standardisation>
Trim to Deployment (Stage 2) <methods/trimming>
Automatic QC (Stage 3) <methods/auto_qc>
Apply Calibration (Stage 4) <methods/calibration>
Convert to OceanSites <methods/conversion>

.. toctree::
:maxdepth: 1
:caption: Methods for Moorings

Grid in Time <methods/time_gridding>
Grid Vertically <methods/vertical_gridding>
Concatenate Deployments <methods/concatenation>
Grid in Time (Step 1) <methods/time_gridding>
Grid Vertically (Step 2) <methods/vertical_gridding>
Combine Deployments (Step 3) <methods/concatenation>

.. toctree::
:maxdepth: 1
Expand Down
11 changes: 4 additions & 7 deletions docs/source/methods/time_gridding.rst
Original file line number Diff line number Diff line change
Expand Up @@ -247,10 +247,7 @@ The time gridding processor includes several quality control features:
- **Comprehensive logging**: Detailed processing logs for debugging and validation

7. Time-Domain Filtering Details
=================================

7.1. Filtering Applications
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
----------------------------------------------

Time-domain filtering is particularly useful for:

Expand All @@ -264,7 +261,7 @@ But it is not necessarily appropriate for:
- **High-frequency process studies**: Where tidal and inertial signals are of interest
- **Short-term deployments**: Where filtering may remove significant portions of the record

7.2. RAPID Array Context: De-tiding for Long-term Records
7.1. RAPID Array Context: De-tiding for Long-term Records
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The filtering implementation is based on the RAPID array processing workflow, where **2-day low-pass Butterworth filtering** (6th order) was applied to remove tidal and inertial variability from year-long mooring records.
Expand Down Expand Up @@ -296,8 +293,8 @@ The Python implementation in :mod:`oceanarray.time_gridding` provides equivalent
- **Modern formats**: NetCDF output with CF conventions
- **Gap-aware processing**: Intelligent handling of data gaps

7.3. Filter Implementation Details
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
7.2. Filter Implementation Details (not yet implemented)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

**Gap Handling**

Expand Down
111 changes: 111 additions & 0 deletions docs/source/methods/trimming_old.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
2. Trimming to Deployed period
==============================

This document outlines the **trimming** step in the oceanarray processing workflow. Trimming refers to the process of isolating the valid deployment period from a time series—typically corresponding to the interval between instrument deployment and recovery. It also produces summary plots.

This step may need to be run several times, adjusting the start and end of the deployment periods.

It corresponds to **Stage 2** from RAPID data processing and management, which uses as input the RDB format and outputs (in RDB format) the `*.use`` file.

1. Overview
-----------

Raw mooring records often contain extraneous data before deployment or after recovery (e.g., deck recording, values during ascent/descent, post-recovery handling). These segments must be trimmed to retain only the time interval when the instrument was collecting valid in-situ measurements at the nominal depth during deployment. In this stage:

- Visualised to identify data issues (e.g., deployment start/end spikes only)
- Optionally low-pass filtered (e.g., 2-day Butterworth)
- Inspected manually
- Optionally adjusted:
- Revised trimming bounds
- Prepared for further processing (e.g., gridding)

2. Purpose
----------

- Remove non-oceanographic data before deployment and after recovery
- Ensure temporal consistency across instruments on the same mooring
- Define the usable deployment interval for each instrument
- Generate summary plots for review and archiving


3. Input
--------

- Standardised `xarray.Dataset` containing raw time series (`TIME`, `TEMPERATURE`, etc.)
- Metadata indicating deployment and recovery times, either:
- Attached to the dataset (e.g., `ds.attrs["start_time"]`)
- Auxiliary logs (optional): info.dat with mooring latitude and longitude, and deployment times (anchor away) and recovery times (all gear on deck)

4. Output
---------

- Trimmed `xarray.Dataset` with:
- `TIME` restricted to deployment period
- Attributes updated with `trimmed = True` and `start_time`, `end_time`
- Summary plots (e.g., `.png` or `.pdf` of raw and filtered time series)
- Updated deployment metadata (e.g., revised start/end time if needed) in `info.dat` or similar human-readable format


Common plots include:

- Full record: `TEMPERATURE`, `CONDUCTIVITY`, `PRESSURE`
- 2-day Butterworth filtered versions


5. Example
----------

.. code-block:: python

from oceanarray.methods.trimming import trim_to_deployment

ds_trimmed = trim_to_deployment(ds_std, start="2021-01-05T20:00", end="2023-02-25T17:00")

.. code-block:: text

<xarray.Dataset>
Dimensions: (TIME: 104576)
Coordinates:
* TIME (TIME) datetime64[ns] ...
Data variables:
TEMPERATURE (TIME) float32 ...
PRESSURE (TIME) float32 ...
Attributes:
start_time: 2021-01-05T20:00
end_time: 2023-02-25T17:00
trimmed: True

6. Implementation Notes
-----------------------

- Deployment periods should be specified in UTC or provided in the mooring `*info.dat` file as in the RAPID data processing and management framework.
- Start and end times may come from cruise logs or recovery reports
- Optionally: implement automatic detection of stable deployment periods (e.g., threshold in pressure)

7. FAIR Considerations
----------------------

- Trimmed intervals should be **documented transparently** in dataset attributes
- Retain untrimmed data for provenance if needed
- Enhances **reusability** and **accuracy** in further processing

See also: :doc:`standardisation`

Legacy Script (RAPID)
---------------------

The RAPID script shows how the Seabird format data were changed into the RDB format.



- `microcat_raw2use_003.m <../_static/code/microcat_raw2use_003.m>`__

This script was part of the original RAPID processing chain used to convert MicroCAT `.asc` or `.cnv` output to `.raw` files in RODB format. It includes parsing logic, metadata extraction from `info.dat`, time adjustment, and basic diagnostic plotting.



.. literalinclude:: ../_static/code/microcat_raw2use_003.m
:language: matlab
:lines: 1-40
:linenos:
:caption: Excerpt from `microcat_raw2use_003.m`
4 changes: 2 additions & 2 deletions docs/source/methods/vertical_gridding.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@

Vertical Gridding
=================
Step 2: Vertical Gridding
==========================

For a tall mooring with multiple instruments at different depths, vertical gridding can be used to create an evenly spaced (in pressure or depth coordinates) profile of measured properties.

Expand Down