Skip to content

FPM-942: Flag decoding#71

Merged
richjam merged 3 commits intomainfrom
feature/FPM-942-flag-decoding
Mar 27, 2026
Merged

FPM-942: Flag decoding#71
richjam merged 3 commits intomainfrom
feature/FPM-942-flag-decoding

Conversation

@richjam
Copy link
Copy Markdown
Collaborator

@richjam richjam commented Mar 27, 2026

Background

Previously, flag columns store bitwise integers, but there is no way to recover human-readable flag names. This PR adds a decoding (and encoding) ability.

Main changes

FlagColumn (flag_manager.py)

  • Added is_decoded: bool = False to keep the state of where the column is decoded or not.
  • Added decode(df) - converts bit integer column to List(String) of flag names in ascending bit-value order. Flag value of 0 produces []
  • Added encode(df) - converts List(String) back to the bit-value.
  • add_flag and remove_flag take into account the is_decoded state. If the column is decoded, they do a round rtip of encode -> add/remove flag -> decode

TimeFrame (base.py)

  • decode_flag_column - returns new TimeFrame with the integer flag column replaced by List(String)
  • encode_flag_column - returns new TimeFrame with the List(String) column replaced by it's equivalent bit integer values

Notes

  • Documentation to be addressed in future PR

Usage

# Set up a TimeFrame with some pre-existing flag values
tf = TimeFrame(
    pl.DataFrame({
        "time": [datetime(2025, 1, 1), datetime(2025, 1, 2), datetime(2025, 1, 3), datetime(2025, 1, 4)],
        "value": [10.0, 20.0, 30.0, 40.0],
        "qc_flags": [0, 1, 5, 7],
    }),
    "time",
).with_flag_system("qc", {"SPIKE": 1, "OUT_OF_RANGE": 2, "LOW_BATTERY": 4})

tf.register_flag_column("qc_flags", "qc")

> tf.df["qc_flags"]
> [0, 1, 5, 7]

tf_decoded = tf.decode_flag_column("qc_flags")

> tf_decoded.df["qc_flags"]
> [
>	[],
>	["SPIKE"],
>	["SPIKE", "LOW_BATTERY"],
>	["SPIKE", "OUT_OF_RANGE", "LOW_BATTERY"],
> ]

# Add flag works on a decoded column
tf_decoded.add_flag("qc_flags", "OUT_OF_RANGE", pl.col("value") < 11)

> tf_decoded.df["qc_flags"]
> [
>	["OUT_OF_RANGE"],
>	["SPIKE"],
>	["SPIKE", "LOW_BATTERY"],
>	["SPIKE", "OUT_OF_RANGE", "LOW_BATTERY"],
> ]

# Re-encode
tf_encoded = tf_decoded.encode_flag_column("qc_flags")

> tf_encoded .df["qc_flags"]
> [2, 1, 5, 7]

@richjam richjam merged commit b463ada into main Mar 27, 2026
3 checks passed
@richjam richjam deleted the feature/FPM-942-flag-decoding branch March 27, 2026 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants