Skip to content

config reload swaps the config and reconciles nothing: sources are not added, extractors and ingest ignore it, and specs are never resolved #134

Open

Description

@catinspace-au

Found while fixing #119. A reload today is scalo's reloader loading the file, validating it, and writing it into SharedConfig with a version bump (crates/fetcher/src/main.rs:334-373 and :705-717). Intervals, jitter, filters and routes follow it because the tasks re-read them per tick. Nothing else does. #119 lands the REMOVE half for fetch sources; this is the rest of the same gap.

  • A source ADDED by a reload never starts. builtin_instances() has one production caller, at startup (main.rs:516). Today a second source reaches a running pod by the pod roll the chart's config checksum triggers, which is why b3b49f0 could prove the idle case with tests alone. The reconciler from a source removed by a config reload keeps ticking its provider until the process restarts #119 warns by name when it sees a new id; making it spawn needs resolve_config_specs (crates/fetcher/src/config/resolve.rs) run on the reloaded config first, and the reloader callback is sync.
  • The reload path never resolves env: or vault: specs. resolve_config_specs runs once at startup (main.rs:262), so every reload writes a config with those fields still literal into SharedConfig. No live effect today because drivers bake identity at construction. It is the landmine under the ADD half and under anything else that starts reading identity from the shared config.
  • A source MUTATED under an unchanged id keeps its baked auth, region and endpoints. Only filter, routes, topic suffix and interval follow the config (crates/fetcher/src/driver.rs:310-327, crates/fetcher/src/scheduler/mod.rs:119-122). Change sources.aws.region on a live pod and it keeps polling the old region, without a warning.
  • Container extractors bake their config at spawn and exit only on the global token (crates/fetcher/src/extractor/container/mod.rs:343-352, spawned at main.rs:636-653), so an extractor removed by a reload keeps running its image on its interval.
  • The ingest listener reads enabled once (crates/fetcher/src/ingest/mod.rs:193-196) from a config cloned at main.rs:617, so ingest.enabled: false in a reload leaves the port bound and accepting until restart.
  • PipelineState::reload_config (crates/fetcher/src/pipeline/mod.rs:496-502) is dead outside its own test. It reads as if the fetcher has a reload hook it does not have. Delete it or make it the entry point.

Done when a reload that adds, removes or changes a source, an extractor or the ingest listener has the same effect as a restart on that config, pinned by a test per lifecycle.

Activity

  1. github-actions commented on Sep 24, 2026

    @github-actions
  2. removed
    bugRETIRED: use type Bug
    post-gaRETIRED: use backlog or next-patch
    on Oct 23, 2026
  3. added
    next-patchWanted in an upcoming patch release. Does not hold a cut
    and removed
    backlogCaptured future work, not scheduled now
    on Oct 28, 2026
  4. catinspace-au commented on Nov 12, 2026

    @catinspace-au
    ContributorAuthor

    #286 (16ab848) covers the fetch sources. A reload starts an added source, stops a removed one and rebuilds a changed one. Each source has one generation at a time.

    Changes to the container extractors or ingest listener still need a restart, as docs/internals.md says. The extractors are spawned once in main.rs (extractor/container/mod.rs:343-352). The listener reads its config once (ingest/mod.rs:193-196).

  5. github-actions commented on Nov 12, 2026

    @github-actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    next-patchWanted in an upcoming patch release. Does not hold a cut

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions