Skip to content

fix(store): preserve first_seen ordering when merging background chunks #114

Description

@dborup

Upstream source

Evaluate and adapt Kpa-clawbot/CoreScope#2050: merge background-loaded chunks without violating the packet store's oldest-first ordering.

Current fork evidence

Verified against fork master 1bdf088e:

  • PacketStore.packets must be sorted by first_seen ASC.
  • evictStaleInternal walks from the head and stops at the first in-window transmission.
  • The background merge still publishes with s.packets = append(localPackets, s.packets...).
  • Chunks selected by recent last_seen may contain much older first_seen, so prepending silently breaks ordering and can under-evict.

Fork-specific adaptation

Preserve the fork's index-before-slice-publication invariant, parked route masks, and resolved-path/node indexes. Replace only the final publication with a bounded linear merge of two sorted runs; do not sort the complete store under s.mu.

Acceptance criteria

  • s.packets remains ordered after every chunk merge, including interleaved/equal timestamps and repeated last_seen activity.
  • No packet is lost, duplicated, or visible without its indexes.
  • Time- and memory-based eviction remove all eligible oldest entries.
  • Complexity is O(existing + chunk), with at most a bounded incoming-chunk sort.
  • A benchmark guards against a full-store comparison sort under the write lock.

Verification

Add direct merge and eviction regressions, run the full server suite under -race, benchmark a production-shaped merge, and run static checks.

No deploy or live-environment mutation is part of this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions