Skip to content

perf(store): evaluate compact, concurrency-safe observation timestamp cache #490

Closed

Description

@dborup

Assessment baseline: fork master b6852ba0876b02426b6200c1c79c7e0ebe3b9389, checked on 16 October 2026. Recheck master and competing work before implementation.

Problem and priority

Medium priority: investigate replacing each observation's sync.Once, cached time.Time and success flag with a compact, concurrency-safe parsed-time cache.

Upstream uses an atomic 64-bit value with explicit sentinel states. Its reported structure-size and heap savings concern its own layout; our StoreObs also contains fork-specific per-observation routing evidence, so the benefit must be measured here.

Fork adaptation

  • Preserve the complete ParsedTime contract, including parse order, returned location, precision, zero value and success/failure distinction.
  • Define unambiguous encodings for uninitialized, invalid and fallback states. Valid epoch, negative timestamps or sentinel-colliding values must never become invalid or incorrect cached results.
  • Preserve offset locations and dates outside the exact Unix-nanosecond range, using a correct fallback when necessary.
  • Preserve wireRoute and all fork-specific fields. Update structure-size guards and memory accounting from the actual resulting layout, not upstream's stated 160-byte target.
  • Coordinate with observer string sharing and other store-layout/accounting work before implementation. No schema, config, workflow or deployment changes.

Acceptance criteria

  • Differential tests compare first and cached calls with the existing parser for RFC3339/RFC3339Nano, SQL timestamps, offsets, empty/invalid input, epoch, negative/zero times, range boundaries and sentinel collisions.
  • Concurrent first reads and repeated reads pass race tests. Existing retransmission, observer analytics and route-evidence tests remain green.
  • Measure unsafe.Sizeof, effective heap size class, per-observation/accounted bytes, first-parse and cache-hit costs on our supported architecture(s).
  • Demonstrate real retained-heap savings on representative fork data and report trade-offs for fallback timestamps. Mutation tests must catch incorrect sentinel handling and lost offset/route semantics.

Source provenance

Repository: Kpa-clawbot/CoreScope
PR: 2196
Title: perf(store): cache an observation's parsed time in 8 bytes
Head: 682c40a1ed84d600e94fa433d97ffb22be7261de
Merge: 8d2ae405e8f2ba4b9bdb09866f563053375a908b
Changed files:
cmd/server/obs_parsed_time_test.go
cmd/server/retransmission_pressure_test.go
cmd/server/store.go

Activity

  1. added a commit that references this issue on Oct 24, 2026
  2. dborup commented on Oct 24, 2026

    @dborup
    OwnerAuthor

    Fixed by #555 (merged as 6aa03db367cde71f88230c7c4b65836545ff4c0a).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions