diff --git a/.quality_assurance/2026-08-31_04-27-29_EST.md b/.quality_assurance/2026-08-31_04-27-29_EST.md new file mode 100644 index 000000000..db882db88 --- /dev/null +++ b/.quality_assurance/2026-08-31_04-27-29_EST.md @@ -0,0 +1,473 @@ +# Quality Assurance Journal: internal/usage Subsystem + +## Overview + +This journal entry reviews the `internal/usage` module in the codenerd system. The module tracks AI model usage (tokens and cost) across various dimensions like model, provider, shard, operation, and session. It is designed to be highly durable and safe under concurrent load. + +## Gap Analysis: Boundary Value Analysis and Negative Testing + +### Vector 1: Null/Undefined/Empty Strings + +1. Empty model name, empty provider name, empty operation type, empty shard +info. Currently the code groups by these strings as map keys. If a user doesn't +pass these, they map to an empty string key, which might overlap with a generic +bucket but lacks an explicit fallback bucket. If `model` is empty, pricing fails +to find a match and attributes cost 0, which is technically handled but could be +cleaner. + +2. We should test that a nil or canceled context given to `TrackFromContext` or +`shardMetaFromContext` gracefully results in 'unknown' default values and no +crash. + +### Vector 2: Type Coercion / Data Corruption + +1. The tracker correctly rejects negative token counts in `Track()`. However, we +must explicitly test for integer overflow (e.g., passing `math.MaxInt` or +numbers so large they wrap around `int64` bounds in `AddCost`). + +2. The tests cover a malformed `usage.json` (e.g. `{{{corrupt`), but do we test +what happens when `events` or `aggregate` contain incorrect primitive types? +(e.g. a string where an integer token count is expected). Go's json unmarshaler +handles some type mismatches by erroring out, which results in the tracker +starting fresh and overwriting the corrupt file. This behavior is good, but +should be explicitly verified. + +### Vector 3: User Request Extremes + +1. Extremely high volume of unique models, providers, shards, operations. +Currently `maxSessions` bounds the session map, but there is no bound on +`ByModel`, `ByProvider`, etc. If a malformed or malicious script generates a +unique 'model' string for millions of calls, the map in `AggregatedStats` will +grow unbounded, leading to memory exhaustion (OOM). We should add a cap to these +maps or sanitize inputs. + +2. Extremely long string inputs for model names, providers, shard types. +Extremely long strings in map keys could lead to memory ballooning or slow hash +map lookups. + +### Vector 4: State Conflicts / Race Conditions + +1. The file lock and debounce mechanisms are robust, but we should test the race +between `Track()` writing an event, `autoSaveFlush()` saving to disk, and +`Close()` being called simultaneously. While locks protect the state, the +debounce timer behavior under heavy contention and rapid shutdowns must be +thoroughly exercised. + +## Performance Considerations + +The tracker's performance is generally excellent due to debounced file writes +and an in-memory bounded ring buffer for events. However, the lack of bounds on +maps other than `BySession` poses a risk if high-cardinality data is ingested +(such as random UUIDs being mistakenly passed as a model name). The system +easily handles standard use cases, but edge cases involving unbounded map growth +could degrade performance during `Stats()` copies and JSON serialization. + +## Deeper Analysis on Unbounded Memory Growth and Map Saturation + +A critical vulnerability observed in the `usage_tracker.go` architecture relates to memory management during pathological operational scenarios. Specifically, the tracker uses several map structures (`ByProvider`, `ByModel`, `ByShardType`, `ByShardName`, `ByOperation`) within the `AggregatedStats` struct to persist token counts and cost estimates. + +While the `BySession` map is explicitly capped via `maxSessions = 500` and lowest-spend sessions are pruned into a `prunedSessionKey`, no such pruning logic exists for the other maps. This asymmetry in bounds management exposes the subsystem to multiple failure modes. + +### Evaluation of `Provider` Map Bounds Safety + +The `ByProvider` map accumulates keys based on the `provider` string passed to `Track()`. In normal operations, the cardinality of this set is expected to be low (e.g., < 100). However, if an external actor or malfunctioning subagent starts passing dynamically generated strings (like UUIDs, timestamps, or high-entropy hash strings) as `provider` identifiers, the map will grow continuously. + +**Failure Mode (OOM):** The `AggregatedStats` object is held in memory for the lifetime of the `tracker` instance. As the `ByProvider` map size increases into the millions of keys, memory consumption will scale linearly, eventually leading to an Out-Of-Memory (OOM) panic in Go. + +**Failure Mode (Serialization Block):** During the debounced `saveLocked()` flush operation, the entire `UsageData` structure is marshaled to JSON. A saturated `ByProvider` map will significantly inflate the time required for `json.MarshalIndent`. Because this operation must complete to flush data safely, severe map bloat could delay flushing or block other runtime routines. + +**Failure Mode (Stats Copy Block):** When a component requests `tracker.Stats()`, the function deeply copies every map while holding `t.mu.Lock()`. A massive `ByProvider` map will keep the mutex locked for an extended duration, severely blocking concurrent `Track()` calls and starving the LLM execution threads that rely on usage tracking. + +**Mitigation Recommendation:** Implement a generalized top-K caching mechanism or a rigid cardinality bound for all map structures. Analogous to `maxSessions`, if the `ByProvider` map exceeds `maxProviders` (e.g., 500), the lowest-contributing keys should be folded into an 'other' or 'pruned' category. + +### Evaluation of `Model` Map Bounds Safety + +The `ByModel` map accumulates keys based on the `model` string passed to `Track()`. In normal operations, the cardinality of this set is expected to be low (e.g., < 100). However, if an external actor or malfunctioning subagent starts passing dynamically generated strings (like UUIDs, timestamps, or high-entropy hash strings) as `model` identifiers, the map will grow continuously. + +**Failure Mode (OOM):** The `AggregatedStats` object is held in memory for the lifetime of the `tracker` instance. As the `ByModel` map size increases into the millions of keys, memory consumption will scale linearly, eventually leading to an Out-Of-Memory (OOM) panic in Go. + +**Failure Mode (Serialization Block):** During the debounced `saveLocked()` flush operation, the entire `UsageData` structure is marshaled to JSON. A saturated `ByModel` map will significantly inflate the time required for `json.MarshalIndent`. Because this operation must complete to flush data safely, severe map bloat could delay flushing or block other runtime routines. + +**Failure Mode (Stats Copy Block):** When a component requests `tracker.Stats()`, the function deeply copies every map while holding `t.mu.Lock()`. A massive `ByModel` map will keep the mutex locked for an extended duration, severely blocking concurrent `Track()` calls and starving the LLM execution threads that rely on usage tracking. + +**Mitigation Recommendation:** Implement a generalized top-K caching mechanism or a rigid cardinality bound for all map structures. Analogous to `maxSessions`, if the `ByModel` map exceeds `maxModels` (e.g., 500), the lowest-contributing keys should be folded into an 'other' or 'pruned' category. + +### Evaluation of `ShardType` Map Bounds Safety + +The `ByShardType` map accumulates keys based on the `shardtype` string passed to `Track()`. In normal operations, the cardinality of this set is expected to be low (e.g., < 100). However, if an external actor or malfunctioning subagent starts passing dynamically generated strings (like UUIDs, timestamps, or high-entropy hash strings) as `shardtype` identifiers, the map will grow continuously. + +**Failure Mode (OOM):** The `AggregatedStats` object is held in memory for the lifetime of the `tracker` instance. As the `ByShardType` map size increases into the millions of keys, memory consumption will scale linearly, eventually leading to an Out-Of-Memory (OOM) panic in Go. + +**Failure Mode (Serialization Block):** During the debounced `saveLocked()` flush operation, the entire `UsageData` structure is marshaled to JSON. A saturated `ByShardType` map will significantly inflate the time required for `json.MarshalIndent`. Because this operation must complete to flush data safely, severe map bloat could delay flushing or block other runtime routines. + +**Failure Mode (Stats Copy Block):** When a component requests `tracker.Stats()`, the function deeply copies every map while holding `t.mu.Lock()`. A massive `ByShardType` map will keep the mutex locked for an extended duration, severely blocking concurrent `Track()` calls and starving the LLM execution threads that rely on usage tracking. + +**Mitigation Recommendation:** Implement a generalized top-K caching mechanism or a rigid cardinality bound for all map structures. Analogous to `maxSessions`, if the `ByShardType` map exceeds `maxShardTypes` (e.g., 500), the lowest-contributing keys should be folded into an 'other' or 'pruned' category. + +### Evaluation of `ShardName` Map Bounds Safety + +The `ByShardName` map accumulates keys based on the `shardname` string passed to `Track()`. In normal operations, the cardinality of this set is expected to be low (e.g., < 100). However, if an external actor or malfunctioning subagent starts passing dynamically generated strings (like UUIDs, timestamps, or high-entropy hash strings) as `shardname` identifiers, the map will grow continuously. + +**Failure Mode (OOM):** The `AggregatedStats` object is held in memory for the lifetime of the `tracker` instance. As the `ByShardName` map size increases into the millions of keys, memory consumption will scale linearly, eventually leading to an Out-Of-Memory (OOM) panic in Go. + +**Failure Mode (Serialization Block):** During the debounced `saveLocked()` flush operation, the entire `UsageData` structure is marshaled to JSON. A saturated `ByShardName` map will significantly inflate the time required for `json.MarshalIndent`. Because this operation must complete to flush data safely, severe map bloat could delay flushing or block other runtime routines. + +**Failure Mode (Stats Copy Block):** When a component requests `tracker.Stats()`, the function deeply copies every map while holding `t.mu.Lock()`. A massive `ByShardName` map will keep the mutex locked for an extended duration, severely blocking concurrent `Track()` calls and starving the LLM execution threads that rely on usage tracking. + +**Mitigation Recommendation:** Implement a generalized top-K caching mechanism or a rigid cardinality bound for all map structures. Analogous to `maxSessions`, if the `ByShardName` map exceeds `maxShardNames` (e.g., 500), the lowest-contributing keys should be folded into an 'other' or 'pruned' category. + +### Evaluation of `Operation` Map Bounds Safety + +The `ByOperation` map accumulates keys based on the `operation` string passed to `Track()`. In normal operations, the cardinality of this set is expected to be low (e.g., < 100). However, if an external actor or malfunctioning subagent starts passing dynamically generated strings (like UUIDs, timestamps, or high-entropy hash strings) as `operation` identifiers, the map will grow continuously. + +**Failure Mode (OOM):** The `AggregatedStats` object is held in memory for the lifetime of the `tracker` instance. As the `ByOperation` map size increases into the millions of keys, memory consumption will scale linearly, eventually leading to an Out-Of-Memory (OOM) panic in Go. + +**Failure Mode (Serialization Block):** During the debounced `saveLocked()` flush operation, the entire `UsageData` structure is marshaled to JSON. A saturated `ByOperation` map will significantly inflate the time required for `json.MarshalIndent`. Because this operation must complete to flush data safely, severe map bloat could delay flushing or block other runtime routines. + +**Failure Mode (Stats Copy Block):** When a component requests `tracker.Stats()`, the function deeply copies every map while holding `t.mu.Lock()`. A massive `ByOperation` map will keep the mutex locked for an extended duration, severely blocking concurrent `Track()` calls and starving the LLM execution threads that rely on usage tracking. + +**Mitigation Recommendation:** Implement a generalized top-K caching mechanism or a rigid cardinality bound for all map structures. Analogous to `maxSessions`, if the `ByOperation` map exceeds `maxOperations` (e.g., 500), the lowest-contributing keys should be folded into an 'other' or 'pruned' category. + +## File Locking and Cross-Process Race Conditions + +The module correctly implements a robust file-locking mechanism to support multiple processes modifying `usage.json` simultaneously. However, the interactions between the lock, the debounce timer, and OS-level file buffering warrant specific negative test scenarios. + +### Scenario: Immediate Flush Contention + +When `Flush()` is called manually, it stops the `autoSaveTimer` and immediately writes to disk. If multiple processes trigger `Flush()` concurrently while rapidly mutating state, the lock acquisition wait times can spike. A negative test should simulate high-frequency `Flush()` calls across three separate goroutines to measure lock latency and verify that the `autoSaveFlush()` callback does not execute if `Flush()` has already handled the state. + +### Scenario: Atomic Rename Integrity + +The persistence layer uses a temporary file (e.g., `.usage-*.json`) and relies on an atomic `os.Rename()` to swap it into place. A critical test vector involves simulating an OS failure or process crash exactly after `os.WriteFile()` completes but before `os.Rename()` is called. The test should verify that the system leaves the temporary file behind without corrupting the original `usage.json`, and that the next load operation successfully parses the original file. + +## Type Strictness and JSON Marshaling Integrity + +The `TokenCounts` struct relies on `int64` for input and output, and `float64` for cost. + +### Negative Test Vector: Floating Point NaN and Infinity + +If the `EstimateCost` function returns `math.NaN()` or `math.Inf()`, the subsequent `json.Marshal` call in `saveLocked()` will fail, as Go's JSON encoder does not support NaN or Infinity values for floats. This will cause the entire tracker state to become un-saveable, resulting in data loss. The tests must inject extreme token values to verify that costs remain finite and safely serializable. + +### Negative Test Vector: Float Precision Loss + +Repeatedly accumulating small `float64` cost values could lead to precision loss over an extensive session. The test suite should loop 1,000,000 rapid updates adding small costs (e.g., $0.0000001) and assert that the final accumulated cost in the tracker matches the expected mathematical sum within an acceptable epsilon. This validates that the `AddCost` operations do not suffer from severe floating-point arithmetic degradation. + +## Mangle Language Integration and Type Assumptions + +Given the codenerd context, usage statistics are likely consumed by Mangle predicates to govern resource limits. Mangle logic is strictly typed (e.g., atoms vs strings). + +### Negative Test Vector: Atom and String Type Confusion in Mangle + +When usage metrics are projected into a Mangle `factstore`, the string keys (`model`, `provider`) are often translated to Mangle strings, but logic predicates might expect Mangle Atoms (e.g., `/model/gpt-4o`). If an unexpected string containing slashes or invalid atom characters is tracked, the projection might fail silently or produce facts that cannot be joined. We must write a Mangle-level integration test that asserts that the exported usage facts are strictly typed as expected by the logic engine. + +## Thorough Review of Context and Usage Initialization + +The `WithShardContext` function places crucial strings onto the Context using strongly-typed keys. However, `shardMetaFromContext` handles fallbacks to older raw-string keys for legacy compatibility. + +### Context Injection Edge Case 1 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 2 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 3 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 4 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 5 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 6 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 7 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 8 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 9 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 10 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 11 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 12 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 13 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 14 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 15 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 16 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 17 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 18 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 19 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 20 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 21 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 22 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 23 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 24 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Context Injection Edge Case 25 + +If a deeply nested goroutine passes a detached context (like `context.WithoutCancel`), the context's values are usually preserved. But if the context is wrapped in a custom implementation that fails to delegate `Value()` correctly for the `shardMetaKey`, the tracking data defaults to 'unknown'. The test suite needs to simulate an arbitrary context wrapping (e.g., a custom logger context) that overrides the value to an unexpected type like an `int` instead of a `string`. The tracker should gracefully recover from this type assertion failure and fall back to the default 'unknown' state without panicking. + +### Initialization Timing Edge Case 1 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 2 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 3 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 4 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 5 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 6 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 7 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 8 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 9 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 10 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 11 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 12 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 13 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 14 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 15 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 16 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 17 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 18 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 19 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 20 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 21 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 22 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 23 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 24 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +### Initialization Timing Edge Case 25 + +The `NewTracker()` initialization reads the existing `usage.json`. If a concurrent system tool is rotating logs or zipping `.nerd` files, `usage.json` might be temporarily locked or renamed out of place. The tests should simulate `os.ErrPermission` or temporary read failures during `Load()`. Instead of defaulting to zero usage, the system must either block and retry or cleanly report an error. Starting from zero due to a temporary lock would overwrite the true usage history on the next flush. + +## Advanced Boundary Analysis on Pricing Resolution + +The pricing module in `pricing.go` implements prefix-based matching for model names. This is an efficient heuristic but susceptible to boundary condition failures. + +### Pricing Boundary Case 1 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 2 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 3 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 4 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 5 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 6 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 7 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 8 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 9 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 10 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 11 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 12 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 13 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 14 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 15 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 16 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 17 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 18 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 19 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 20 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 21 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 22 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 23 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 24 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +### Pricing Boundary Case 25 + +Consider the scenario where a model is introduced with a name that is an exact substring of an entirely different model family (e.g., 'gpt-4' vs 'gpt-4-turbo'). The current `priceTable` map relies on explicit keys and longest-prefix matching. However, if a user specifies 'gpt-4-unknown-variant', it will fall back to 'gpt-4' pricing. This might significantly underestimate costs if the unknown variant is inherently more expensive. The testing suite should assert that specifically engineered model strings (like trailing hyphens 'gpt-4-', casing differences 'Gpt-4', or embedded non-printable characters) do not falsely resolve to unintended pricing tiers. + +## Final Conclusion on Usage Subsystem Resilience + +Through this rigorous application of Boundary Value Analysis and Negative Testing, it is evident that while the core mechanisms of debouncing and file locking are robust, the unbounded nature of internal metric maps (Provider, Model, Operation) represents a significant threat to long-term memory stability. Implementing explicit truncation limits and strict type-checking on Mangle projections will vastly improve the architectural durability of the codenerd platform. diff --git a/internal/usage/usage_tracker_test.go b/internal/usage/usage_tracker_test.go index a25fbced0..8cd23d7d5 100644 --- a/internal/usage/usage_tracker_test.go +++ b/internal/usage/usage_tracker_test.go @@ -9,6 +9,12 @@ import ( ) func TestTracker_TrackAggregatesAndPersists(t *testing.T) { + // TODO: What happens if model or provider is an empty string? + // TODO: What happens if inputs cause integer overflow? + // TODO: What happens if we provide extremely long string inputs for model or provider? + // TODO: What happens if we have unbounded map growth for models or providers (unlike sessions)? + // TODO: What happens if context cancellation occurs precisely during saveLocked()? + // TODO: What happens if context keys are corrupted into non-string types causing panic? ws := t.TempDir() tracker, err := NewTracker(ws) if err != nil {