SLT.BETA.014 – Configurable run log retention strategy #76
Description
[SLT.BETA.014] Configurable run log retention strategy
Overview
Provide operators with controls for handling large git shiplog run logs, including thresholds, truncation, compression, and alternate storage refs to prevent oversized notes.
References & Assets
- Figma / Design: N/A
- Product Spec: N/A
- Related Issues / PRs: run logging feedback thread
- Feature Flags / Experiments: N/A
- Other Assets: N/A
User Story
As a deployment operator, I want to configure how Shiplog stores large run outputs so that git notes stay manageable without losing critical logs.
Acceptance Criteria
- CLI measures log size before attaching and warns when threshold exceeded
- Config option controls handling: truncate, compress, or store in alternate ref
- Default threshold keeps common logs (<1 MiB) inline and routes large logs appropriately
- Documentation updated with configuration examples and recommendations
- Tests cover each retention mode
Definition of Done
Run command supports configurable log retention modes with documentation and automated tests.
Scope
In-Scope
- Size detection and threshold configuration
- Multiple handling strategies (truncate/compress/ref)
- Documentation and tests
Out-of-Scope
- UI for browsing archived logs
- Retrofitting historical entries
Deliverables
- Est. Lines of Code: ~240
- Est. Blast Radius:
lib/commands.sh, logging helpers, docs
Implementation Details
High-Level Approach
Introduce size check prior to note attachment, add configuration via git config/environment, implement strategies, and integrate with run workflow.
Affected Areas
- lib/commands.sh
- scripts/run_log.sh
- docs/runbooks
- tests/run_log_retention.bats
Implementation Steps
- Define configuration keys and defaults
- Implement size measurement and warning logic
- Add truncation, compression, and alt-ref storage handlers
- Update docs with usage guidance
- Add tests for each mode and threshold crossing
Test Plan
Happy Path
- Run with normal log below threshold attaches inline
- Run with large log uses configured strategy and surfaces summary
Edge Cases
- Compression and alt-ref modes validated across hosts
- Threshold set to zero behaves as "always alternate"
Failure Cases
- Invalid config value surfaces helpful error
Monitoring & Success Metrics
- CLI summary indicates chosen retention strategy
QA Sign-off Matrix
| Environment | Surface | Owner | Status | Notes |
|---|---|---|---|---|
| Local Docker | CLI | TBD | Pending | Covered via make test |
Requirements
Hard Requirements
- Prevent git note bloat by handling large logs safely
Soft Requirements
- Provide operators with defaults and guidance
Runtime Requirements
- Works offline with standard POSIX tools
Dependencies & Approvals
- Docs review
Production Notes
Priority: 3 / 5
Important for Beta operations (P2).
Complexity: 4 / 5
Multiple code paths and configuration handling.
Estimate: 20 - 28 hours
Includes implementation, docs, tests.
Risk & Rollback
- Primary Risks: Losing critical logs due to misconfiguration
- Mitigations: Warnings and safe defaults
- Rollback / Kill Switch: Config flag to disable alternate strategies
Additional Notes
Consider storing oversized logs under refs/_shiplog/artifacts/<sha> as default alternate mode.