Tiered storage: archive-before-prune + frozen backups

[menonpg]

Summary

Implement tiered storage architecture so nothing is ever lost while keeping RLM bounded.

The Problem

Currently, when old daily notes are pruned from hot storage, the information is lost. RLM only reads local files, so archived memories become inaccessible unless explicitly retrieved via RAG.

Proposed Solution

Three-Tier Architecture

• Hot (files): Last 30-90 days, RLM reads directly
• Cold (vector DB): All embeddings, forever — RAG retrieves here
• Frozen (S3/GCS): Raw file backups for disaster recovery

Archive-Before-Prune Flow

When pruning old files:

1. Embed all chunks → push to vector DB (cold)
2. Upload raw markdown → push to object storage (frozen)
3. Delete local file

Configuration

agent = HybridAgent(
    archive_backend="s3",  # or "gcs", "local"
    archive_bucket="my-agent-memories",
    prune_after_days=90,
    auto_archive=True
)

Benefits

• RLM stays fast (bounded hot files)
• Nothing ever truly deleted (cold vectors persist)
• Time-travel to any memory via RAG
• Disaster recovery from frozen backups

References

• Memory Architecture Patterns cheatsheet (soul-book)
• Blog open-source page roadmap