Skip to content

Prove the no-network guarantee with an automated test #25

Description

@himanshu231204

Problem

"Nothing leaves your machine" is the entire premise of this project. Today it's backed by readable code and a promise in the docs — nothing enforces it. A dependency upgrade could introduce a telemetry call and every test would still pass.

The core claim of the project deserves a test, not just a paragraph.

What to build

A CI job that runs the full suite with outbound network access blocked, and fails if anything tries to reach out.

Acceptance criteria

  • A test or CI job that runs the suite with egress blocked
  • Covers the real paths: store_memory, search_memory, recall_memory, and the CLI equivalents
  • The model download is pre-warmed before the block goes up, so the test proves runtime silence rather than just failing at the download
  • Any connection attempt fails the job with a message naming what tried to connect
  • Runs on every PR, not nightly. A privacy regression should never reach main
  • Documented in docs/guide/privacy.md — "verified by CI" is a much stronger claim than "we promise", and it's the kind of thing that decides adoption

Approaches worth considering

  • Socket-level: monkeypatch socket.socket in a fixture to raise on connect. Simple, in-process, catches anything using Python sockets — which is everything here.
  • Network namespace: run the job with no route out. Stronger, catches subprocesses and native extensions too, but more CI plumbing.
  • A proxy that logs and denies, asserting the log is empty. Gives the best failure message.

The socket patch is probably the right first step — cheap, and catches the realistic regression. A stricter namespace-level job could follow.

Notes

Worth testing the negative case too: a deliberately-connecting test should fail under this harness. A guard that can't fail is worse than no guard, because it's actively misleading.

Pointers

  • docs/guide/privacy.md — the claim being enforced
  • .github/workflows/ci.yml — the integration job already handles model pre-warming with a cache

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededprivacyTouches the no-network, no-API-key guaranteev0.5: trustEncryption, audit log, provenance, no-network proof

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions