PeekProof

Your model isn't smart. It's peeking.

PeekProof is a target-leakage auditor for ML models, powered by the DataHub context graph. Before a model ships, it walks column-level lineage to catch features that secretly derive from the label or from post-outcome data, then writes the verdict back into your catalog with the exact lineage path as proof.

License: Apache-2.0

Why

Target leakage is the classic silent ML failure: the model trains on information that will not exist at prediction time, aces validation, and falls apart in production. The evidence of leakage is already sitting in your metadata: lineage. PeekProof turns that lineage into an automated pre-ship guardrail.

How it works

1. Read — pulls the model's features, label, and column-level lineage from DataHub (SDK or MCP server).
2. Detect — deterministic taint analysis over the lineage graph:
   • R2 direct-target: the label (or a renamed copy) is used as a feature
   • R1 label-ancestry: a feature is computed from the label
   • R3 post-outcome: a feature derives from data created after the prediction moment
   • R4 semantic (optional): an LLM adjudicator reviews ambiguous cases
3. Act — fails your CI gate before a leaky model merges.
4. Write back — tags the model, attaches the evidence path as structured properties, and saves a full audit report to DataHub, so the context graph gets smarter with every audit.

Status

Built during the DataHub Agent Hackathon (July–August 2026). Core engine complete; DataHub integration, CI gate, and demo scenario in progress. Developed with AI assistance (Claude), reviewed and tested by a human.

Quickstart

pip install -e ".[datahub]"
peekproof audit --model "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn-predictor,PROD)"

License

Apache-2.0 © 2026 adamjali