What to build
Add a new CLI command that scans the active knowledge graph for semantically similar claims using the existing findSimilarClaims() implementation and reports potential duplicate groups.
This command is intended as an audit tool only. It does not modify graph state or merge claims.
Why
The proposal-time duplicate check (findSimilarClaims() integrated into reviewProposal() / applyProposal()) prevents new duplicate claims from being introduced during proposal application. However, claims written before that feature was added remain in existing graphs.
A standalone audit command allows users to identify historical duplicate claims and decide how they should be cleaned up.
Proposed approach
Add a standalone auditDuplicateClaims() method to KnowledgeGraphService that:
- Loads the active graph view (excluding superseded claims)
- Uses the existing
findSimilarClaims() helper to perform a pairwise similarity scan across active claims
- Groups claims whose similarity is greater than or equal to the configured
dedupe.similarityThreshold
- Returns the matching groups without modifying graph state
- Export
DuplicateAuditResult and DuplicateAuditGroup types from the service module
- Add
auditDuplicateClaims() to the GraphMemoryProvider interface
Register a new CLI command following the same pattern as the existing graph audit commands.
The CLI should report each detected duplicate group with enough information to identify the claims, including:
- Claim ID
- Truncated claim text
- Similarity score for each matching claim
What's already in place
findSimilarClaims() in libs/knowledge-graph/dedupe/find-similar-claims.ts provides the similarity comparison logic
dedupe.similarityThreshold already exists in GraphContextConfig
- Claim embeddings are already available to the knowledge graph service
- Existing graph audit commands provide a precedent for the CLI implementation
Acceptance criteria
Notes
A prototype implementation was benchmarked on a graph containing 600 active claims (~179,700 unique pairwise comparisons). Across ten runs, the audit completed in an average of ~139 ms (approximately 160 ms end-to-end), indicating that a full graph audit is inexpensive at this scale while reusing the existing findSimilarClaims() implementation.
What to build
Add a new CLI command that scans the active knowledge graph for semantically similar claims using the existing
findSimilarClaims()implementation and reports potential duplicate groups.This command is intended as an audit tool only. It does not modify graph state or merge claims.
Why
The proposal-time duplicate check (
findSimilarClaims()integrated intoreviewProposal()/applyProposal()) prevents new duplicate claims from being introduced during proposal application. However, claims written before that feature was added remain in existing graphs.A standalone audit command allows users to identify historical duplicate claims and decide how they should be cleaned up.
Proposed approach
Add a standalone
auditDuplicateClaims()method toKnowledgeGraphServicethat:findSimilarClaims()helper to perform a pairwise similarity scan across active claimsdedupe.similarityThresholdDuplicateAuditResultandDuplicateAuditGrouptypes from the service moduleauditDuplicateClaims()to theGraphMemoryProviderinterfaceRegister a new CLI command following the same pattern as the existing graph audit commands.
The CLI should report each detected duplicate group with enough information to identify the claims, including:
What's already in place
findSimilarClaims()inlibs/knowledge-graph/dedupe/find-similar-claims.tsprovides the similarity comparison logicdedupe.similarityThresholdalready exists inGraphContextConfigAcceptance criteria
auditDuplicateClaims()toKnowledgeGraphServiceDuplicateAuditResultandDuplicateAuditGroupauditDuplicateClaims()to theGraphMemoryProviderinterfacededupe.similarityThresholdconfiguration (no additional CLI flags or threshold configuration)Notes
A prototype implementation was benchmarked on a graph containing 600 active claims (~179,700 unique pairwise comparisons). Across ten runs, the audit completed in an average of ~139 ms (approximately 160 ms end-to-end), indicating that a full graph audit is inexpensive at this scale while reusing the existing
findSimilarClaims()implementation.