Skip to content

Add a CLI command to audit duplicate claims #169

Description

@TanishValesha

What to build

Add a new CLI command that scans the active knowledge graph for semantically similar claims using the existing findSimilarClaims() implementation and reports potential duplicate groups.

This command is intended as an audit tool only. It does not modify graph state or merge claims.


Why

The proposal-time duplicate check (findSimilarClaims() integrated into reviewProposal() / applyProposal()) prevents new duplicate claims from being introduced during proposal application. However, claims written before that feature was added remain in existing graphs.

A standalone audit command allows users to identify historical duplicate claims and decide how they should be cleaned up.


Proposed approach

Add a standalone auditDuplicateClaims() method to KnowledgeGraphService that:

  • Loads the active graph view (excluding superseded claims)
  • Uses the existing findSimilarClaims() helper to perform a pairwise similarity scan across active claims
  • Groups claims whose similarity is greater than or equal to the configured dedupe.similarityThreshold
  • Returns the matching groups without modifying graph state
  • Export DuplicateAuditResult and DuplicateAuditGroup types from the service module
  • Add auditDuplicateClaims() to the GraphMemoryProvider interface

Register a new CLI command following the same pattern as the existing graph audit commands.

The CLI should report each detected duplicate group with enough information to identify the claims, including:

  • Claim ID
  • Truncated claim text
  • Similarity score for each matching claim

What's already in place

  • findSimilarClaims() in libs/knowledge-graph/dedupe/find-similar-claims.ts provides the similarity comparison logic
  • dedupe.similarityThreshold already exists in GraphContextConfig
  • Claim embeddings are already available to the knowledge graph service
  • Existing graph audit commands provide a precedent for the CLI implementation

Acceptance criteria

  • Add auditDuplicateClaims() to KnowledgeGraphService
  • Export DuplicateAuditResult and DuplicateAuditGroup
  • Add auditDuplicateClaims() to the GraphMemoryProvider interface
  • Register a new CLI command
  • CLI reports duplicate groups with claim identifiers, truncated claim text, and similarity scores
  • Superseded claims are excluded from the audit
  • Uses the existing dedupe.similarityThreshold configuration (no additional CLI flags or threshold configuration)

Notes

A prototype implementation was benchmarked on a graph containing 600 active claims (~179,700 unique pairwise comparisons). Across ten runs, the audit completed in an average of ~139 ms (approximately 160 ms end-to-end), indicating that a full graph audit is inexpensive at this scale while reusing the existing findSimilarClaims() implementation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions