Skip to content

codeSignatureForAnchor's comment-drift immunity doesn't cover most already-bundled tree-sitter grammars (yaml, toml, css, html, vue, elixir...) #162

Description

@priyamkarn

Summary

libs/knowledge-graph/code-anchors/resolver.ts currently maps only around 25 file extensions in wasmByExtension.

For any extension not present in that map, codeSignatureForAnchor falls back to:

normalizeLines(sliceLines(...))

This trims whitespace but does not remove comments.

That contradicts the documented guarantee above codeSignatureForAnchor, and the corresponding comment on hasDrifted in audit.ts, that formatting and comment-only edits should not change an anchor fingerprint.

Reproduction

  1. Create a claim anchored to a YAML or TOML file containing a code_anchor comment.
  2. Modify only a comment in the anchored section.

For example:

# note
services:
  app:
    image: example

or:

# note
[package]
name = "example"
  1. Run auditClaimCodeAnchors.

The anchor is reported as drifted even though no semantic code change occurred.

Root Cause

When a Tree-sitter grammar is available, codeSignatureForAnchor can generate a normalized signature without comments.

For unsupported extensions, the fallback path only normalizes whitespace. Comments remain part of the fingerprint, so adding, removing, or editing a comment changes the stored signature.

This is not caused by a missing dependency.

The existing direct dependency, tree-sitter-wasms@0.1.13, already ships prebuilt grammars for:

  • CSS
  • HTML
  • YAML
  • TOML
  • Vue
  • Elixir
  • Solidity
  • Zig
  • OCaml
  • Elm

This was confirmed by running:

npm pack tree-sitter-wasms@0.1.13

and inspecting the package's out/ directory.

None of these grammars are currently connected through wasmByExtension.

As a result, both symbol resolution and drift fingerprinting silently fall back to weaker behavior for these file types, even though parser support is already installed.

Suggested Fix

1. Map the already-bundled grammars

Extend wasmByExtension with the relevant extensions:

  • .yaml and .ymltree-sitter-yaml.wasm
  • .tomltree-sitter-toml.wasm
  • .csstree-sitter-css.wasm
  • .html and .htmtree-sitter-html.wasm
  • .vuetree-sitter-vue.wasm
  • .ex and .exstree-sitter-elixir.wasm
  • .soltree-sitter-solidity.wasm
  • .zigtree-sitter-zig.wasm

The bundled OCaml and Elm grammars could also be mapped to their standard extensions where appropriate.

2. Align C-family header extensions

.hh and .hxx are already included in cFamilyExtensions, which means they participate in the C-family symbol-resolution fallback.

However, they are missing from wasmByExtension.

This means fingerprints for .hh and .hxx files remain comment-sensitive, while related extensions such as .h, .hpp, and .cpp use parser-backed normalization.

These extensions should be aligned while updating the map.

3. Improve the no-grammar fallback

Longer term, the fallback path should consider removing common comment syntax even when no Tree-sitter grammar is available.

Examples of extensions not currently covered by tree-sitter-wasms include:

  • .sql
  • .proto
  • .graphql
  • .hcl
  • .tf

A generic fallback could support common comment forms such as:

// comment
# comment
-- comment
/* block comment */

This would allow unsupported languages to degrade to:

Comment-insensitive fingerprints without symbol resolution

rather than:

Comment-sensitive fingerprints without symbol resolution

The former is much closer to the documented behavior and avoids false-positive drift reports caused only by comment edits.

Expected Behavior

Editing only comments or formatting inside an anchored code range should not mark the anchor as drifted, regardless of whether the file extension was already present in wasmByExtension.

Scope

The immediate fix can remain limited to wiring grammars that are already included in the existing dependency.

No new dependency is required.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions