Skip to content

docs: correct claims the code does not back - #78

Open
r0h1tb wants to merge 4 commits into
mainfrom
docs/truth-audit
Open

docs: correct claims the code does not back#78
r0h1tb wants to merge 4 commits into
mainfrom
docs/truth-audit

Conversation

@r0h1tb

@r0h1tb r0h1tb commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Description

Audited README, AGENTS.md, CONTRIBUTING.md and docs/ against the current
source. Every change below was checked by running the thing it describes.

Commands and flags that don't exist as documented

  • README's CLI list had 11 of 20 commands. create-database, call-graph,
    symbol-impact, blocks, lambdas, summarize, analyze-stacktrace,
    cache-stats and stats were undocumented.
  • ast-rag update <path>Missing option '--from-commit'. Both
    --from-commit and --to-commit are required.
  • ast-rag sandbox <lang> <cmd>FileNotFoundError: workdir does not exist.
    The second positional is workdir; the command goes in --cmd.
  • QUICKSTART used --from/--to and a --current-branch that doesn't exist.
  • query --verbose and sig --kindNo such option.
  • QUICKSTART used callers to "find what it calls"; that's
    call-graph --direction callees.
  • AGENTS.md described workspace as "Update from git diff"; it shows
    uncommitted changes.

Python API examples that raise

  • get_diff(from_commit=, to_commit=, file_path=) — real signature is
    get_diff(repo_path, from_commit, to_commit=None, limit, offset), no
    per-file filter.
  • full_index(driver, path, commit=) — real is
    full_index(driver, nodes, edges, commit_hash=...) -> None.
  • update_from_git(root=, from_commit=, to_commit=) — real kwargs are
    repo_path, old_commit, new_commit.
  • index_directory, ast_rag.graph_schema, ast_rag.embeddings and
    ast_rag.benchmarks.evaluator do not exist.
  • find_references returns a dict of dicts, not objects with .node.
  • ASTNode.node_type is kind.
  • ProjectConfig.model_validate_json("ast_rag_config.json") takes JSON text,
    not a path.

Config

  • Bare NEO4J_URI / QDRANT_URL / EMBEDDING_URL are read nowhere. The real
    set is the six AST_RAG_* vars in cli.py:132-137; there is no
    embedding-URL override.
  • The example set remote_url with no dimension, which raises
    EmbeddingConfig.dimension must be set when using remote_url. Embeddings are
    local by default and docker-compose.yml starts no embedding server.
  • model_name default is BAAI/bge-m3, not bge-m3.
  • "Tune thresholds in embeddings.py" — no such module. The real knobs are
    vector_weight/keyword_weight.
  • Nothing documented the [mcp] extra, without which ast-rag-mcp and
    ast-rag-watch are on PATH but fail to import.

Shell scripts that never existed

./scripts/index-remaining.sh and index-sequential.sh are referenced across
four docs. git log --all -- 'scripts/*.sh' is empty and scripts/ contains
only README.md.

Graph schema

Both ID formulas were wrong — the node ID includes project_id and the edge ID
includes dep_kind:raw_type_string:confidence. Verified by computing both
hashes. Added the missing Block label and TYPES/DEPENDS_ON/
CROSS_FILE_CALL edges, and noted that HAS_PARAMETER, VIRTUAL_CALL and
LAMBDA_CALL are declared and indexed but never emitted.

Roadmap drift

summarize shipped but was listed as planned; Go was still under "More
languages" in README and CONTRIBUTING after landing in #17.

Related Issue

None — a sweep. #66 was already fixed and closed; only its roadmap half remained.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Checklist

  • My code follows the code style of this project
  • I have added tests — documentation only
  • All new and existing tests passed (unchanged: no source touched)
  • I have updated the documentation accordingly
  • I have run ast-rag evaluate --all — needs live services
  • My changes generate no new warnings

Testing

No source changes. Commands and flags checked against ast-rag <cmd> --help,
API signatures and imports against the source, config fields against
dto/config.py and the EmbeddingManager guard, ID formulas by computing the
hashes both ways.

Not re-measured: the Phase-2 numbers themselves need live Neo4j and Qdrant.
git log -S "F1 Score" -- README.md dates them to 1852590, 2026-02-26, so
they are dated and marked as not re-measured rather than restated as current.
Also left alone with reasons: QUICKSTART's indexing-time estimates, the
roadmap rows with no evidence either way, and the SUMMARIZATION model
recommendations.

Three things left for you

  • scripts/README.md documents both nonexistent shell scripts in detail, and
    README links it as "Indexing utilities". Needs deleting or rewriting — out of
    scope here.
  • docs/STACK_TRACE_MAPPING.md is in Russian while AGENTS.md says
    documentation is English. A policy call, not a false claim; only its imports
    were fixed.
  • AGENTS.md line 8 has a paste artifact: .qweAlways confirm documentation language preference.... Left as-is.

r0h1tb added 4 commits August 25, 2026 02:58
Audited README, AGENTS.md and docs/ against the current source.

- The CLI list was missing nine commands and gave `update` and `sandbox`
  signatures that do not parse.
- `docs/python-api.md` documented `get_diff(from_commit=, to_commit=,
  file_path=)`; `repo_path` is required and there is no per-file filter, so
  the documented call raises TypeError.
- The example config used `"bge-m3"` and an unexplained `remote_url`. The
  default is `BAAI/bge-m3` and embeddings are local; `remote_url` additionally
  requires `dimension`, which EmbeddingManager raises about.
- Nothing said the `ast-rag-mcp` and `ast-rag-watch` entry points need the
  `[mcp]` extra.
- The Phase-2 quality table read as current. Dated it, and said what the
  benchmarks measure and how to re-run them.
- `summarize` shipped but was still listed as planned; Go was still listed
  under "More Languages" after landing.
… source

extract_nodes returns the same 31 nodes for ast_rag_api.py with and without
source, so the note was wrong.
… weights

There is no ast_rag/embeddings.py. The knobs are embedding.vector_weight /
keyword_weight, exposed as --vector-weight / --keyword-weight on query.
@lexasub

lexasub commented Aug 25, 2026

Copy link
Copy Markdown
Owner

docs/STACK_TRACE_MAPPING.md - it's not a big deal that it's in English. If it's a nuisance, you can translate it into English. typo in agents.md, it's best to fix it.

@lexasub lexasub self-assigned this Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

2 participants