Repository navigation
fix(server): resource links that resolve, theorem index kept intact - #552
Merged
Merged
Conversation
search_papers built one resource_link per result row from the uncapped row list, after the byte cap had already dropped trailing rows from structuredContent. When the cap fired, content[] pointed at chunks that were not in the payload it shipped with. The links are now built from structuredContent["results"], the rows that survived. The docstring said search_papers does not cap at all, though it has for a long time. It now says what the code does, including that the links are outside the cap's measurement: about 200 bytes each, ~10 KB for 50 rows against the 256 KB cap. A new test drives the handler with a cap that keeps about half the rows and expects exactly one link per kept row. The unfixed handler emits ten links for five rows. Closes #352.
search_papers emits a resource_link per result, and get_chunk emits
one when a body is over the byte cap, all to arxmcp://chunks/<id>. No
resource was registered at that URI, so every link failed with
"Unknown resource". The get_chunk tool description tells agents to
follow the link for the full payload.
Register an arxmcp://chunks/{chunk_id} template. It returns the same
object a scalar get_chunk does, with the same sanitized, delimiter-
wrapped body, but without the per-response byte cap: that cap is what
produced the link, so applying it again would make the link useless.
Ingest bounds a chunk body to a few thousand tokens, so the uncapped
read stays small. A malformed id is an error; an id not in the corpus
returns found:false.
Making the advertised contract true was preferred over dropping the
links, because rewording the tool description would move
EXPECTED_TOOL_SCHEMA_SHA256. Resources are a separate JSON-RPC method,
so tools/list is unchanged, and test_mcp_resources and
test_server_tool_schema pass.
New tests drive get_chunk over a small cap, then read the
resource_link_uri it returns and get the whole body back. On the
unfixed server that read raises "Unknown resource".
Closes #328.
When the theorem-names SQLite file predated the store's schema version, Resources.startup opened it with the migrating path, which drops and recreates the tables empty. The served process, which writes no corpus state, wiped the index. find_lemma_by_name then answered every lookup with no matches under retrieval_mode fuzzy_jaccard, which reads exactly like a real "no such lemma". An index that was simply empty answered the same way. TheoremNamesStore.open gains migrate (default True, so the offline indexer still rebuilds). The server opens with migrate=False: an out-of-date file raises TheoremNamesSchemaStale and is left untouched. Both that and an empty store now return None, which routes the handler to its documented in_memory_scan_fallback until `python -m ingest.index_theorem_names` rebuilds the index. The cost is a slower scan, not a wrong answer. New tests: a schema-0 file with a row stays byte-for-byte as it was and is not served; an empty store is not served; a populated one is; and a lookup over the stale file finds the lemma via the fallback. On the unfixed server the first two are served and the lookup returns fuzzy_jaccard with no matches. Closes #359.
#547 removed the formal resources from server/mcp_resources.py, next to where this branch adds the chunk resource. Kept the chunk resource and dropped everything formal: the resource count becomes four, and the read_corpus_version import goes with the code that used it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Three findings from the architecture review, one commit each. Review commit by commit; each commit stands alone and reverts cleanly.
Closes #352. Closes #328. Closes #359.
d4b5f48search_papersbuilt oneresource_linkper row from the uncapped row list, after the byte cap had already dropped trailing rows fromstructuredContent. When the cap fired,content[]pointed at chunks that were not in the payload. The links are now built fromstructuredContent["results"]. The docstring claiming the handler does not cap is corrected, and now says the links sit outside the cap's measurement: about 200 bytes each, ~10 KB at 50 rows against 256 KB.1b5f67farxmcp://chunks/<id>) failed withUnknown resource, because nothing was registered there. This adds anarxmcp://chunks/{chunk_id}template, served byserver.handlers.chunk.read_chunk_in_full. It returns the same object as a scalarget_chunk, with the same sanitized, delimiter-wrapped body, but without the per-response byte cap, since that cap is what produced the link. A malformed id is an error; an id not in the corpus returnsfound: false.40f147cResources.startupopened the file through the migrating path, which drops and recreates the tables empty. The served process wiped the index, andfind_lemma_by_namethen answered every lookup with no matches underretrieval_mode: fuzzy_jaccard, which reads like a real "no such lemma". An empty index answered the same way.TheoremNamesStore.opengainsmigrate(defaultTrue, so the offline indexer still rebuilds). The server opens withmigrate=False: a stale file raisesTheoremNamesSchemaStaleand is left untouched. A stale or empty store now returnsNone, so lookups fall back to the documentedin_memory_scan_fallbackuntilpython -m ingest.index_theorem_namesrebuilds the index.Why serve the links rather than drop them (#328)
The issue offers both options. The
get_chunktool description tells agents to follow the link for the full payload, and rewording that description would moveEXPECTED_TOOL_SCHEMA_SHA256, which waits for the next bundledTOOL_SCHEMA_VERSIONre-pin. Registering the resource makes the advertised contract true now. Resources are a separate JSON-RPC method fromtools/call, sotools/listis byte-identical:test_mcp_resources.pyandtest_server_tool_schema.pypass.The uncapped read stays small because ingest bounds a chunk body to a few thousand tokens (
STMT_MAX_TOKENS1920; proofs are windowed at 448).Verification
Every new test fails on the unfixed code unless it says otherwise.
TestResourceLinksFollowTheCapdriveshandle_search_paperswith a cap that keeps about half of ten rows, and expects exactly one link per kept row. The unfixed handler emits ten links for five rows.tests/test_chunk_resource.py:get_chunkover a small cap truncates a ~19 KB body; reading theresource_link_uriit returned gives back the whole body.found: false, and a malformed id is an error.ValueError: Unknown resource. The test module itself cannot import there, so this was checked separately.TestServedProcessNeverRebuildsTheStore:in_memory_scan_fallback.fuzzy_jaccardwith no matches.pytestat8ac1dda(this branch merged withmainatd1906ba, run in the cloud sandbox): 5680 tests, 196 skipped, 1 failure. That failure istest_symlink_loop_is_rejected, which fails in the sandbox for a reason unrelated to this change.ruff check .is clean.Notes
mainafter Remove release pins and the formal-contract tooling #547 (8ac1dda). Remove release pins and the formal-contract tooling #547 removed the formal resources fromserver/mcp_resources.py, next to where this adds the chunk resource. The five conflicting hunks keep the chunk resource and drop everything formal: the resource count is four, and theread_corpus_versionimport goes with the code that used it.get_chunkreads. A chunk that only a per-callfilters.notebooksearch can reach is not routable this way either (Notebook routing is a private feature of search_papers, so the canonical search→get_chunk workflow silently breaks across notebooks — and the session middleware reaches into get_chunk's argument shape to compensate for the missing seam #334).