Skip to content

fix(tools): strip NetCDF tokens; anchor list_available_output_files to parquet - #23

Merged
romer8 merged 1 commit into
mainfrom
fix/parquet-only-anchor-no-netcdf-tokens
May 28, 2026
Merged

fix(tools): strip NetCDF tokens; anchor list_available_output_files to parquet#23
romer8 merged 1 commit into
mainfrom
fix/parquet-only-anchor-no-netcdf-tokens

Conversation

@romer8

@romer8 romer8 commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • _tool_descriptions.py: rewrite QUERY_FILES_BY_SELECTOR_DESCRIPTION with Parquet only. hoisted next to the opener; drop unsupported_format, _excluded_netcdf_count, and mixed-format sentences. Add LIST_AVAILABLE_OUTPUT_FILES_DESCRIPTION ending with Parquet only.
  • tools.py:296 routes through the new constant; tools.py:444 drops "NetCDF files do not consume index slots" from the index Field description.
  • test_tool_descriptions.py: drop the positive lockstep on unsupported_format and _excluded_netcdf_count (they locked the bug in); add negative "netcdf" not in desc.lower() on query_files_by_selector and a new test_list_available_output_files_description_anchor.

Why

A weak Q4 quantized model (SimonPu/GLM-4.7-Flash:Q4_K_M) replied to the user's prompt list the last 10 available dates for model cfe_nom with a hallucinated query_data tool and the prose:

For NetCDF files: Use list_available_output_files to find files, then I can help you access them.

That's factually wrong — list_available_output_files is parquet-only since v0.5.0. The cause is description-token leakage. query_files_by_selector mentioned NetCDF five times across _tool_descriptions.py and tools.py:444 as negative constraints (unsupported_format, _excluded_netcdf_count, "do not consume index slots"). Weak quantized models token-match on the format name regardless of polarity, so "NetCDF unsupported" reads as "supports NetCDF." Combined with list_available_output_files having no parquet anchor on a generically-named tool, the model assigned the NetCDF role to that tool and surfaced it to the user.

Runtime envelope keys (unsupported_format:, _excluded_netcdf_count) keep emitting on the wire — we just stop teaching the LLM their names from description prose.

Generalizes the existing "no concrete example values" convention: no format names of formats we don't support either, even as negative constraints. The rationale is added to the _tool_descriptions.py module docstring.

Test plan

  • .venv-mcp/bin/python -m pytest test_mcp/ -q — 175 passed locally
  • Defense-in-depth: grep -niE "netcdf|nc4|\.nc[ ,\"']" nextgen_mcp/{prompts,tools,_tool_descriptions}.py — only the module-docstring intentional reference remains
  • Manual smoke against the same prompt + a weak local model — confirm the reply no longer surfaces NetCDF and list_available_dates is chosen (model-fitness dependent; this PR does not promise it for sub-7B Q4 models)

…s to parquet

Weak Q4 quantized models (observed SimonPu/GLM-4.7-Flash:Q4_K_M) token-match
the format name regardless of polarity. With NetCDF named four times across
query_files_by_selector's description and `index` arg as negative constraints,
plus no parquet anchor on the generically-named list_available_output_files,
the model assigned the NetCDF role to list_available_output_files and emitted
user-facing prose routing NetCDF prompts there — contradicting the v0.5.0
parquet-only decision.

- _tool_descriptions.py: rewrite QUERY_FILES_BY_SELECTOR_DESCRIPTION with
  "Parquet only." hoisted next to the opener; drop unsupported_format,
  _excluded_netcdf_count, and mixed-format sentences. Add
  LIST_AVAILABLE_OUTPUT_FILES_DESCRIPTION ending with "Parquet only."
- tools.py:296: route through the new constant.
- tools.py:444: drop "NetCDF files do not consume index slots" from the
  `index` Field description.
- test_tool_descriptions.py: drop positive lockstep on unsupported_format
  and _excluded_netcdf_count (they locked the bug in); add negative
  `"netcdf" not in desc.lower()` on query_files_by_selector and a new
  test_list_available_output_files_description_anchor.

Runtime envelope keys (`unsupported_format:`, `_excluded_netcdf_count`) are
untouched on the wire — the LLM just no longer learns their names from
description prose.
@romer8
romer8 merged commit 1e6cb39 into main May 28, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant