feat: improve agent UX for search_docs tool

.beads/issues.jsonl

@@ -3,3 +3,10 @@
 {"id":"agentic-knowledge-1.2","title":"Plan","description":"Create a detailed implementation strategy based on your exploration:  - If  exists: Base your strategy on requirements from it - Otherwise: Use existing task context  Break down the work into specific, actionable tasks. Consider edge cases, dependencies, and potential challenges.  - If architectural changes needed and  exists: Document in  - Otherwise: Create tasks to track architectural decisions  - If  exists: Adhere to the design in it - Otherwise: Elaborate design options and present them to the user  Document the planning work thoroughly and create implementation tasks as part of the code phase as needed.","status":"open","priority":3,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-10T09:55:15.524416+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-10T09:55:15.524416+01:00","dependencies":[{"issue_id":"agentic-knowledge-1.2","depends_on_id":"agentic-knowledge-1","type":"parent-child","created_at":"0001-01-01T00:00:00Z"},{"issue_id":"agentic-knowledge-1.2","depends_on_id":"agentic-knowledge-1.1","type":"blocks","created_at":"0001-01-01T00:00:00Z"}]}
 {"id":"agentic-knowledge-1.3","title":"Code","description":"Follow your plan to build the solution:  - If  exists: Follow the design from it - Otherwise: Elaborate design options and present them to the user - If  exists: Build according to the architecture from it - Otherwise: Elaborate architectural options and present them to the user - If  exists: Ensure requirements from it are met - Otherwise: Ensure existing requirements are met based on your task context  Write clean, well-structured code with proper error handling. Prevent regression by building, linting, and executing existing tests. Stay flexible and adapt the plan as you learn more during implementation. Update task progress and create new tasks as needed.","status":"open","priority":3,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-10T09:55:15.67957+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-10T09:55:15.67957+01:00","dependencies":[{"issue_id":"agentic-knowledge-1.3","depends_on_id":"agentic-knowledge-1","type":"parent-child","created_at":"0001-01-01T00:00:00Z"},{"issue_id":"agentic-knowledge-1.3","depends_on_id":"agentic-knowledge-1.2","type":"blocks","created_at":"0001-01-01T00:00:00Z"}]}
 {"id":"agentic-knowledge-1.4","title":"Commit","description":"Ensure code quality and documentation accuracy through systematic cleanup and review. **STEP 1: Code Cleanup** Systematically clean up development artifacts: 1. **Remove Debug Output**: Search for and remove all temporary debug output statements used during development.    Look for language-specific debug output methods (console logging, print statements, debug output functions).    Remove any debugging statements that were added for development purposes.  2. **Review TODO/FIXME Comments**:    - Address each TODO/FIXME comment by either implementing the solution or documenting why it's deferred    - Remove completed TODOs    - Convert remaining TODOs to proper issue tracking if needed  3. **Remove Debugging Code Blocks**:    - Remove temporary debugging code, test code blocks, and commented-out code    - Clean up any experimental code that's no longer needed    - Ensure proper error handling replaces temporary debug logging  **STEP 2: Documentation Review** Review and update documentation to reflect final implementation: 1. **Update Long-Term Memory Documents**: Based on what was actually implemented:    - If  exists: Update it if requirements changed during development    - If  exists: Update it if architectural impacts were identified    - If  exists: Update it if design details were refined or changed    - Otherwise: Document any changes in the plan file 2. **Compare Against Implementation**: Review documentation against actual implemented functionality 3. **Update Changed Sections**: Only modify documentation sections that have functional changes 4. **Remove Development Progress**: Remove references to development iterations, progress notes, and temporary decisions 5. **Focus on Final State**: Ensure documentation describes the final implemented state, not the development process 6. **Ask User to Review Document Updates** **STEP 3: Final Validation** - Run existing tests to ensure cleanup didn't break functionality - Verify documentation accuracy with a final review - Ensure code is ready for production/delivery Update task progress and mark completed work as you finalize the feature.","status":"open","priority":3,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-10T09:55:15.837066+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-10T09:55:15.837066+01:00","dependencies":[{"issue_id":"agentic-knowledge-1.4","depends_on_id":"agentic-knowledge-1","type":"parent-child","created_at":"0001-01-01T00:00:00Z"},{"issue_id":"agentic-knowledge-1.4","depends_on_id":"agentic-knowledge-1.3","type":"blocks","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5","title":"Improve agent UX for search_docs tool","status":"open","priority":2,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:20.15613+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:45:20.15613+01:00","dependencies":[{"issue_id":"agentic-knowledge-1.5","depends_on_id":"agentic-knowledge-1","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.1","title":"Rename keywords→pattern, generalized_keywords→fallback_pattern in MCP schema + handler","description":"Parameter names carry implicit semantics. 'keywords' suggests space-separated list; 'pattern' signals single regex expression","status":"closed","priority":1,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:25.754448+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:56:36.44898+01:00","closed_at":"2026-03-26T18:56:36.44898+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.1","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.2","title":"Change context_lines default from 0 to 2","description":"Default of 0 forces agents to make multiple calls. Default of 2 provides useful context on first try","status":"closed","priority":1,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:30.230618+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:56:58.835616+01:00","closed_at":"2026-03-26T18:56:58.835616+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.2","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.3","title":"Add recovery guidance to no-match response","description":"When search returns 0 matches, suggest using | alternation, relaxing pattern, or using fallback_pattern","status":"closed","priority":1,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:35.402586+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:57:22.864588+01:00","closed_at":"2026-03-26T18:57:22.864588+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.3","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.4","title":"Surface invalid regex as warning in result","description":"Currently invalid regex is silently treated as literal. Agent should know pattern was escaped","status":"closed","priority":2,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:39.290333+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:58:37.319396+01:00","closed_at":"2026-03-26T18:58:37.319396+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.4","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.5","title":"Make truncated message actionable","description":"Suggest narrowing with more specific pattern when results are truncated at 50","status":"closed","priority":2,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:43.069704+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T18:58:52.038744+01:00","closed_at":"2026-03-26T18:58:52.038744+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.5","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"}]}
+{"id":"agentic-knowledge-1.5.6","title":"Update internal types to stay in sync with parameter rename","description":"SearchDocsParams, TemplateContext, DEFAULT_TEMPLATE still use keywords - update to pattern","status":"closed","priority":2,"issue_type":"task","owner":"github@beimir.net","created_at":"2026-03-26T18:45:46.586209+01:00","created_by":"Oliver Jägle","updated_at":"2026-03-26T19:12:40.947871+01:00","closed_at":"2026-03-26T19:12:40.947871+01:00","close_reason":"Closed","dependencies":[{"issue_id":"agentic-knowledge-1.5.6","depends_on_id":"agentic-knowledge-1.5","type":"parent-child","created_at":"0001-01-01T00:00:00Z"},{"issue_id":"agentic-knowledge-1.5.6","depends_on_id":"agentic-knowledge-1.5.1","type":"blocks","created_at":"0001-01-01T00:00:00Z"}]}

.beads/last-touched

@@ -1 +1 @@
-agentic-knowledge-1.4
+agentic-knowledge-1.5.6

.vibe/development-plan-rename-search-parameter.md

@@ -0,0 +1,75 @@
+# Development Plan: agentic-knowledge (rename-search-parameter branch)
+
+_Generated on 2026-03-26 by Vibe Feature MCP_
+_Workflow: [epcc](https://mrsimpson.github.io/responsible-vibe-mcp/workflows/epcc)_
+
+## Goal
+
+Improve agent UX for the `search_docs` MCP tool. Agents currently misuse the tool by sending space-delimited keyword lists (e.g., "user authentication login") instead of regex patterns (e.g., "user|auth|login"). This causes failed searches and wasted round-trips.
+
+**Final Scope (simplified):**
+
+1. ✅ Rename `keywords` → `pattern` (strongest signal for regex)
+2. ✅ Remove `fallback_pattern` entirely — agents can refine patterns based on no-match hints
+3. ✅ Change `context_lines` default from 0 → 2 (reduce round-trips)
+4. ✅ Add recovery guidance to no-match responses (actionable feedback)
+5. ✅ Surface invalid regex as warning (transparency)
+6. ✅ Make truncated message actionable (guide narrowing)
+7. ✅ Update internal types to stay in sync
+
+**Out of scope:** Exposing `include` filter to agents (intentionally internal for now)
+
+## Explore
+
+<!-- beads-phase-id: TBD -->
+
+### Tasks
+
+_Tasks managed via `bd` CLI_
+
+## Plan
+
+<!-- beads-phase-id: TBD -->
+
+### Tasks
+
+_Tasks managed via `bd` CLI_
+
+## Code
+
+<!-- beads-phase-id: TBD -->
+
+### Tasks
+
+_Tasks managed via `bd` CLI_
+
+## Commit
+
+<!-- beads-phase-id: TBD -->
+
+### Tasks
+
+_Tasks managed via `bd` CLI_
+
+## Key Decisions
+
+- **Parameter naming:** Use `pattern` (not `regex` or `query`) — familiar from grep/ripgrep, signals single expression without being intimidating
+- **Removed fallback_pattern:** The no-match recovery guidance makes a separate fallback parameter redundant — agents can simply refine their pattern using the suggested techniques (|, .\*, etc.)
+- **Default context_lines:** 2 lines before/after — enough for understanding, not overwhelming
+- **Recovery guidance style:** Concise, actionable hints inline with the no-match message
+- **Backward compatibility:** This is a breaking change to the MCP tool schema, but acceptable since tool is new
+
+## Notes
+
+**Implementation complete** — All 265 tests passing. Changes:
+
+1. Renamed `keywords` → `pattern` in MCP schema, handler, types, and all tests
+2. Completely removed `fallback_pattern` / `fallbackPattern` from the API
+3. Changed `DEFAULT_CONTEXT_LINES` from 0 to 2
+4. Added comprehensive no-match hints with regex tips (`|`, `.*`, `\b`, etc.)
+5. Added warning when invalid regex is auto-escaped to literal
+6. Made truncated message actionable (suggests narrowing pattern)
+
+---
+
+_This plan is maintained by the LLM and uses beads CLI for task management. Tool responses provide guidance on which bd commands to use for task management._

index.ts

@@ -1,18 +1,17 @@
 /**
  * Agentic Knowledge System
- * 
+ *
  * A standalone agentic knowledge guidance system with search_docs() interface
- * that returns intelligent navigation instructions based on docset, keywords,
- * and generalized_keywords - delegating language processing to the agent
- * while providing structured guidance.
+ * that returns intelligent search results based on docset and regex pattern -
+ * delegating language processing to the agent while providing structured guidance.
  */
 
 // Export main MCP server functionality
-export { 
-  createAgenticKnowledgeServer, 
-  startMCPServer, 
-  startServer 
-} from './packages/mcp-server/dist/index.js';
+export {
+  createAgenticKnowledgeServer,
+  startMCPServer,
+  startServer,
+} from "./packages/mcp-server/dist/index.js";
 
 // Export core functionality for advanced usage
 export {
@@ -23,8 +22,8 @@ export {
   createTemplateContext,
   getEffectiveTemplate,
   validateTemplate,
-  extractVariables
-} from './packages/core/dist/index.js';
+  extractVariables,
+} from "./packages/core/dist/index.js";
 
 // Export types
 export type {
@@ -34,7 +33,7 @@ export type {
   SearchDocsResponse,
   ListDocsetsResponse,
   TemplateContext,
-  ErrorType
-} from './packages/core/dist/index.js';
+  ErrorType,
+} from "./packages/core/dist/index.js";
 
-export { KnowledgeError } from './packages/core/dist/index.js';
+export { KnowledgeError } from "./packages/core/dist/index.js";

packages/core/src/__tests__/loader.test.ts

@@ -35,13 +35,13 @@ docsets:
     sources:
       - type: "local_folder"
         paths: ["./docs"]
-    template: "Custom template for {{keywords}} in {{local_path}}"
+    template: "Custom template for {{pattern}} in {{local_path}}"
   - id: "api-docs"
     name: "API Documentation"
     sources:
       - type: "local_folder"
         paths: ["/absolute/path"]
-template: "Global template for {{keywords}} in {{local_path}}"`;
+template: "Global template for {{pattern}} in {{local_path}}"`;
     await fs.writeFile(validConfigPath, validConfig);
 
     // Create invalid config file (missing required fields)
@@ -133,7 +133,7 @@ docsets:
     sources:
       - type: "local_folder"
         paths: ["./docs"]
-template: "Global template with {{keywords}} and {{invalid_variable}}"`;
+template: "Global template with {{pattern}} and {{invalid_variable}}"`;
       await fs.writeFile(invalidTemplateConfigPath, invalidTemplateConfig);
 
       await expect(loadConfig(invalidTemplateConfigPath)).rejects.toThrow(
@@ -165,7 +165,7 @@ docsets:
     sources:
       - type: "local_folder"
         paths: ["./docs"]
-    template: "Docset template missing {{keywords}} but has {{bad_var}}"`;
+    template: "Docset template missing {{pattern}} but has {{bad_var}}"`;
       await fs.writeFile(
         invalidDocsetTemplateConfigPath,
         invalidDocsetTemplateConfig,
@@ -197,8 +197,8 @@ docsets:
     sources:
       - type: "local_folder"
         paths: ["./docs"]
-    template: "Search {{keywords}} in {{local_path}}"
-template: "Global: {{keywords}} in {{local_path}}"`;
+    template: "Search {{pattern}} in {{local_path}}"
+template: "Global: {{pattern}} in {{local_path}}"`;
       await fs.writeFile(validTemplateConfigPath, validTemplateConfig);
 
       const config = await loadConfig(validTemplateConfigPath);

packages/core/src/__tests__/searcher.test.ts

@@ -190,27 +190,6 @@ describe("searchDocset – context lines", () => {
   });
 });
 
-// ---------------------------------------------------------------------------
-// Fallback behaviour
-// ---------------------------------------------------------------------------
-
-describe("searchDocset – fallback pattern", () => {
-  test("uses fallbackPattern when primary yields no results", async () => {
-    const result = await searchDocset(fixtureDir, "xyzzy_no_match_ever", {
-      fallbackPattern: "authentication",
-    });
-    expect(result.total_matches).toBeGreaterThan(0);
-    expect(result.used_pattern).toBe("authentication");
-  });
-
-  test("does not fall back when primary yields results", async () => {
-    const result = await searchDocset(fixtureDir, "authentication", {
-      fallbackPattern: "xyzzy_no_match_ever",
-    });
-    expect(result.used_pattern).toBe("authentication");
-  });
-});
-
 // ---------------------------------------------------------------------------
 // Ignored paths
 // ---------------------------------------------------------------------------