Enhance MCP server manifest with metadata for better LLM discovery

[wspringer]

Summary

Enhance the HiGHS MCP server manifest with rich metadata to improve LLM discovery and usage of optimization capabilities.

Problem

Currently, the HiGHS MCP server has minimal metadata that makes it difficult for LLMs to:

• Recognize when optimization problems are being discussed
• Understand the full capabilities of the server
• Know which problem types can be solved

Proposed Changes

Update the server manifest structure to include:

{
  "name": "highs-mcp",
  "version": "0.4.0",
  "description": "Intelligent optimization solver with natural language support",
  "tools": {
    "primary": ["optimize-mip-lp-tool"],
    "assistive": [
      "parse-optimization-problem",
      "build-problem-interactive",
      "what-if-analysis",
      "diagnose-problem"
    ],
    "reporting": [
      "visualize-solution",
      "generate-optimization-report"
    ]
  },
  "capabilities": {
    "natural_language": true,
    "interactive_mode": true,
    "visualization": true,
    "learning": true,
    "data_formats": ["csv", "json", "excel"],
    "problem_types": ["single_stage", "multi_stage", "stochastic"],
    "max_variables": 1000000,
    "max_constraints": 1000000
  },
  "prompts": {
    "system": "Enhanced optimization assistant prompt...",
    "examples": ["transportation.md", "scheduling.md", "portfolio.md"]
  },
  "keywords": ["maximize", "minimize", "optimize", "allocate", "schedule", "route"],
  "trigger_phrases": [
    "find the best combination",
    "minimize total cost",
    "maximize profit",
    "optimal allocation"
  ]
}

Impact Measurement

To validate the effectiveness of these changes, we should measure:

1. Discovery Metrics

• Baseline: Count how often LLMs currently suggest HiGHS for optimization problems
• Test Set: Create 50 optimization problem descriptions in natural language
• Success Rate: Measure % of problems where LLMs correctly identify HiGHS as relevant
• Target: Increase discovery rate from current baseline by 50%

2. Usage Metrics

• Tool Selection Time: Measure time between problem presentation and tool selection
• False Positives: Track when HiGHS is suggested for non-optimization problems
• Tool Switching: Monitor how often users need to switch from other approaches to HiGHS

3. A/B Testing Protocol

Group A: Current minimal manifest
Group B: Enhanced manifest with metadata

Test scenarios:
- Transportation problems
- Resource allocation
- Scheduling tasks
- Portfolio optimization
- General "find the best" queries

4. Qualitative Measures

• Survey LLM responses for appropriateness of tool selection
• Analyze the quality of problem formulations generated
• Track user satisfaction with tool discovery

5. Automated Testing

Create a test suite that:

test_cases = [
    {
        "query": "I need to minimize shipping costs between 5 warehouses and 10 stores",
        "expected_tool": "highs-mcp",
        "problem_type": "transportation"
    },
    {
        "query": "What's the optimal way to allocate my $100k budget across 7 projects?",
        "expected_tool": "highs-mcp",
        "problem_type": "allocation"
    }
]

Implementation Steps

1. Create enhanced manifest structure
2. Add keyword and trigger phrase detection
3. Implement capability reporting
4. Set up A/B testing framework
5. Create measurement dashboard
6. Run 2-week testing period
7. Analyze results and iterate

Success Criteria

• 50% increase in correct tool discovery rate
• 30% reduction in time to tool selection
• Less than 5% false positive rate
• Positive user feedback on tool suggestions

Related Work

See ENHANCEMENT_PLAN.md for the full enhancement roadmap.