Orchestrator/bahn/teamlandkarte-mcp/openspec/changes/add-capacity-matching-mcp-server/proposal.md

Change: Add Capacity Matching MCP Server (Stage 1)

Why

DB Systel employees have free work capacities stored in the Open Data Lake, but there's no efficient way for AI assistants to match these capacities with tasks. Tasks are also stored in the database, but finding suitable capacities for them requires manual work. This MCP server will enable AI clients to automatically:

1. Browse and search published tasks from the database
2. Validate task requirements (comparing DB skills vs description-extracted skills)
3. Find the best-suited employees for both database tasks and ad-hoc requirements
4. Match capacities based on competences and role, with optional availability (date-range) filtering

What Changes

• Add MCP server implementation with 10-tool workflow (task management, requirement gathering, search, and refinement)
• Add database task management (4 new tools): list tasks, view details, validate requirements, automated search
• Add task database integration: Query published tasks with skills from Beschaffungstool KMP tables
• Add requirement validation: Compare database skills vs LLM-extracted skills from description
• Add common extraction function: DRY principle with _extract_requirements_from_description()
• Add internal helper functions: extract_requirements_from_task(), extract_roles_from_task()
• Add database connection layer for Trino/Presto with single join query
• Requirement extraction and semantic matching are implemented via Azure OpenAI:
  • Chat completions for role/requirement extraction
  • Embeddings + cosine similarity for semantic similarity scoring
  • Errors are deterministic (no heuristic fallbacks)

Historical note (2026-02): The initial Stage-1 proposal assumed only client-side LLM access via MCP sampling and used heuristic fallbacks because sampling was not available in the runtime. The current implementation has been migrated to an Azure OpenAI-only integration (see change replace-heuristics-with-azure-openai).

• Add session state management for requirement tracking across tool calls
• Add configurable scoring system with categorical results (Top/Good/Partial/Low)
• Add two-tier caching: database cache (12h TTL) and search results cache (60min TTL) with task_id tracking
• Emit deterministic, machine-readable headers in search-related tool outputs so MCP clients can reliably capture search_id / filter_id:
  • first line: Using SEARCH_ID=<uuid>
  • plus SEARCH_ID=<uuid>, FILTER_ID=<uuid> (if applicable), META=<json>
• Reject non-UUID search_id values early with a clear message
• Track the latest search id internally for debugging/consistency, but do not expose a "last search id" tool (multi-user/session unsafe)
• Add requirement workflow: extract from free text, collect structured, or update requirements
• Add explicit confirmation before search execution
• Remove availability from weighted scoring (availability is displayed and can be used as a filter)
• Update default weights: competence 0.8, role 0.2
• Keep date range as an optional availability filter (when provided)
• Add interactive result exploration with search session management (search_id)
• Add fuzzy filtering for post-search refinement (role and/or competences as list)
• Add filter tracking with filter_id (no search_id proliferation)
• Add pagination for result browsing (20 results per page)
• Add database configuration in database.toml (Trino)
• Add Arc42 architecture documentation with ADRs

Strict two-step confirmation (Review → Ask → Confirm)

When matching.require_confirmation = true (default), the server enforces a strict, user-driven confirmation sequence before any matching run:

1. Requirements are captured/updated into pending state (e.g. via extract_requirements, collect_structured_requirement_data, update_requirements, or guided capture tools).
2. The assistant must review pending requirements using show_pending_requirements() (preferred) or, if requirements were already shown to the user in chat, call request_requirements_confirmation() as a marker.
3. The assistant must ask the user explicitly for Yes/No confirmation.
4. Only after user approval, the assistant calls confirm_requirements(confirm=true). If the user says No, it calls confirm_requirements(confirm=false) and continues requirement capture.

The server refuses to confirm requirements unless step 2 has happened first (prevents auto-confirm in the same turn as requirement capture).

Stage 1 Scope: This is the first development stage focusing on basic matching functionality and database task integration. Future stages will add more sophisticated features (to be specified later).

Key Architectural Decisions:

• Azure OpenAI integration: Server calls Azure OpenAI directly for extraction and embeddings (no sampling dependency)
• 10-tool workflow: Separate tools for task management (4), requirement gathering (3), search execution (1), and result refinement (2)
• Database task integration: Direct access to published tasks with skills from Beschaffungstool KMP
• Validation-only approach: Validation compares DB vs description but always uses DB skills for search
• Common extraction function: _extract_requirements_from_description() as single source of truth (DRY principle)
• Automatic task workflow: find_capacities_for_task() fully automated with validation display
• Task reference tracking: Search cache includes task_id field (null for ad-hoc, task ID for DB tasks)
• Explicit confirmation: Requirements returned for user confirmation before search
• Session state: In-memory storage of requirements for update workflow
• Filter tracking: Filters stored within search cache using filter_id (not new search_id)
• Search sessions: Complete results stored server-side, accessed via search_id (60min expiry)
• Single query pattern: One JOIN query fetches all data, processed in Python
• Markdown output: All tables formatted as markdown for better client rendering
• No truncation: Full competence lists always shown
• No availability scoring: Availability is displayed and can be used as a date-range filter, but does not influence the score
• Weights: competence 0.8, role 0.2
• Fuzzy matching: Filter tool uses fuzzywuzzy for flexible result refinement
• LLM integration: Azure OpenAI (gpt-4.1 chat + text-embedding-3-large) with embedding cache; no heuristic fallback

Impact

• New capability: capacity-matching - Core matching functionality with database task integration
• New files:
  • database.toml - Database connection configuration
  • architecture.md - Arc42 architecture documentation with ADRs
  • src/teamlandkarte_mcp/mcp_server.py - MCP server implementation (10 tools)
  • src/teamlandkarte_mcp/database/trino_client.py - Trino client (capacity + task queries)
  • src/teamlandkarte_mcp/matching/task_analyzer.py - Azure-based task analysis
  • src/matching/matcher.py - Matching algorithm with common extraction function
  • src/matching/scorer.py - Scoring logic
  • src/cache/query_cache.py - Database query caching (12h TTL)
  • src/cache/search_cache.py - Search results caching (60min TTL) with task_id tracking
  • src/config.py - Configuration management
  • requirements.txt or updated pyproject.toml - Dependencies
• Modified files:
  • main.py - Launch MCP server
  • pyproject.toml - Add dependencies
  • README.md - Update with task management features, validation workflow
• Dependencies added:
  • mcp - Model Context Protocol SDK
  • trino - Trino/Presto database client
  • cachetools - Caching implementation
  • fuzzywuzzy - Fuzzy string matching for filters

Security & Constraints

• Read-only operations: Server must NEVER modify database data
• Credential storage: Credentials are provided via environment variables (recommended: local .env): DATA_LAKE_USERNAME, DATA_LAKE_PASSWORD. database.toml contains non-secret connection settings and must not be committed.
• Network security: Trino/Presto connection over port 8446
• Data filtering:
  • Capacities: Only consider where deletion_reason IS NULL
  • Tasks: Only consider where status__c = "Veröffentlicht"
• LLM access: Not available in the current server runtime. If reintroduced, it must use a supported MCP API.
• SQL injection prevention: All queries use proper parameterization
• Validation scope: Comparison only (skills and dates), role not validated (not in DB yet)

Non-Goals (Future Stages)

• Advanced filtering options beyond fuzzy matching
• Capacity booking/reservation
• Task modification or status updates
• Historical analysis
• Integration with other systems
• Multi-language support
• Role storage in database (currently extracted from description)