Orchestrator/bahn/teamlandkarte-mcp/openspec/changes/add-capacity-matching-mcp-server/tasks.md

1. Project Setup

• 1.1 Update pyproject.toml with required dependencies (mcp, trino, cachetools)
• 1.2 Create database.toml with connection configuration structure (template only, no real credentials)
• 1.3 Create project structure: src/, src/database/, src/matching/, src/cache/
• 1.4 Add .gitignore entry for database.toml to prevent credential exposure

2. Configuration Management

• 2.1 Create src/config.py to load database and matching configuration
• 2.2 Implement configuration validation (ensure required fields present)
• 2.3 Add configurable scoring thresholds (Top: ≥80%, Good: 60-79%, Partial: 40-59%, Low: <40%)
• 2.4 Add configurable matching weights (competence: 0.8, role: 0.2)
• 2.5 Validate that weights sum to 1.0
• 2.6 Support two cache TTL settings: db_ttl_hours (12h), search_ttl_minutes (60min)
• 2.7 Add fuzzy matching configuration: min_similarity (default: 0.7)

3. Database Layer

• 3.1 Create Trino client with connection handling
• 3.2 Implement connection pooling/management
• 3.3 VERIFY SCHEMA: Confirm database schema and field names with actual Open Data Lake tables (via Trino)
• 3.4 Implement single join query method:
  • 3.4.1 get_all_capacities_with_competences() - Single query joining all 3 tables
  • 3.4.2 Filter: deletion_reason IS NULL
  • 3.4.3 Use proper parameterized queries (no string interpolation)
  • 3.4.4 Group results by capacity_id in Python to build List[Capacity]
• 3.5 Ensure read-only operations (no INSERT/UPDATE/DELETE queries)
• 3.6 Add error handling for connection failures with retry logic (3x, exponential backoff)
• 3.7 Add query logging for debugging (without sensitive data)
• 3.8 NEW: Implement task database queries:
  • 3.8.1 Create Task dataclass with fields: id, title__c, description__c, startdate__c, enddate__c, createddate, skills
  • 3.8.2 get_open_tasks(limit) - Fetch published tasks sorted by createddate DESC
  • 3.8.3 JOIN task table with task skills table (task.id = skill.task__c)
  • 3.8.4 Filter: status__c = "Veröffentlicht"
  • 3.8.5 Group results by task_id in Python to build List[Task]
  • 3.8.6 get_task_by_id(task_id) - Fetch single task with skills by ID
  • 3.8.7 Use parameterized queries for task_id parameter

4. Caching Layer

• 4.1 Create src/cache/query_cache.py for database cache (12h TTL)
• 4.2 Create src/cache/search_cache.py for search results cache (60min TTL)
• 4.3 Implement QueryCache with cachetools.TTLCache:
  • 4.3.1 Key: "all_capacities_with_competences"
  • 4.3.2 TTL: 12 hours
  • 4.3.3 Cache statistics (hits/misses)
• 4.4 Implement SearchCache with cachetools.TTLCache:
  • 4.4.1 Key: search_id (UUID)
  • 4.4.2 Value: JSON-serialized search results with task_id (null or task ID), filters dict
  • 4.4.3 TTL: 60 minutes
  • 4.4.4 Methods: store_search() returns search_id, get_search(search_id) returns dict, update_search() for adding filters
• 4.5 Ensure thread-safe cache operations

5. Matching Engine

• 5.1 Create src/matching/matcher.py for matching logic
• 5.2 Implement task analysis using Azure OpenAI (chat completion):
  • 5.2.1 NEW: Create common internal function _extract_requirements_from_description(description) as single source of truth
  • 5.2.2 Free-text mode: analyze_free_text() delegates to common extraction function
  • 5.2.3 Structured mode: Parse provided parameters directly
  • 5.2.4 Generate and return task analysis summary for free-text mode
  • 5.2.5 Handle Azure API errors (timeouts, invalid JSON) with retry logic
  • 5.2.6 NEW: Implement internal helper extract_requirements_from_task(task_id) - fetches task from DB and uses common extraction
  • 5.2.7 UPDATED: Implement internal helper extract_roles_from_task(description) - returns ranked role list; primary role is roles[0] (if any)
• 5.3 Implement matching algorithm:
  • 5.3.1 Semantic competence matching via Azure OpenAI embeddings
  • 5.3.2 Role name matching (exact and similarity)
  • 5.3.3 Availability date range filtering (open-ended capacities: missing end_date means available without limit)
  • 5.3.4 Availability is NOT scored. If date_start/date_end are provided, availability is used as a filter only.
  • 5.3.5 Apply configured weights (competence + role) to calculate overall score
• 5.4 Create src/matching/scorer.py for scoring logic
• 5.5 Implement categorical scoring (Top/Good/Partial/Low) with configurable thresholds
• 5.6 Generate match score and category for each capacity
• 5.7 Group results by category for storage

6. MCP Server Implementation

• 6.1 Create src/mcp_server.py with MCP SDK integration
• 6.2 Integrate Azure OpenAI client for LLM access (chat + embeddings)
• 6.3 Implement session state management (in-memory, per client session)
  • 6.3.1 Store last_requirements for use with update_requirements
  • 6.3.2 Clear session state when new search is executed
• 6.4 NEW: Implement MCP tool: list_open_tasks (Phase 0)
  • 6.4.1 Tool parameters: limit (default: 20)
  • 6.4.2 Query database for published tasks (status = "Veröffentlicht")
  • 6.4.3 Sort by createddate DESC (newest first)
  • 6.4.4 Format as markdown table: Task ID, Title, Created Date, Start/End Dates, Skills Count
  • 6.4.5 Return table with next steps hints
• 6.5 NEW: Implement MCP tool: get_task_details (Phase 0)
  • 6.5.1 Tool parameters: task_id (required)
  • 6.5.2 Fetch task from database by ID
  • 6.5.3 Extract ranked roles from description using extract_roles_from_task() helper; assign primary role as roles[0]
  • 6.5.4 Format task details with metadata, description, DB skills, extracted role
  • 6.5.5 Return formatted details with next steps hints
• 6.6 NEW: Implement MCP tool: validate_task_requirements (Phase 0)
  • 6.6.1 Tool parameters: task_id (required)
  • 6.6.2 Fetch task from database
  • 6.6.3 Extract requirements from description using extract_requirements_from_task() helper
  • 6.6.4 Compare DB skills vs extracted skills (set operations)
  • 6.6.5 Compare DB dates vs extracted dates
  • 6.6.6 Build comparison table: Type, In Description, In Database, Status
  • 6.6.7 Generate summary with matched/missing counts
  • 6.6.8 Return validation table (informational only, no modifications)
• 6.7 NEW: Implement MCP tool: find_capacities_for_task (Phase 0)
  • 6.7.1 Tool parameters: task_id (required)
  • 6.7.2 Fetch task from database
  • 6.7.3 Extract requirements for validation display
  • 6.7.4 Run validation internally (build comparison table)
  • 6.7.5 Build search requirements using DB skills, extracted role, DB dates
  • 6.7.6 Execute capacity search (same as find_matching_capacities)
  • 6.7.7 Store in search cache with task_id field set to task ID
  • 6.7.8 Return validation table + search results (combined response)
• 6.8 EXISTING: Implement MCP tool: extract_requirements (Phase 1)
  • 6.8.1 Tool parameters: task_description (required), confirm_requirements (default: True)
  • 6.8.2 Use Azure OpenAI (chat completion) to extract structured requirements from free text
  • 6.8.3 Store extracted requirements in session state
  • 6.8.4 Return JSON-formatted requirements
  • 6.8.5 Include confirmation prompt if confirm_requirements=True
• 6.9 EXISTING: Implement MCP tool: collect_structured_requirement_data (Phase 1)
  • 6.9.1 Tool parameters: role_name, competences, date_start, date_end (all optional), confirm_requirements (default: True)
  • 6.9.2 Check completeness: require at least role_name AND competences
  • 6.9.3 Prompt for missing required parameters with examples
  • 6.9.4 Store collected requirements in session state
  • 6.9.5 Return JSON-formatted requirements with confirmation prompt
• 6.10 EXISTING: Implement MCP tool: update_requirements (Phase 1)
  • 6.10.1 Tool parameters: change_description (required), confirm_requirements (default: True)
  • 6.10.2 Check if requirements exist in session state
  • 6.10.3 Use Azure OpenAI (chat completion) to interpret change and update requirements
  • 6.10.4 Update session state with modified requirements
  • 6.10.5 Return updated JSON-formatted requirements with confirmation prompt
• 6.11 EXISTING: Implement MCP tool: find_matching_capacities (Phase 2)
  • 6.11.1 Tool parameters: role_name (required), competences (required), date_start (optional), date_end (optional)
  • 6.11.2 NO extraction or interpretation - accept only explicit structured parameters
  • 6.11.3 Execute matching algorithm (uses Azure OpenAI embeddings for competence matching; score uses competences+role only; availability is filter-only)
  • 6.11.4 Generate summary table with count by score category (markdown format)
  • 6.11.5 Store complete results in SearchCache with task_id=null, empty filters dict
  • 6.11.6 Return search_id + summary + Top category results (markdown table)
  • 6.11.7 Clear session state after successful search
• 6.12 EXISTING: Implement MCP tool: filter_search_results (Phase 3)
  • 6.12.1 Ensure fuzzywuzzy dependency is installed
  • 6.12.2 Tool parameters: search_id (required), role_filter (optional), competence_filter (optional List[str]), availability_date_start (optional), availability_date_end (optional), min_similarity (default: 0.7)
  • 6.12.3 ALWAYS use original search_id as input (not filtered search_id)
  • 6.12.4 Validate at least one filter is provided
  • 6.12.5 Retrieve original search results from SearchCache
  • 6.12.6 Collect all results across all categories
  • 6.12.7 Apply fuzzy matching: role_filter (fuzzy match on role_name), competence_filter (ALL competences must match at least one capacity competence)
  • 6.12.8 Combine filters with AND logic
  • 6.12.9 Re-categorize filtered results using Scorer
  • 6.12.10 Generate unique filter_id (e.g., "filter-1", "filter-2")
  • 6.12.11 Store filtered results under search_data["filters"][filter_id]
  • 6.12.12 Update search cache with filter data
  • 6.12.13 Return filter_id + search_id + filter summary + Top category results
• 6.13 EXISTING: Implement MCP tool: get_results_by_category (Phase 3)
  • 6.13.1 Tool parameters: search_id (required), filter_id (optional), category (default: "Top"), page (default: 1), page_size (default: 20)
  • 6.13.2 Retrieve search results from SearchCache by search_id
  • 6.13.3 If filter_id provided, retrieve filtered results from search_data["filters"][filter_id]
  • 6.13.4 If filter_id not provided, use original search results
  • 6.13.5 Extract category results, apply pagination
  • 6.13.6 Format as markdown table with full competence lists
  • 6.13.7 Return paginated results with page info and next page hint
• 6.14 Format results as markdown tables with fields: id, owner_name, role_name, role_level, begin_date, end_date, competences
• 6.15 Show full competence lists (no truncation)
• 6.16 Add tool descriptions and parameter schemas for MCP client

Update (2026-02): Items previously described as "via MCP sampling" are now implemented with Azure OpenAI (chat + embeddings) as part of the change replace-heuristics-with-azure-openai.

There is no runtime MCP sampling dependency and no heuristic fallback path in the current implementation.

7. Integration & Entry Point

• 7.1 Update main.py to launch MCP server
• 7.2 Add command-line arguments for server configuration (port, debug mode, etc.)
• 7.3 Implement graceful startup and shutdown
• 7.4 Add logging configuration

8. Testing & Validation

• 8.1 Manual testing with sample task descriptions
• 8.2 Verify database connection and single join query execution
• 8.3 Test both input modes (free-text via Azure OpenAI and structured)
• 8.4 NEW: Test incremental structured mode (partial parameters with prompts)
• 8.5 Validate scoring and categorization with updated weights (0.8/0.2)
• 8.6 Test that availability does not affect score and is only applied as an optional filter
• 8.7 Test two-tier caching behavior (DB cache 12h, search cache 60min)
• 8.8 Verify read-only constraint (no writes to database)
• 8.9 Test error handling (database unavailable, Azure OpenAI timeout, invalid search_id)
• 8.10 Test search session management (search_id creation and retrieval)
• 8.11 Test pagination in get_results_by_category
• 8.12 Verify markdown table formatting and full competence display
• 8.13 NEW: Test filter_search_results with role filter only
• 8.14 NEW: Test filter_search_results with competence filter only
• 8.15 NEW: Test filter_search_results with combined filters
• 8.16 NEW: Test multi-step filtering (filter a filtered search)
• 8.17 NEW: Test fuzzy matching with various similarity thresholds

9. Documentation

• 9.1 Update README.md with project description and setup instructions
• 9.2 Document database.toml structure and required fields
• 9.3 Document three MCP tools and their parameters (find_matching_capacities, get_results_by_category, filter_search_results)
• 9.4 Add usage examples for both input modes (free-text and incremental structured)
• 9.5 Document configuration options (weights, thresholds, cache TTLs, fuzzy matching)
• 9.6 Add troubleshooting guide
• 9.7 Document Azure OpenAI architecture and client-side LLM requirement
• 9.8 Document pagination and search session workflow
• 9.9 Document availability behavior (display + date-range filtering; open-ended capacities)
• 9.10 NEW: Document fuzzy filtering workflow and multi-step filtering
• 9.11 NEW: Add examples of filter_search_results usage

10. Security & Configuration

• 10.1 Ensure database.toml is in .gitignore
• 10.2 Create database.toml.example as template (already done)
• 10.3 Validate secure credential handling
• 10.4 Document security best practices
• 10.5 Add warning about credential rotation if database.toml was committed

11. Architecture Documentation

• 11.1 Review architecture.md (Arc42 format) - already created with 9 ADRs
• 11.2 Validate ADRs (Architecture Decision Records) align with implementation
• 11.3 Update component diagrams if needed during implementation
• 11.4 Verify ADR-007 (Availability is a Filter, Not a Scoring Criterion) implementation
• 11.5 NEW: Verify ADR-008 (Incremental Structured Mode) implementation
• 11.6 NEW: Verify ADR-009 (Fuzzy Filtering) implementation
• 11.7 Document any additional architectural decisions made during development

Notes

• All database operations must be read-only
• LLM access only via Azure OpenAI (chat + embeddings)
• Two separate caches with different TTLs (DB: 12h, Search: 60min)
• Single join query for all data retrieval
• Markdown tables for all output
• Full competence lists (no truncation)
• Search sessions via search_id (60min expiry)
• Initial implementation focuses on core functionality; refinements in future stages
• Total: tracked via openspec list (this markdown count is not authoritative)