Orchestrator/bahn/teamlandkarte-mcp/openspec/changes/add-capacity-to-task-matching-tools/tasks.md

Tasks: Add capacity→task matching tools and embedding-based inference

Change: add-capacity-to-task-matching-tools

Status: Implemented

Phase 1: Requirements + Configuration decisions

• 1.1 Define list_free_capacities ordering precisely:
  • order by most recent creation_date (descending)
• 1.2 Confirm “Owner/Team” display:
  • use owner_name (contains the team name)
• 1.3 Introduce a new role inference tool and remove the old:
  • add infer_primary_role(task_id? | task_text?) that returns the single closest role as a Markdown table with Role | Similarity columns
  • remove infer_roles(...) MCP tool completely (no deprecation period)
  • remove TaskAnalyzer.extract_ranked_roles() method completely
  • update extract_requirements() and other callers to use new primary-role inference
• 1.4 Add configuration for competence inference:
  • matching.inference.max_competences (int, default: 8; if not specified, no limit)
  • matching.inference.min_similarity (float in [0,1], default: 0.4; if not specified, no threshold)
  • at least one parameter MUST be specified to avoid unbounded results
  • selection rule: pick top X among those >= threshold when both are set
• 1.5 Define startup-only refresh behavior:
  • on startup embed only cache-missing role/competence vocab entries
  • on startup embed all open tasks (title+description by task_id)
  • at runtime still embed on-demand for misses (strict failure semantics)

Phase 2: Data Lake queries (roles, competences, capacities)

• 2.0 Verify Data Lake schema:
  • confirm teamlandkarte_v_capacity_roles_latest has columns: name, active, staffing_board_relevant
  • confirm capacities view has creation_date column
• 2.1 Add TrinoClient.get_recent_free_capacities(limit) query returning:
  • capacity_id, owner_name (team), role_name, creation_date, begin_date, end_date, competences[]
  • ordering: creation_date DESC
• 2.2 Add TrinoClient.get_capacity_by_id(capacity_id) query returning the same fields.
• 2.3 Add TrinoClient.get_all_role_names():
  • source: teamlandkarte_v_capacity_roles_latest
  • filter: active=true AND staffing_board_relevant=true
  • return unique, non-empty role names
• 2.4 Add TrinoClient.get_all_competence_names():
  • source: beschaffungstool_kmp_skill_latest.skillname__c
  • return unique, non-empty competence names
• 2.5 Add unit tests for SQL read-only guard + query shape (where applicable).

Phase 3: Startup embedding preload (vocabularies + tasks)

• 3.1 Define stable cache-key scheme (explicit format as documented in design.md):
  • role vocab entries: f"role_vocab:{model}:{dims}:{normalized_role_name}"
  • competence vocab entries: f"comp_vocab:{model}:{dims}:{normalized_competence_name}"
  • task text embeddings by task_id: f"task_id:{model}:{dims}:{task_id}"
• 3.2 Implement VocabularyCache component (src/teamlandkarte_mcp/matching/vocabulary.py):
  • fetch role vocabulary + competence vocabulary from Trino
  • normalize/deduplicate
  • embed only cache-missing entries (batch + chunking via SimilarityEngine.prefetch_embeddings(...))
  • store into SQLite embedding cache
  • keep in-memory memoization for fast repeated inference
  • invoked from build_server() after DB + similarity engine init
• 3.3 Implement a “task embedding preload” routine executed on startup:
  • fetch all open tasks (no limit)
  • build task_text = title + "\n\n" + description (title optional)
  • embed only cache-missing task_id vectors
  • store into SQLite embedding cache
• 3.4 Runtime fallback:
  • if a role/competence/task embedding is missing during a tool call, embed on-demand (strict semantics)
• 3.5 Tests:
  • preload embeds only cache misses
  • preload writes to SQLite
  • missing-at-runtime triggers on-demand embedding

Phase 4: Embedding-only inference (roles + competences)

• 4.1 Implement primary role inference against role vocabulary:
  • input: combined title+description (or description-only for ad-hoc)
  • output: single closest role with similarity score as Markdown table: Role | Similarity
• 4.2 Add a new MCP tool:
  • infer_primary_role(task_id? | task_text?)
  • returns: Markdown table Role | Similarity (single best matching role)
  • validation: exactly one of task_id or task_text must be provided
• 4.3 Remove deprecated tools and methods:
  • remove infer_roles(...) MCP tool completely
  • remove TaskAnalyzer.extract_ranked_roles() method completely
• 4.4 Update callers to use new primary-role inference:
  • update get_task_details to use infer_primary_role
  • update extract_requirements() to use new primary-role inference
  • update any internal helpers relying on TaskAnalyzer.extract_ranked_roles()

Phase 5: Rework task validation (embedding-only)

• 5.1 Replace chat-based extraction in validate_task_requirements / extract_requirements_from_task with embedding-based inference:
  • roles: infer primary role via role vocabulary (Markdown table output)
  • competences: infer competences via competence vocabulary with max X and threshold (Markdown table output)
• 5.2 Update output format:
  • keep current table-first display
  • add inferred primary role as Markdown table: Role | Similarity (single row)
  • add inferred competences as Markdown table: Competence | Similarity (descending order by similarity)
• 5.3 Tests:
  • validation uses embeddings path only (no chat calls)
  • respects max X and threshold rules
  • output includes role and competence similarity tables

Phase 6: New MCP tools (capacity browsing + capacity→task matching)

• 6.1 Implement list_free_capacities(limit=20):
  • ordering: creation_date DESC
  • Markdown table columns: capacity_id, Owner/Team, Role, Competences, Availability
  • stable empty-table handling (dummy row)
• 6.2 Implement get_capacity_details(capacity_id):
  • table-first, same fields as list
  • include a “Next steps” pointer to find_matching_tasks(capacity_id=...)
• 6.3 Implement find_matching_tasks(capacity_id):
  • signature: takes only capacity_id (no optional params)
  • fetch capacity + all open tasks
  • infer each task’s primary role via embedding similarity (title+description)
  • infer each task’s competences via embedding similarity (title+description) with max/threshold
  • compute a similarity score (role + competences) and categorize results
  • return output as Markdown table with columns:
    • task_id, Title, Required Competences, Availability, Score, Category
  • results are sorted by similarity score (descending)
  • output structure matches find_matching_capacities (includes summary counts table + results table for first non-empty category)
  • paginated via same search session conventions used by find_matching_capacities
• 6.4 Tests for new tools:
  • list_free_capacities: ordering by creation_date DESC, empty-table handling
  • get_capacity_details: not-found handling, "Next steps" section
  • find_matching_tasks: categorization (Top/Good/Partial/Low), search session conventions, table structure, score sorting

Phase 7: Remove Azure chat API

• 7.1 Remove chat config fields and environment variables if no longer needed (AZURE_OPENAI_LLM_API_KEY, chat deployment, chat codepaths).
• 7.2 Delete/retire chat client methods and TaskAnalyzer chat-based methods.
• 7.3 Update docs to remove chat setup instructions.
• 7.4 Ensure all gates pass (ruff/mypy/pytest).

Phase 8: Documentation + assistant prompt + architecture

• 8.1 Update README.md:
  • describe new tools and capacity→task workflow
  • document startup embedding preload and inference config knobs
• 8.2 Update docs/assistant_system_prompt.md:
  • add guidance for new tools and matching direction
  • update role/competence inference description (embeddings-only)
• 8.3 Update architecture docs (Arc42 / design docs):
  • reflect removal of chat API
  • describe vocab/task embedding preload and cache keys

Phase 9: OpenSpec hygiene

• 9.1 Update spec deltas to reflect confirmed tool shapes + config.
• 9.2 Run openspec validate add-capacity-to-task-matching-tools --strict.