Orchestrator/bahn/teamlandkarte-mcp/.kiro/specs/remove-embedding-competence-similarity/tasks.md

# Implementation Plan: Entfernung Embedding-basierter Similarity – Umstellung auf BM25 + LLM

## Overview

Incremental removal of all embedding infrastructure and replacement with BM25 + LLM. Tasks are ordered to avoid breaking the system mid-way: first add new LLM-based methods, then remove embedding code paths, then clean up config and delete dead modules.

## Tasks

- [x] 1. Simplify AzureOpenAIClient to remove embedding methods
  - [x] 1.1 Remove `embeddings` and `get_embeddings_batch` methods from `AzureOpenAIClient`
    - Remove the `self._emb` AsyncAzureOpenAI client instance
    - Remove constructor parameters: `embedding_api_key`, `embedding_deployment`, `embedding_batch_size`
    - Keep only `chat_completion` method and its supporting `self._chat` client
    - Make `self._chat` the primary client (no longer optional/conditional)
    - Remove the guard clause in `chat_completion` that checks for missing config
    - _Requirements: 6.7_

  - [x] 1.2 Update `AzureOpenAIClient` constructor signature
    - New required params: `endpoint`, `api_version`, `chat_deployment`, `llm_api_key`
    - Keep optional: `timeout_s`, `max_retries`, `cost_tracker`
    - Remove cost tracker calls for embedding requests (`log_embedding_request`, `log_embedding_batch_request`)
    - _Requirements: 6.7_

- [-] 2. Refactor SimilarityEngine to remove embedding methods and add LLM role similarity
  - [x] 2.1 Remove embedding-related methods and properties from `SimilarityEngine`
    - Remove methods: `prefetch_embeddings`, `_embed`, `get_embedding_for_cache_key`, `get_embeddings_for_cache_keys`, `_aggregate_similarity`, `_per_skill_similarity`
    - Remove properties: `embedding_model`, `embedding_dimensions`, `use_bm25_search`
    - Remove constructor parameters: `cache`, `embedding_model`, `embedding_dimensions`, `strategy`, `use_bm25_search`
    - Remove module-level helpers: `_cache_key`, `cosine_similarity`
    - Update constructor to accept only: `client`, `cost_tracker`, `use_auto_tagging`, `auto_tagger`
    - Add `_role_similarity_cache: dict[tuple[str, str], float]` to constructor
    - _Requirements: 1.3, 1.4, 1.5, 1.6, 1.7, 6.1, 6.2_

  - [x] 2.2 Simplify `compute_competence_similarity` to always use BM25+RRF
    - Remove any strategy/conditional branching
    - Always call `_bm25_rrf_similarity` directly (with optional auto-tagging expansion)
    - _Requirements: 1.1, 1.2_

  - [x] 2.3 Implement LLM-based `compute_role_similarity`
    - Replace embedding cosine similarity with LLM chat completion
    - Add `_ROLE_SIMILARITY_SYSTEM_PROMPT` constant
    - Implement symmetric cache key: `tuple(sorted((role_a_lower, role_b_lower)))`
    - Return 0.0 for bad/empty/None/"(unknown)" roles without LLM call
    - Return 1.0 for identical roles (case-insensitive) without LLM call
    - Parse JSON response, clamp score to [0.0, 1.0]
    - On any exception: return 0.0, do not cache failed results
    - _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.8_

  - [x] 2.4 Add `clear_role_cache` method
    - Clears `_role_similarity_cache` dict
    - _Requirements: 5.8_

  - [x] 2.5 Write property test for BM25 zero-score on disjoint tokens
    - **Property 1: BM25 Zero-Score für fehlenden Token-Overlap**
    - **Validates: Requirements 1.1**

  - [x] 2.6 Write property test for role similarity value range and identity
    - **Property 5: compute_role_similarity Wertebereich**
    - **Validates: Requirements 5.1**

  - [x] 2.7 Write property test for role similarity cache symmetry
    - **Property 6: Rollen-Similarity-Cache ist symmetrisch und idempotent**
    - **Validates: Requirements 5.8**

- [-] 3. Refactor VocabularyCache to use LLM-based role inference
  - [x] 3.1 Rewrite `VocabularyCache` class
    - Remove all embedding-related methods: `preload`, `_preload_role_vocab`, `_preload_competence_vocab`, `ensure_task_embedding`, `infer_competences`
    - Remove old `infer_primary_role` (embedding-based)
    - Remove properties: `roles`, `competences`
    - Remove constructor dependencies on `SimilarityEngine` and `EmbeddingCache`
    - New constructor accepts only: `db: DBClient`, `client: AzureOpenAIClient`
    - Remove module-level helpers: `role_vocab_cache_key`, `competence_vocab_cache_key`, `task_id_cache_key`, `VocabItem`
    - _Requirements: 4.3, 6.3, 6.4, 6.5_

  - [x] 3.2 Implement new LLM-based `infer_primary_role`
    - Accept `task_text: str` keyword argument (no more `task_embedding`)
    - Fetch all role names from DB via `self._db.get_all_role_names()`
    - Build LLM prompt with task text and available roles list
    - Parse JSON response: `{"role": "...", "confidence": 0.0-1.0}`
    - Validate returned role exists in DB role list
    - Return `None` on empty text, empty role list, or any exception
    - _Requirements: 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8_

  - [x] 3.3 Write property test for infer_primary_role output validity
    - **Property 3: infer_primary_role Ausgabe-Validität**
    - **Validates: Requirements 3.1, 3.4**

  - [x] 3.4 Write property test for graceful degradation on LLM failure
    - **Property 4: Graceful Degradation bei LLM-Fehler**
    - **Validates: Requirements 3.5, 5.6**

- [x] 4. Checkpoint - Ensure all tests pass
  - Ensure all tests pass, ask the user if questions arise.

- [-] 5. Update Matcher to unconditionally build BM25 index
  - [x] 5.1 Remove `use_bm25_search` conditional in `Matcher.match`
    - Remove `if self._sim.use_bm25_search` guard around BM25 index construction
    - Always build global BM25 index when `filtered` is non-empty
    - _Requirements: 2.6_

  - [x] 5.2 Write property test for unconditional BM25 index building
    - **Property 2: Matcher baut BM25-Index bedingungslos**
    - **Validates: Requirements 2.6**

- [x] 6. Update MCP Server to remove embedding preload and wire new components
  - [x] 6.1 Remove embedding preload infrastructure from `mcp_server.py`
    - Remove `_startup_preload_embeddings` function
    - Remove `_deferred_preload` function and `_preload_started` flag
    - Remove `_ensure_preloaded` function
    - Remove all `await _ensure_preloaded()` calls in tool handlers
    - Remove `EmbeddingCache` import and instantiation
    - _Requirements: 4.1, 4.2, 4.4, 4.5, 6.12_

  - [x] 6.2 Update component wiring in `build_server`
    - Construct `AzureOpenAIClient` with new simplified signature (no embedding params)
    - Construct `SimilarityEngine` with new signature (client, cost_tracker, use_auto_tagging, auto_tagger)
    - Construct `VocabularyCache` with new signature (db, client)
    - _Requirements: 6.7_

  - [x] 6.3 Update `infer_primary_role` tool handler
    - Remove embedding generation step
    - Call `vocab_cache.infer_primary_role(task_text=text)` directly with task text
    - _Requirements: 3.9_

  - [x] 6.4 Remove or simplify `validate_task_requirements` tool if it depends on embeddings
    - Remove embedding-based `infer_competences` usage
    - Either remove the tool entirely or replace with a DB-field-only version
    - _Requirements: 6.5_

- [x] 7. Simplify config.py and config.toml
  - [x] 7.1 Remove embedding-related dataclasses and fields from `config.py`
    - Delete `EmbeddingCacheConfig` dataclass
    - Delete `InferenceConfig` dataclass
    - Remove from `AzureOpenAIConfig`: `embedding_deployment`, `embedding_batch_size`
    - Remove from `SimilarityConfig`: `embedding_model`, `embedding_dimensions`, `strategy`, `use_bm25_search`
    - Remove `inference` field from `MatchingConfig`
    - Remove `embedding_cache` field from `AppConfig`
    - _Requirements: 2.1, 2.2, 6.8, 6.9, 6.10_

  - [x] 7.2 Update `load_config` / `_parse_azure_openai` validation
    - Remove `AZURE_OPENAI_EMBEDDING_API_KEY` environment variable check
    - Make `AZURE_OPENAI_LLM_API_KEY` always required
    - Make `chat_deployment` required
    - Remove `embedding_dimensions == 3072` validation
    - Remove `strategy` validation
    - Remove `[matching.inference]` parsing
    - Remove `[embedding_cache]` parsing
    - _Requirements: 2.3, 2.4_

  - [x] 7.3 Update `config.toml` and `config.toml.example`
    - Remove `[embedding_cache]` section
    - Remove from `[matching.similarity]`: `embedding_model`, `embedding_dimensions`, `strategy`, `use_bm25_search`
    - Remove from `[azure_openai]`: `embedding_deployment`, `embedding_batch_size`
    - Remove `[matching.inference]` section
    - _Requirements: 6.11_

- [x] 8. Delete EmbeddingCache module and related tests
  - [x] 8.1 Delete `src/teamlandkarte_mcp/cache/embedding_cache.py`
    - _Requirements: 6.6_

  - [x] 8.2 Remove or update tests that reference embedding functionality
    - Delete tests for `EmbeddingCache`
    - Update tests for `SimilarityEngine` to use new constructor
    - Update tests for `VocabularyCache` to use new constructor
    - Update tests for `AzureOpenAIClient` to use new constructor
    - Remove any test fixtures that create embedding mocks
    - _Requirements: 6.6_

- [x] 9. Final checkpoint - Ensure all tests pass
  - Ensure all tests pass, ask the user if questions arise.

## Notes

- Tasks marked with `*` are optional and can be skipped for faster MVP
- Each task references specific requirements for traceability
- Checkpoints ensure incremental validation
- Property tests validate universal correctness properties from the design document
- Order ensures no broken intermediate states: new LLM methods added before old embedding paths removed