Migrate all repos into monorepo context folders

Bahn: aisupport, Analyse-O2C-C2S, awesome-bahn-mcp-servers, beam-mcp,
      Confluence_Bot, db-planet-mcp-server, O2C-Harness, project-audit,
      Projekt-KIQ-HP, teamlandkarte-mcp
Dhive: Jury-Voting
Privat: CV, NoteGraph (NOTE: NoteGraph needs complete redo after consolidation)
Shared: AI-Orchestrator, OrgMyLife, power_skills_and_more
Shared/references: symphony (read-only)

Bahn repos remain available as independent remotes - this monorepo
pulls them in via subtree, the originals are untouched.

bahn/teamlandkarte-mcp/openspec/changes/replace-heuristics-with-azure-openai/tasks.md

@@ -0,0 +1,359 @@
+# Tasks: Replace Heuristics with Azure OpenAI
+
+> Change: `replace-heuristics-with-azure-openai`
+> 
+> Status: **Approved / Implemented**
+
+## Overview
+
+Replace legacy client-side sampling/heuristic logic with direct Azure OpenAI integration:
+- Embeddings (`text-embedding-3-large`) for competence/role similarity
+- LLM (`gpt-4.1`) for requirements extraction and validation
+- Persistent SQLite embedding cache
+- Two configurable similarity strategies
+
+# Removed: migration note about database.toml rename (now a completed internal detail).
+
+## Estimated Effort
+
+**Total: 28-35 hours** (1 week full-time or 2 weeks part-time)
+
+## Task Breakdown
+
+### Phase 1: Configuration & Infrastructure (3-4 hours)
+
+- [x] 1.1 Rename `database.toml` → `config.toml`
+- [x] 1.2 Rename `database.toml.example` → `config.toml.example`
+- [x] 1.3 Update all file references in code (`README.md`, `docs/`, `src/config.py`, etc.)
+- [x] 1.4 Extend `src/config.py`: Add `AzureOpenAIConfig` model
+  - `endpoint`, `api_version_embeddings`, `api_version_llm`
+  - `embeddings.model`, `embeddings.deployment` (deployment must equal model name)
+  - `llm.model`, `llm.deployment` (deployment must equal model name), `llm.temperature`, `llm.max_tokens`
+- [x] 1.5 Extend `src/config.py`: Add `EmbeddingCacheConfig` model
+  - `enabled`, `db_path`, `ttl_days`
+- [x] 1.6 Extend `src/config.py`: Add `matching.similarity.strategy` field (`"per_skill"` or `"aggregate"`)
+- [x] 1.7 Update `config.toml.example` with new `[azure_openai]`, `[embedding_cache]`, `[matching.similarity]` sections
+- [x] 1.8 Load `AZURE_OPENAI_EMBEDDING_API_KEY` from environment in `load_config()`
+- [x] 1.9 Load `AZURE_OPENAI_LLM_API_KEY` from environment in `load_config()`
+- [x] 1.10 Add `openai` package to `pyproject.toml` dependencies
+- [x] 1.11 Run `uv sync` to install dependencies
+- [x] 1.12 Add `config.toml` to `.gitignore` (if not already there)
+- [x] 1.13 Configure pytest integration marker in `pytest.ini` (if not already present):
+  - `integration: marks tests that call real Azure OpenAI API (deselect with '-m "not integration"')`
+
+### Phase 2: Embedding Cache (2-3 hours)
+
+- [x] 2.1 Create `src/teamlandkarte_mcp/cache/embedding_cache.py`
+- [x] 2.2 Implement `EmbeddingCache` class:
+  - `__init__(db_path: str, ttl_days: int)`
+  - Create SQLite database and schema if not exists:
+    ```sql
+    CREATE TABLE IF NOT EXISTS embeddings (
+        id INTEGER PRIMARY KEY AUTOINCREMENT,
+        text TEXT UNIQUE NOT NULL,
+        model TEXT NOT NULL,
+        embedding BLOB NOT NULL,
+        created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
+    )
+    CREATE INDEX IF NOT EXISTS idx_text_model ON embeddings(text, model)
+    ```
+- [x] 2.3 Implement `EmbeddingCache.get(text: str, model: str) -> Optional[list[float]]`
+  - Query by text + model
+  - Check if `created_at` is within TTL
+  - Deserialize BLOB to `list[float]`
+  - Return `None` if missing or expired
+- [x] 2.4 Implement `EmbeddingCache.put(text: str, model: str, embedding: list[float]) -> None`
+  - Serialize embedding as UTF-8 bytes of JSON: `json.dumps(embedding).encode("utf-8")`
+  - INSERT OR REPLACE into database
+- [x] 2.5 Implement `EmbeddingCache.cleanup_expired() -> int`
+  - DELETE embeddings older than TTL
+  - Return count of deleted rows
+- [x] 2.6 Add graceful handling for concurrent access (SQLite locks)
+- [x] 2.7 Create `tests/test_embedding_cache.py`
+  - Test cache hit/miss
+  - Test TTL expiration
+  - Test cleanup
+  - Test concurrent access
+- [x] 2.8 Add `embeddings_cache.db` to `.gitignore`
+
+### Phase 3: Azure OpenAI Client (3-4 hours)
+
+- [x] 3.1 Create `src/teamlandkarte_mcp/azure/__init__.py`
+- [x] 3.2 Create `src/teamlandkarte_mcp/azure/openai_client.py`
+- [x] 3.3 Define `AzureAPIError(RuntimeError)` exception class
+- [x] 3.4 Implement `AzureOpenAIClient` class:
+  - `__init__(config: AzureOpenAIConfig, embedding_cache: Optional[EmbeddingCache])`
+  - Initialize two `AzureOpenAI` clients (embeddings + LLM) using `AzureKeyCredential`
+- [x] 3.5 Implement `AzureOpenAIClient.get_embedding(text: str) -> list[float]`
+  - Check cache first (if enabled)
+  - If cache miss: call Azure API
+  - Store in cache (if enabled)
+  - Raise `AzureAPIError` on failure (no fallback)
+- [x] 3.6 Implement `AzureOpenAIClient.get_embeddings_batch(texts: list[str]) -> list[list[float]]`
+  - Call `get_embedding()` for each text individually (per user requirement #3)
+  - Return list of embeddings in same order
+- [x] 3.7 Implement `AzureOpenAIClient.chat_completion(system: str, user: str, response_format: Optional[dict] = None) -> str`
+  - Call Azure OpenAI chat completions API
+  - Use `gpt-4.1` model with configured temperature/max_tokens
+  - Support `response_format={"type": "json_object"}` for structured JSON
+  - Implement retry logic (3 attempts, exponential backoff)
+  - Raise `AzureAPIError` on final failure (no fallback)
+- [x] 3.8 Add logging for API calls (cache hit/miss, request/response sizes)
+- [x] 3.9 Create `tests/test_azure_client.py`
+  - Mock Azure API with `unittest.mock`
+  - Test embedding retrieval with cache hit/miss
+  - Test LLM completion
+  - Test error handling and retries
+- [x] 3.10 Create `tests/test_azure_client_integration.py` (marked `@pytest.mark.integration`)
+  - Real API calls (requires valid credentials)
+  - Test embedding generation
+  - Test LLM structured JSON response
+
+### Phase 4: Similarity Engine (4-5 hours)
+
+- [x] 4.1 Create `src/teamlandkarte_mcp/matching/similarity.py`
+- [x] 4.2 Implement `cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float`
+  - Compute dot product and magnitudes
+  - Return cosine similarity in [0.0, 1.0]
+  - Handle zero vectors gracefully
+- [x] 4.3 Implement `SimilarityEngine` class:
+  - `__init__(client: AzureOpenAIClient, strategy: str = "per_skill")`
+  - Validate strategy is `"per_skill"` or `"aggregate"`
+- [x] 4.4 Implement `SimilarityEngine._per_skill_similarity(required: list[str], candidate: list[str]) -> dict[str, dict[str, Any]]`
+  - For each required skill:
+    - Get embedding
+    - Compare with all candidate skill embeddings
+    - Find best match (highest cosine similarity)
+    - Return `{required: {best_match: str|None, score: float, rationale: str}}`
+  - Rationale example: `"Cosine similarity: 0.92 (best match: 'React.js')"`
+- [x] 4.5 Implement `SimilarityEngine._aggregate_similarity(required: list[str], candidate: list[str]) -> dict[str, dict[str, Any]]`
+  - Compute average embedding of all required skills
+  - Compute average embedding of all candidate skills
+  - Compute cosine similarity between averages
+  - Return same format as `_per_skill_similarity` (each required skill gets same aggregate score)
+  - Rationale example: `"Aggregate embedding similarity: 0.85"`
+- [x] 4.6 Implement `SimilarityEngine.compute_competence_similarity(required: list[str], candidate: list[str]) -> dict[str, dict[str, Any]]`
+  - Route to `_per_skill_similarity()` or `_aggregate_similarity()` based on configured strategy
+  - Maintain same output format as current `TaskAnalyzer.semantic_competence_similarity()`
+  - Guarantee `rationale` is always present (even when best_match is None)
+  - Edge cases:
+    - `required == []` → return `{}`
+    - `candidate == []` → return per-required entry with `best_match=None`, `score=0.0`, and rationale
+- [x] 4.7 Implement `SimilarityEngine.compute_role_similarity(required_role: str, candidate_role: str) -> float`
+  - Edge cases:
+    - If either role is empty/None → return 0.0
+    - If either role is "(unknown)" (case-insensitive, trimmed) → return 0.0
+  - Otherwise compute cosine similarity of role embeddings
+- [x] 4.8 Create `tests/test_similarity.py`
+  - Test `cosine_similarity()` with known vectors
+  - Mock embeddings and test both strategies
+  - Test edge cases (empty lists, identical skills, no matches)
+- [x] 4.9 Create `tests/test_similarity_integration.py` (marked `@pytest.mark.integration`)
+  - Compare both strategies with real embeddings
+  - Validate scores are reasonable (e.g., "Python" vs "Python 3" → high similarity)
+
+### Phase 5: TaskAnalyzer Refactoring (3-4 hours)
+
+- [x] 5.1 Update `src/teamlandkarte_mcp/matching/task_analyzer.py`
+- [x] 5.2 Change `TaskAnalyzer.__init__` signature:
+  - Remove: `__init__(self, mcp: FastMCP)`
+  - Add: `__init__(self, azure_client: AzureOpenAIClient)`
+- [x] 5.3 **Delete** `_sample_json()` method completely
+- [x] 5.4 **Delete** `_heuristic_competences_from_text()` static method completely
+- [x] 5.5 Refactor `extract_ranked_roles(description: str, limit: int = 5) -> list[RankedRole]`:
+  - Remove all heuristic fallback code
+  - Build prompt for structured JSON response:
+    ```
+    system: "Extract a ranked list of possible roles from a task description. Return STRICT JSON: {roles:[{rank:int, role:str, rationale:str}]}"
+    user: "Limit: {limit}\n\nTask description:\n{description}"
+    ```
+  - Call `self._client.chat_completion(system, user, response_format={"type": "json_object"})`
+  - Parse JSON response and build `list[RankedRole]`
+  - Raise `AzureAPIError` on failure (no fallback)
+- [x] 5.6 Refactor `_extract_requirements_from_description(description: str) -> ExtractedRequirements`:
+  - Remove all heuristic fallback code
+  - Build prompt for structured JSON:
+    ```
+    system: "Extract structured requirements from a task description. Return STRICT JSON with keys: competences (list[str]), date_start (YYYY-MM-DD|null), date_end (YYYY-MM-DD|null), roles ([{rank, role, rationale}])."
+    user: "Task description:\n{description}"
+    ```
+  - Call `self._client.chat_completion(system, user, response_format={"type": "json_object"})`
+  - Parse JSON and build `ExtractedRequirements`
+  - Raise `AzureAPIError` on failure (no fallback)
+- [x] 5.7 Refactor `apply_requirement_update(current: Requirements, change_description: str) -> Requirements`:
+  - Remove all heuristic fallback code
+  - Build prompt with current requirements + change request
+  - Call `self._client.chat_completion(system, user, response_format={"type": "json_object"})`
+  - Parse JSON and build updated `Requirements`
+  - Raise `AzureAPIError` on failure (no fallback)
+- [x] 5.8 **Delete** `semantic_competence_similarity()` method completely (replaced by `SimilarityEngine`)
+- [x] 5.9 Update `ExtractedRequirements` dataclass docstring (no longer mentions heuristics)
+- [x] 5.10 Update `TaskAnalyzer` class docstring (remove FastMCP sampling references, mention Azure OpenAI)
+- [x] 5.11 Remove unused imports (`FastMCP`, `random`, `asyncio` if no longer needed)
+- [x] 5.12 Update all `tests/test_task_analyzer.py` (or similar):
+  - Mock `AzureOpenAIClient` instead of `FastMCP`
+  - Remove heuristic fallback tests
+  - Add tests for Azure API error propagation
+- [x] 5.13 Update `AnalysisError` semantics (or rename/remove if no longer needed):
+  - Previously: heuristic fallback masked missing sampling
+  - Now: Azure failures must surface as exceptions (no heuristics)
+
+### Phase 6: Matcher & Scorer Integration (2-3 hours)
+
+- [x] 6.1 Identify where `Matcher` or scoring logic calls `TaskAnalyzer.semantic_competence_similarity()`
+- [x] 6.2 Update `Matcher.__init__` to accept `SimilarityEngine` as parameter
+- [x] 6.3 Replace `analyzer.semantic_competence_similarity(required, candidate)` with `similarity_engine.compute_competence_similarity(required, candidate)`
+- [x] 6.4 Update role matching to use `similarity_engine.compute_role_similarity(required_role, candidate_role)`
+- [x] 6.5 Ensure scorer aggregation logic works with new similarity output format (should be identical)
+- [x] 6.6 Update `tests/test_scorer.py` (and/or add `tests/test_matcher.py` if introduced):
+  - Mock `SimilarityEngine` instead of `TaskAnalyzer.semantic_competence_similarity`
+  - Test that both similarity strategies produce valid scores
+- [x] 6.7 Update matcher/scorer tests to reflect new edge-case contracts:
+  - role similarity returns 0.0 for empty/"(unknown)" roles
+  - competence similarity returns 0.0 when candidate list is empty
+
+### Phase 7: Server Initialization (2 hours)
+
+- [x] 7.1 Update `build_server()` in `src/teamlandkarte_mcp/mcp_server.py`
+- [x] 7.2 Update server build to use top-level embedding cache config (not nested under azure):
+  - `cfg.embedding_cache.enabled/db_path/ttl_days`
+- [x] 7.3 Initialize `EmbeddingCache` (if enabled) using `cfg.embedding_cache`
+- [x] 7.4 Initialize `AzureOpenAIClient` using `cfg.azure_openai`
+- [x] 7.5 Initialize `SimilarityEngine` using `cfg.matching.similarity.strategy`
+- [x] 7.6 Update `TaskAnalyzer` instantiation
+- [x] 7.7 Update `Matcher` instantiation
+- [x] 7.8 Add startup logging (stderr):
+  - Log Azure OpenAI endpoint
+  - Log embedding cache status (enabled/disabled, path)
+  - Log similarity strategy
+- [x] 7.9 Add graceful error handling if Azure credentials missing:
+  - Log error to stderr
+  - Exit with clear message: "Azure OpenAI credentials not found in environment"
+
+### Phase 8: Cost Tracking & Monitoring (2 hours)
+
+- [x] 8.1 Create `src/teamlandkarte_mcp/azure/cost_tracker.py`
+- [x] 8.2 Implement `CostTracker` class:
+  - Class-level constants for pricing
+  - `log_embedding_request(text: str, cached: bool)`
+  - `log_llm_request(input_tokens: int, output_tokens: int)`
+  - `get_session_costs() -> dict` (return totals)
+- [x] 8.3 Integrate `CostTracker` into `AzureOpenAIClient`:
+  - Track embedding requests (cache hit/miss)
+  - Track LLM requests (token counts from API response)
+- [x] 8.4 Add periodic cost logging to stderr:
+  - Every 10 API calls or every 5 minutes
+  - Log: `"Azure OpenAI costs this session: embeddings=$X.XX, llm=$Y.YY, total=$Z.ZZ"`
+
+### Phase 9: Documentation (2-3 hours)
+
+- [x] 9.1 Update `README.md`:
+  - Add "Azure OpenAI Setup" section with prerequisites
+  - Document environment variables (`AZURE_OPENAI_EMBEDDING_API_KEY`, `AZURE_OPENAI_LLM_API_KEY`)
+  - Document `config.toml` Azure sections
+  - Add cost estimation table
+  - Add embedding cache explanation
+  - Add similarity strategy comparison
+- [x] 9.2 Add migration guide to `README.md`:
+  - Step 1: Rename `database.toml` → `config.toml`
+  - Step 2: Add Azure sections to config
+  - Step 3: Add Azure API keys to `.env`
+  - Step 4: Run `uv sync`
+  - Step 5: Start server (embedding cache will be created automatically)
+- [x] 9.3 Update `docs/troubleshooting.md`:
+  - Add section: "Azure OpenAI API Errors"
+  - Document common errors (rate limits, invalid credentials, network issues)
+  - Document `AzureAPIError` and how to debug
+- [x] 9.4 Update `config.toml.example`:
+  - Add comprehensive comments for all Azure sections
+  - Document both similarity strategies with examples
+  - Document embedding cache TTL trade-offs
+- [x] 9.5 Update OpenSpec architecture docs:
+  - `openspec/changes/add-capacity-matching-mcp-server/architecture.md`
+  - `openspec/changes/add-capacity-matching-mcp-server/design.md`
+  - Update diagrams to show Azure OpenAI dependency
+  - Update component descriptions
+- [x] 9.6 Create `docs/azure_openai_setup.md` (detailed setup guide):
+  - Prerequisites (Azure subscription, OpenAI resource)
+  - API key generation steps
+  - Cost monitoring in Azure portal
+  - Troubleshooting deployment/model availability
+
+### Phase 10: Testing & Validation (3-4 hours)
+
+- [x] 10.1 Run fast unit tests in CI mode: `pytest -m "not integration" -q`
+- [x] 10.2 Run integration tests with real Azure API: `pytest -m integration`
+- [x] 10.3 Test embedding cache behavior:
+  - Start server, run search → check cache DB created
+  - Restart server, run same search → verify cache hit
+  - Wait past TTL, run cleanup → verify expired entries removed
+- [x] 10.4 Test both similarity strategies:
+  - Set `matching.similarity.strategy = "per_skill"` → run search
+  - Set `matching.similarity.strategy = "aggregate"` → run search
+  - Compare results and validate scoring differences
+- [x] 10.5 Test cost tracking:
+  - Run several searches
+  - Check stderr for cost logs
+  - Validate cost estimates are reasonable
+- [x] 10.6 Test error handling:
+  - Comment out Azure API keys → verify graceful startup error
+  - Mock API failure → verify `AzureAPIError` propagates correctly
+  - Mock rate limit error → verify retry logic works
+- [x] 10.7 Performance benchmark:
+  - Cold cache: measure time for first search (100 candidates)
+  - Warm cache: measure time for repeated search
+  - Document speedup ratio
+- [x] 10.8 Manual validation in Cherry Studio:
+  - Browse tasks
+  - Extract requirements from complex task description
+  - Run matching workflow
+  - Validate similarity scores look reasonable
+  - Test guided capture with Azure LLM
+  - Test update_requirements
+
+### Phase 11: Cleanup & Final Review (2-3 hours)
+
+- [x] 11.1 Final sweep: ensure there are no remaining FastMCP sampling references
+- [x] 11.2 Final sweep: ensure no heuristic fallback paths remain
+- [x] 11.3 Tighten type hints, docstrings, and error semantics
+- [x] 11.4 Run `ruff check` (via `uv`) and fix all issues
+- [x] 11.5 Run `mypy` (via `uv`) and fix all issues
+- [x] 11.6 Final code review: validate all modules, remove stray prints/debug logs
+- [x] 11.7 Git commit review: ensure no secrets (config/.env/cache DB) are tracked
+- [x] 11.7a Cleanup: remove obsolete `scripts/trino_smoke_check.py`
+- [x] 11.7b Dependency hygiene: keep `trino` as a required runtime dependency
+- [ ] 11.8 Merge to main and tag release
+
+## Risk Mitigation
+
+**Risk**: Azure API downtime → System unusable  
+**Mitigation**: Document SLA expectations; consider adding a simple health check tool/endpoint and a clear user-facing error message (`AzureAPIError`).
+
+**Risk**: Unexpected high costs  
+**Mitigation**: Enable embedding cache (default); monitor costs weekly; set Azure budget alerts.
+
+**Risk**: Embedding cache grows too large  
+**Mitigation**: Run periodic cleanup; set reasonable TTL (default 30 days); ensure cache DB files stay ignored.
+
+**Risk**: Secrets accidentally committed (`config.toml`, `.env`, caches)  
+**Mitigation**: Keep `config.toml` and `.env` ignored; review `git status` before commit; run a secret scan locally (e.g. `git grep` for key patterns) before merging.
+
+**Risk**: Migration breaks existing deployments  
+**Mitigation**: Provide clear migration guide; test migration path manually; run integration tests against Azure in at least one real environment.
+
+## Success Criteria
+
+- [x] Unit tests pass (non-integration)
+- [ ] Integration tests pass (`pytest -m integration`) in an environment with valid Azure credentials
+- [ ] Server starts successfully with Azure credentials
+- [ ] Matching results show improved semantic understanding vs. heuristics
+- [ ] Embedding cache demonstrates significant performance improvement on repeated searches
+- [ ] Cost tracking shows accurate estimates
+- [ ] Documentation complete and clear
+- [ ] Repo clean: no secrets or local config files tracked (verify before merge)
+
+---
+
+**Total estimated effort**: 28-35 hours  
+**Priority**: Medium-High (significant quality improvement, but breaking change)  
+**Approval status**: ✅ Approved