Migrate all repos into monorepo context folders
Bahn: aisupport, Analyse-O2C-C2S, awesome-bahn-mcp-servers, beam-mcp,
Confluence_Bot, db-planet-mcp-server, O2C-Harness, project-audit,
Projekt-KIQ-HP, teamlandkarte-mcp
Dhive: Jury-Voting
Privat: CV, NoteGraph (NOTE: NoteGraph needs complete redo after consolidation)
Shared: AI-Orchestrator, OrgMyLife, power_skills_and_more
Shared/references: symphony (read-only)
Bahn repos remain available as independent remotes - this monorepo
pulls them in via subtree, the originals are untouched.
This commit is contained in:
@@ -0,0 +1 @@
|
||||
This folder intentionally left blank. (OpenSpec scaffolding will add proposal/tasks/design.)
|
||||
@@ -0,0 +1,84 @@
|
||||
# Design: Accelerate Embedding Similarity
|
||||
|
||||
## Context
|
||||
|
||||
The server uses Azure OpenAI embeddings for competence and role similarity. The matching pipeline can be slow because embeddings are currently requested on-demand and repeatedly from inner loops.
|
||||
|
||||
The system already has a persistent SQLite embedding cache (`EmbeddingCache`). However, without bulk prefetch and without a run-scoped memoization layer, the similarity engine still:
|
||||
|
||||
- repeatedly normalizes and hashes text keys
|
||||
- performs many repeated SQLite reads
|
||||
- performs sequential Azure calls for cache misses
|
||||
|
||||
## Goals
|
||||
|
||||
- Bulk prefetch embeddings for a matching run.
|
||||
- Maintain deterministic behavior and strict error semantics.
|
||||
- Preserve existing similarity contracts.
|
||||
|
||||
## Architecture Changes
|
||||
|
||||
### Components impacted
|
||||
|
||||
- `SimilarityEngine` (`src/teamlandkarte_mcp/matching/similarity.py`)
|
||||
- Add instance variable for in-memory cache (lives with the engine; not created/cleared per invocation)
|
||||
- Add public method `prefetch_embeddings(required_texts, candidate_texts, required_roles, candidate_roles) -> dict[str, list[float]]`
|
||||
- `AzureOpenAIClient` (`src/teamlandkarte_mcp/azure/openai_client.py`)
|
||||
- Modify `get_embeddings_batch()` to use true batch API with `input=[...]`
|
||||
- `CostTracker` (`src/teamlandkarte_mcp/azure/cost_tracker.py`)
|
||||
- May need adjustment to log batch requests (log once per chunk with count)
|
||||
- `EmbeddingCache` (no schema changes expected)
|
||||
|
||||
### Data flow (proposed)
|
||||
|
||||
1. **Collect** all texts that will be embedded in the current run.
|
||||
- This is a **single global collection pass** over the inputs and **all free capacities** (roles + competences), not a per-person loop that embeds incrementally.
|
||||
2. **Normalize + deduplicate** using existing `_normalize_text()` logic.
|
||||
- Skip empty/whitespace-only texts
|
||||
- Emit a **Python logger warning** if all candidate competences normalize to empty
|
||||
3. **Resolve embeddings** via:
|
||||
- in-memory cache (instance variable in `SimilarityEngine`)
|
||||
- SQLite cache (`EmbeddingCache`)
|
||||
- Azure embeddings API (**batch only the cache-missing texts**, **chunked** by batch size)
|
||||
4. **Compute** cosine similarities locally using the prefetched mapping.
|
||||
|
||||
### Normalization and keys
|
||||
|
||||
Current cache key behavior:
|
||||
|
||||
- normalization: trim + collapse whitespace
|
||||
- key: SHA256 of `model|dims|normalized(text).lower()`
|
||||
|
||||
This proposal keeps the same normalization and key derivation to avoid invalidating the on-disk cache.
|
||||
|
||||
## Batch embeddings API
|
||||
|
||||
The OpenAI/Azure embeddings API supports embedding multiple inputs per request (`input=[...]`).
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- preserve stable mapping from input texts → returned embedding vectors
|
||||
- rely on `index` field in API response if available
|
||||
- otherwise assume stable ordering
|
||||
- chunk large requests to avoid request size limits (configurable batch size, default 128)
|
||||
- strict failure semantics:
|
||||
- entire matching operation fails immediately on error
|
||||
- log the failing **chunk input list** via Python logging to aid diagnosis
|
||||
- cost tracking: log once per batch chunk with a count of embeddings requested
|
||||
|
||||
## Trade-offs
|
||||
|
||||
- Batch calls reduce network requests dramatically but may increase blast radius: one failing request could affect multiple inputs.
|
||||
- Mitigation: keep retry logic at the chunk level.
|
||||
- Prefetch requires holding larger embedding dictionaries in memory.
|
||||
- Mitigation: scope to a single invocation, and only store vectors required for that run.
|
||||
|
||||
## Test Strategy
|
||||
|
||||
- Unit test prefetch/dedup: ensure the Azure client is called once per unique normalized text (only for cache misses).
|
||||
- Unit test cache layering: ensure hits are served from in-memory cache (first), then SQLite, and do not call Azure.
|
||||
- Unit test chunking: ensure multiple Azure calls are made when `len(cache_missing_texts) > batch_size`.
|
||||
- Unit test empty input handling: verify warning when all candidate competences normalize to empty; verify prefetch skips empty texts.
|
||||
- Unit test error identification: verify that batch failures report which specific text(s) caused the error.
|
||||
- Integration test (manual/gated): validate real Azure batch API behavior with live credentials.
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
# Change Proposal: Accelerate Embedding Prefetch and Similarity Computation
|
||||
|
||||
- **Change ID**: `accelerate-embedding-similarity`
|
||||
- **Status**: Proposed
|
||||
- **Target**: `teamlandkarte-mcp`
|
||||
- **Author**: Thomas Handke
|
||||
- **Date**: 2026-02-14
|
||||
|
||||
## Summary
|
||||
|
||||
Speed up `find_matching_capacities(...)` by reducing embedding API calls and repeated cache lookups.
|
||||
|
||||
This change introduces three complementary optimizations:
|
||||
|
||||
1. **Dedup + bulk prefetch** of all embedding texts needed in a matching run (required competences, candidate competences, and roles).
|
||||
2. A **per-run in-memory cache** layered on top of the existing SQLite embedding cache to avoid repeated SQLite reads within a single run.
|
||||
3. Support for **true Azure Embeddings batch requests** (single request with multiple inputs), including chunking, while preserving deterministic error semantics.
|
||||
|
||||
## Motivation
|
||||
|
||||
The current implementation calls embeddings repeatedly inside inner loops. While a persistent SQLite cache exists, repeated calls still incur overhead:
|
||||
|
||||
- many redundant `cache.get(...)` reads
|
||||
- repeated normalization/keying for identical strings
|
||||
- unnecessary Azure round-trips when multiple missing terms are encountered and requests are performed sequentially
|
||||
|
||||
The result is that `find_matching_capacities(...)` can be slow, and while the second run is faster (due to cache hits), it can still take noticeable time.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Ensure each unique text is embedded at most once per run (network and cache).
|
||||
2. Prefetch embeddings **before** similarity computation so inner loops only perform vector math.
|
||||
3. Enable Azure embeddings API batch calls to reduce network round-trips.
|
||||
4. Preserve current similarity output contracts and scoring behavior.
|
||||
5. Maintain explicit, deterministic failure semantics (no heuristics).
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Changing match scoring weights or category thresholds.
|
||||
- Changing the similarity strategy semantics ("per_skill" vs "aggregate").
|
||||
- Changing the embedding model or dimensionality.
|
||||
- Introducing new external dependencies unless necessary.
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### 1) Bulk prefetch & dedup in `SimilarityEngine`
|
||||
|
||||
#### Current behavior (problem)
|
||||
|
||||
`SimilarityEngine` currently embeds terms on demand via `_embed(...)`. For large candidate lists, this forces a high number of calls to `_embed`, and repeated SQLite reads.
|
||||
|
||||
#### New behavior
|
||||
|
||||
Add a prefetch step that:
|
||||
|
||||
- collects all texts required to compute similarity for a run (competences **and roles**)
|
||||
- **Important:** this collection is performed **across the entire candidate set** (i.e., roles and competences of **all free capacities at once**), not person-by-person.
|
||||
- normalizes for cache key stability
|
||||
- deduplicates texts
|
||||
- fetches all embeddings from caches/API in bulk
|
||||
- returns a mapping from normalized text → embedding vector
|
||||
|
||||
Then similarity computation uses the prefetched mapping (pure Python vector math).
|
||||
|
||||
### 2) Per-run in-memory cache
|
||||
|
||||
Add a lightweight in-memory dict for the duration of a single matching invocation.
|
||||
|
||||
- key: `(model, dimensions, normalized_text)` or reuse the computed SHA256 cache key
|
||||
- value: `list[float]`
|
||||
|
||||
This prevents repeated SQLite lookups for frequently used items (common competences, repeated role strings, etc.).
|
||||
|
||||
### 3) True Azure embeddings batch calls
|
||||
|
||||
Change `AzureOpenAIClient.get_embeddings_batch(...)` to use the Azure embeddings API with `input=[...]` to embed multiple texts in a single request.
|
||||
|
||||
Key details:
|
||||
|
||||
- supports chunking with a **configurable** batch size
|
||||
- preserves stable output ordering and deterministic error semantics
|
||||
- continues to log costs via `CostTracker`
|
||||
|
||||
If any item fails, the batch call should raise `AzureAPIError` (no fallback).
|
||||
|
||||
## Decisions (resolved)
|
||||
|
||||
- **Scope**: bulk prefetch includes **competences and roles**.
|
||||
- **Batching**: Azure embeddings requests are made in chunks with a **configurable** batch size.
|
||||
- **Normalization**: keep the current text normalization and cache-key behavior.
|
||||
- **Concurrency**: process chunks **sequentially** (no parallel calls).
|
||||
- **In-memory cache lifecycle**: implemented as a new instance variable in `SimilarityEngine` and **not** created/cleaned per matching invocation.
|
||||
- **Prefetch API**: new public method `prefetch_embeddings(required_texts, candidate_texts, required_roles, candidate_roles)` that returns a mapping from normalized text → embedding vector.
|
||||
- **Azure batch response ordering**: rely on the `index` field in the response if available, otherwise assume stable ordering.
|
||||
- **Error handling**: entire matching operation fails immediately; attempt to identify and report which specific text(s) caused the failure.
|
||||
- **Cost tracking**: log once per batch chunk with a count.
|
||||
- **Empty inputs**:
|
||||
- emit a **Python logger warning** when all candidate competences normalize to empty strings
|
||||
- prefetch skips empty/whitespace-only texts
|
||||
- **Cache miss batching**: batch-request only the missing embeddings (after resolving cache hits from SQLite).
|
||||
|
||||
## Configuration
|
||||
|
||||
Introduce (or reuse if already present) one configuration knob:
|
||||
|
||||
- `azure_openai.embedding_batch_size` (integer)
|
||||
- default: `128` (conservative)
|
||||
- no enforced maximum value
|
||||
- used to chunk `input=[...]` for Azure embeddings batch requests
|
||||
- if not specified in `config.toml`, the default value is used
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A single `find_matching_capacities(...)` run should perform at most one embedding generation per unique term (after cache resolution).
|
||||
- Only cache-missing embeddings are batch-requested from Azure (cache hits are not re-requested).
|
||||
- For warmed cache runs, no Azure embedding calls should occur.
|
||||
- Unit tests cover dedup/prefetch, cache layering, and batch request behavior.
|
||||
- Integration tests (manual/gated) verify real Azure batch API behavior.
|
||||
|
||||
## Rollout / Compatibility
|
||||
|
||||
- No configuration changes are strictly required; the baseline refactor should work with existing `config.toml`.
|
||||
- If new config knobs are added (e.g., batch size), defaults should preserve behavior.
|
||||
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# Capability: Embedding Prefetch & Batch Similarity
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Bulk prefetch and deduplicate embedding inputs
|
||||
|
||||
The server MUST collect all texts that require embeddings for a matching run, normalize them, deduplicate them, and prefetch their embeddings before computing similarity scores.
|
||||
|
||||
#### Scenario: Matching run with repeated competences
|
||||
|
||||
- Given a matching run where the required competences contain duplicates (e.g. "Python", "Python")
|
||||
- And a candidate list contains overlapping competences (e.g. multiple candidates list "Python")
|
||||
- When the similarity computation starts
|
||||
- Then the embedding for "Python" is requested at most once from the Azure embeddings API
|
||||
- And all uses of "Python" reuse the same embedding vector for similarity computation
|
||||
|
||||
### Requirement: Compute similarity using prefetched vectors (no inner-loop embedding calls)
|
||||
|
||||
The similarity computation MUST use prefetched embedding vectors and MUST NOT request embeddings from inside per-competence or per-candidate inner loops.
|
||||
|
||||
#### Scenario: Per-skill similarity on large candidate set
|
||||
|
||||
- Given required competences and a large candidate competence list
|
||||
- When `per_skill` similarity is computed
|
||||
- Then all embeddings are resolved prior to the nested loops
|
||||
- And the nested loops perform only cosine similarity calculations
|
||||
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Batch embedding API support in Azure client
|
||||
|
||||
`AzureOpenAIClient.get_embeddings_batch(...)` MUST support true request-level batching using the Azure embeddings API (`input=[...]`) and MUST preserve input ordering.
|
||||
|
||||
#### Scenario: Batch embedding input list
|
||||
|
||||
- Given a list of N texts to embed
|
||||
- When `get_embeddings_batch(texts)` is called
|
||||
- Then the client makes as few Azure embedding requests as possible (subject to chunking)
|
||||
- And the returned list of embeddings aligns with `texts` order
|
||||
|
||||
### Requirement: Run-scoped in-memory embedding cache
|
||||
|
||||
The server MUST maintain a run-scoped in-memory cache to prevent repeated SQLite cache reads for identical texts during a single matching operation.
|
||||
|
||||
#### Scenario: Frequent repeated cache hits within one run
|
||||
|
||||
- Given cached embeddings are present on disk
|
||||
- And a matching run needs the same text embedding multiple times
|
||||
- When the similarity engine resolves embeddings
|
||||
- Then repeated lookups for the same text are served from an in-memory cache after the first lookup
|
||||
- And no additional SQLite reads are required for that text during the run
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Sequential batch behavior requirement
|
||||
|
||||
The previous constraint that `get_embeddings_batch` must call the embeddings API sequentially per text is removed.
|
||||
|
||||
#### Scenario: Legacy sequential constraint
|
||||
|
||||
- Given earlier requirements constrained `get_embeddings_batch` to per-text calls
|
||||
- When the system is upgraded for performance
|
||||
- Then this constraint no longer applies
|
||||
- And request-level batching is allowed
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# Tasks: Accelerate Embedding Similarity
|
||||
|
||||
> Change: `accelerate-embedding-similarity`
|
||||
>
|
||||
> Status: **Approved / Implemented**
|
||||
|
||||
## Overview
|
||||
|
||||
Reduce runtime of `find_matching_capacities(...)` by implementing:
|
||||
|
||||
- embedding text deduplication + **global** bulk prefetch (roles + competences for **all free capacities at once**, not person-by-person)
|
||||
- in-memory embedding memoization
|
||||
- true Azure embeddings batch requests with chunking
|
||||
|
||||
## Task Breakdown
|
||||
|
||||
### Phase 1: Design & Validation Plan
|
||||
|
||||
- [x] 1.1 Identify hot paths during matching (where `_embed(...)` is called most).
|
||||
- [x] 1.2 Confirm normalization + cache key rules are unchanged and documented.
|
||||
- [x] 1.3 Define expected call-count behavior (upper bounds):
|
||||
- per run: max 1 Azure embedding per unique normalized text (subject to chunking, after cache resolution)
|
||||
- per run: max 1 SQLite read per unique normalized text (after in-memory cache check)
|
||||
|
||||
### Phase 2: Global Bulk Prefetch in `SimilarityEngine`
|
||||
|
||||
- [x] 2.1 Add a new instance variable `_run_cache: dict[str, list[float]]` to `SimilarityEngine` (in-memory cache; **not** created/cleared per matching invocation).
|
||||
- [x] 2.2 Implement a public method `prefetch_embeddings(required_texts: list[str], candidate_texts: list[str], required_roles: list[str], candidate_roles: list[str]) -> dict[str, list[float]]`:
|
||||
- collect all input texts (required competences, **required roles**, all candidate competences, **all candidate roles**)
|
||||
- normalize + deduplicate using existing `_normalize_text()` logic
|
||||
- skip empty/whitespace-only texts
|
||||
- emit a **Python logger warning** if all candidate competences normalize to empty
|
||||
- [x] 2.3 Implement prefetch resolution logic:
|
||||
1) check in-memory cache (`_run_cache`)
|
||||
2) check SQLite embedding cache
|
||||
3) batch-request only cache-missing texts from Azure API (chunked by batch size)
|
||||
- [x] 2.4 Refactor `_per_skill_similarity` to use prefetched embeddings only (no embedding calls inside inner loops).
|
||||
- [x] 2.5 Refactor `_aggregate_similarity` to use prefetched embeddings only.
|
||||
- [x] 2.6 Ensure output format is unchanged and remains deterministic.
|
||||
|
||||
### Phase 3: In-memory Run Cache
|
||||
|
||||
- [x] 3.1 Verify `_run_cache` is an instance-level cache that does not require per-invocation clearing.
|
||||
- [x] 3.2 Add tests ensuring repeated embedding lookups for the same text:
|
||||
- are served from `_run_cache` after first lookup
|
||||
- do not call SQLite/API multiple times for the same text
|
||||
|
||||
### Phase 4: Azure Batch Embeddings Support
|
||||
|
||||
- [x] 4.1 Modify `AzureOpenAIClient.get_embeddings_batch()` to use Azure embeddings API with `input=[...]`.
|
||||
- [x] 4.2 Implement chunking using `azure_openai.embedding_batch_size` (default: 128, configurable, no enforced maximum).
|
||||
- [x] 4.3 Preserve ordering using `index` field in API response if available, otherwise assume stable ordering.
|
||||
- [x] 4.4 Ensure strict error semantics:
|
||||
- entire matching operation fails immediately on error
|
||||
- log the failing **chunk input list** via Python logging
|
||||
- [x] 4.5 Update `CostTracker` to log batch requests: log once per chunk with count of embeddings requested.
|
||||
|
||||
### Phase 5: Configuration
|
||||
|
||||
- [x] 5.1 Extend config model to include `azure_openai.embedding_batch_size` (integer, default: 128, no enforced maximum).
|
||||
- [x] 5.2 Update `config.toml.example` with the new setting and comments.
|
||||
- [x] 5.3 Ensure fallback: if `embedding_batch_size` is not specified in `config.toml`, use the default value (128).
|
||||
|
||||
### Phase 6: Tests
|
||||
|
||||
- [x] 6.1 Unit tests: global prefetch/dedup
|
||||
- verify dedup across required + all candidates (incl. roles)
|
||||
- verify embedding client is called once per unique normalized text (only for cache misses)
|
||||
- verify empty/whitespace-only texts are skipped
|
||||
- verify a Python logger warning is emitted when all candidate competences normalize to empty
|
||||
- [x] 6.2 Unit tests: cache layering
|
||||
- verify in-memory `_run_cache` prevents repeated SQLite reads within a run
|
||||
- verify SQLite hits prevent Azure calls
|
||||
- [x] 6.3 Unit tests: Azure batch request behavior (mocked)
|
||||
- verifies batch call for N texts when `N <= batch_size`
|
||||
- verifies chunking when `N > batch_size` (multiple batch calls)
|
||||
- verifies output ordering is stable (using `index` field or assuming stable order)
|
||||
- verifies strict error propagation
|
||||
- verifies the failing chunk input list is logged
|
||||
- verifies cost tracking logs once per chunk with count
|
||||
- [x] 6.4 Integration test (manual/gated only): real Azure batch embeddings
|
||||
- confirm request uses batch `input=[...]` semantics
|
||||
- ensure behavior matches expected semantics for a representative set of texts
|
||||
|
||||
### Phase 7: Documentation + Assistant Prompt Updates
|
||||
|
||||
- [x] 7.1 Update `README.md` to describe the optimization at a high level and mention the new batch size knob.
|
||||
- [x] 7.2 Update `docs/assistant_system_prompt.md` to reflect:
|
||||
- embeddings are prefetched globally per run
|
||||
- batching/chunking is used and configurable
|
||||
- failure semantics remain strict (no heuristics)
|
||||
- [x] 7.3 Update any Azure setup docs if needed (e.g., `docs/azure_openai_setup.md`) to mention embedding batch behavior and tuning.
|
||||
|
||||
### Phase 8: Quality Gates / Release Hygiene
|
||||
|
||||
- [x] 8.1 Run unit tests: `uv run pytest -m "not integration" -q`
|
||||
- [x] 8.2 Run Ruff and mypy via `uv`.
|
||||
- [ ] 8.3 (Manual/gated only) Run integration tests (Azure) to validate real batch calls.
|
||||
- [x] 8.4 Update change docs status/checklists and ensure OpenSpec validates (`openspec validate accelerate-embedding-similarity --strict`).
|
||||
|
||||
Reference in New Issue
Block a user