Migrate all repos into monorepo context folders
Bahn: aisupport, Analyse-O2C-C2S, awesome-bahn-mcp-servers, beam-mcp,
Confluence_Bot, db-planet-mcp-server, O2C-Harness, project-audit,
Projekt-KIQ-HP, teamlandkarte-mcp
Dhive: Jury-Voting
Privat: CV, NoteGraph (NOTE: NoteGraph needs complete redo after consolidation)
Shared: AI-Orchestrator, OrgMyLife, power_skills_and_more
Shared/references: symphony (read-only)
Bahn repos remain available as independent remotes - this monorepo
pulls them in via subtree, the originals are untouched.
This commit is contained in:
@@ -0,0 +1,84 @@
|
||||
# Design: Accelerate Embedding Similarity
|
||||
|
||||
## Context
|
||||
|
||||
The server uses Azure OpenAI embeddings for competence and role similarity. The matching pipeline can be slow because embeddings are currently requested on-demand and repeatedly from inner loops.
|
||||
|
||||
The system already has a persistent SQLite embedding cache (`EmbeddingCache`). However, without bulk prefetch and without a run-scoped memoization layer, the similarity engine still:
|
||||
|
||||
- repeatedly normalizes and hashes text keys
|
||||
- performs many repeated SQLite reads
|
||||
- performs sequential Azure calls for cache misses
|
||||
|
||||
## Goals
|
||||
|
||||
- Bulk prefetch embeddings for a matching run.
|
||||
- Maintain deterministic behavior and strict error semantics.
|
||||
- Preserve existing similarity contracts.
|
||||
|
||||
## Architecture Changes
|
||||
|
||||
### Components impacted
|
||||
|
||||
- `SimilarityEngine` (`src/teamlandkarte_mcp/matching/similarity.py`)
|
||||
- Add instance variable for in-memory cache (lives with the engine; not created/cleared per invocation)
|
||||
- Add public method `prefetch_embeddings(required_texts, candidate_texts, required_roles, candidate_roles) -> dict[str, list[float]]`
|
||||
- `AzureOpenAIClient` (`src/teamlandkarte_mcp/azure/openai_client.py`)
|
||||
- Modify `get_embeddings_batch()` to use true batch API with `input=[...]`
|
||||
- `CostTracker` (`src/teamlandkarte_mcp/azure/cost_tracker.py`)
|
||||
- May need adjustment to log batch requests (log once per chunk with count)
|
||||
- `EmbeddingCache` (no schema changes expected)
|
||||
|
||||
### Data flow (proposed)
|
||||
|
||||
1. **Collect** all texts that will be embedded in the current run.
|
||||
- This is a **single global collection pass** over the inputs and **all free capacities** (roles + competences), not a per-person loop that embeds incrementally.
|
||||
2. **Normalize + deduplicate** using existing `_normalize_text()` logic.
|
||||
- Skip empty/whitespace-only texts
|
||||
- Emit a **Python logger warning** if all candidate competences normalize to empty
|
||||
3. **Resolve embeddings** via:
|
||||
- in-memory cache (instance variable in `SimilarityEngine`)
|
||||
- SQLite cache (`EmbeddingCache`)
|
||||
- Azure embeddings API (**batch only the cache-missing texts**, **chunked** by batch size)
|
||||
4. **Compute** cosine similarities locally using the prefetched mapping.
|
||||
|
||||
### Normalization and keys
|
||||
|
||||
Current cache key behavior:
|
||||
|
||||
- normalization: trim + collapse whitespace
|
||||
- key: SHA256 of `model|dims|normalized(text).lower()`
|
||||
|
||||
This proposal keeps the same normalization and key derivation to avoid invalidating the on-disk cache.
|
||||
|
||||
## Batch embeddings API
|
||||
|
||||
The OpenAI/Azure embeddings API supports embedding multiple inputs per request (`input=[...]`).
|
||||
|
||||
Implementation requirements:
|
||||
|
||||
- preserve stable mapping from input texts → returned embedding vectors
|
||||
- rely on `index` field in API response if available
|
||||
- otherwise assume stable ordering
|
||||
- chunk large requests to avoid request size limits (configurable batch size, default 128)
|
||||
- strict failure semantics:
|
||||
- entire matching operation fails immediately on error
|
||||
- log the failing **chunk input list** via Python logging to aid diagnosis
|
||||
- cost tracking: log once per batch chunk with a count of embeddings requested
|
||||
|
||||
## Trade-offs
|
||||
|
||||
- Batch calls reduce network requests dramatically but may increase blast radius: one failing request could affect multiple inputs.
|
||||
- Mitigation: keep retry logic at the chunk level.
|
||||
- Prefetch requires holding larger embedding dictionaries in memory.
|
||||
- Mitigation: scope to a single invocation, and only store vectors required for that run.
|
||||
|
||||
## Test Strategy
|
||||
|
||||
- Unit test prefetch/dedup: ensure the Azure client is called once per unique normalized text (only for cache misses).
|
||||
- Unit test cache layering: ensure hits are served from in-memory cache (first), then SQLite, and do not call Azure.
|
||||
- Unit test chunking: ensure multiple Azure calls are made when `len(cache_missing_texts) > batch_size`.
|
||||
- Unit test empty input handling: verify warning when all candidate competences normalize to empty; verify prefetch skips empty texts.
|
||||
- Unit test error identification: verify that batch failures report which specific text(s) caused the error.
|
||||
- Integration test (manual/gated): validate real Azure batch API behavior with live credentials.
|
||||
|
||||
Reference in New Issue
Block a user