Migrate all repos into monorepo context folders
Bahn: aisupport, Analyse-O2C-C2S, awesome-bahn-mcp-servers, beam-mcp,
Confluence_Bot, db-planet-mcp-server, O2C-Harness, project-audit,
Projekt-KIQ-HP, teamlandkarte-mcp
Dhive: Jury-Voting
Privat: CV, NoteGraph (NOTE: NoteGraph needs complete redo after consolidation)
Shared: AI-Orchestrator, OrgMyLife, power_skills_and_more
Shared/references: symphony (read-only)
Bahn repos remain available as independent remotes - this monorepo
pulls them in via subtree, the originals are untouched.
This commit is contained in:
@@ -0,0 +1,124 @@
|
||||
# Change Proposal: Accelerate Embedding Prefetch and Similarity Computation
|
||||
|
||||
- **Change ID**: `accelerate-embedding-similarity`
|
||||
- **Status**: Proposed
|
||||
- **Target**: `teamlandkarte-mcp`
|
||||
- **Author**: Thomas Handke
|
||||
- **Date**: 2026-02-14
|
||||
|
||||
## Summary
|
||||
|
||||
Speed up `find_matching_capacities(...)` by reducing embedding API calls and repeated cache lookups.
|
||||
|
||||
This change introduces three complementary optimizations:
|
||||
|
||||
1. **Dedup + bulk prefetch** of all embedding texts needed in a matching run (required competences, candidate competences, and roles).
|
||||
2. A **per-run in-memory cache** layered on top of the existing SQLite embedding cache to avoid repeated SQLite reads within a single run.
|
||||
3. Support for **true Azure Embeddings batch requests** (single request with multiple inputs), including chunking, while preserving deterministic error semantics.
|
||||
|
||||
## Motivation
|
||||
|
||||
The current implementation calls embeddings repeatedly inside inner loops. While a persistent SQLite cache exists, repeated calls still incur overhead:
|
||||
|
||||
- many redundant `cache.get(...)` reads
|
||||
- repeated normalization/keying for identical strings
|
||||
- unnecessary Azure round-trips when multiple missing terms are encountered and requests are performed sequentially
|
||||
|
||||
The result is that `find_matching_capacities(...)` can be slow, and while the second run is faster (due to cache hits), it can still take noticeable time.
|
||||
|
||||
## Goals
|
||||
|
||||
1. Ensure each unique text is embedded at most once per run (network and cache).
|
||||
2. Prefetch embeddings **before** similarity computation so inner loops only perform vector math.
|
||||
3. Enable Azure embeddings API batch calls to reduce network round-trips.
|
||||
4. Preserve current similarity output contracts and scoring behavior.
|
||||
5. Maintain explicit, deterministic failure semantics (no heuristics).
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Changing match scoring weights or category thresholds.
|
||||
- Changing the similarity strategy semantics ("per_skill" vs "aggregate").
|
||||
- Changing the embedding model or dimensionality.
|
||||
- Introducing new external dependencies unless necessary.
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### 1) Bulk prefetch & dedup in `SimilarityEngine`
|
||||
|
||||
#### Current behavior (problem)
|
||||
|
||||
`SimilarityEngine` currently embeds terms on demand via `_embed(...)`. For large candidate lists, this forces a high number of calls to `_embed`, and repeated SQLite reads.
|
||||
|
||||
#### New behavior
|
||||
|
||||
Add a prefetch step that:
|
||||
|
||||
- collects all texts required to compute similarity for a run (competences **and roles**)
|
||||
- **Important:** this collection is performed **across the entire candidate set** (i.e., roles and competences of **all free capacities at once**), not person-by-person.
|
||||
- normalizes for cache key stability
|
||||
- deduplicates texts
|
||||
- fetches all embeddings from caches/API in bulk
|
||||
- returns a mapping from normalized text → embedding vector
|
||||
|
||||
Then similarity computation uses the prefetched mapping (pure Python vector math).
|
||||
|
||||
### 2) Per-run in-memory cache
|
||||
|
||||
Add a lightweight in-memory dict for the duration of a single matching invocation.
|
||||
|
||||
- key: `(model, dimensions, normalized_text)` or reuse the computed SHA256 cache key
|
||||
- value: `list[float]`
|
||||
|
||||
This prevents repeated SQLite lookups for frequently used items (common competences, repeated role strings, etc.).
|
||||
|
||||
### 3) True Azure embeddings batch calls
|
||||
|
||||
Change `AzureOpenAIClient.get_embeddings_batch(...)` to use the Azure embeddings API with `input=[...]` to embed multiple texts in a single request.
|
||||
|
||||
Key details:
|
||||
|
||||
- supports chunking with a **configurable** batch size
|
||||
- preserves stable output ordering and deterministic error semantics
|
||||
- continues to log costs via `CostTracker`
|
||||
|
||||
If any item fails, the batch call should raise `AzureAPIError` (no fallback).
|
||||
|
||||
## Decisions (resolved)
|
||||
|
||||
- **Scope**: bulk prefetch includes **competences and roles**.
|
||||
- **Batching**: Azure embeddings requests are made in chunks with a **configurable** batch size.
|
||||
- **Normalization**: keep the current text normalization and cache-key behavior.
|
||||
- **Concurrency**: process chunks **sequentially** (no parallel calls).
|
||||
- **In-memory cache lifecycle**: implemented as a new instance variable in `SimilarityEngine` and **not** created/cleaned per matching invocation.
|
||||
- **Prefetch API**: new public method `prefetch_embeddings(required_texts, candidate_texts, required_roles, candidate_roles)` that returns a mapping from normalized text → embedding vector.
|
||||
- **Azure batch response ordering**: rely on the `index` field in the response if available, otherwise assume stable ordering.
|
||||
- **Error handling**: entire matching operation fails immediately; attempt to identify and report which specific text(s) caused the failure.
|
||||
- **Cost tracking**: log once per batch chunk with a count.
|
||||
- **Empty inputs**:
|
||||
- emit a **Python logger warning** when all candidate competences normalize to empty strings
|
||||
- prefetch skips empty/whitespace-only texts
|
||||
- **Cache miss batching**: batch-request only the missing embeddings (after resolving cache hits from SQLite).
|
||||
|
||||
## Configuration
|
||||
|
||||
Introduce (or reuse if already present) one configuration knob:
|
||||
|
||||
- `azure_openai.embedding_batch_size` (integer)
|
||||
- default: `128` (conservative)
|
||||
- no enforced maximum value
|
||||
- used to chunk `input=[...]` for Azure embeddings batch requests
|
||||
- if not specified in `config.toml`, the default value is used
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A single `find_matching_capacities(...)` run should perform at most one embedding generation per unique term (after cache resolution).
|
||||
- Only cache-missing embeddings are batch-requested from Azure (cache hits are not re-requested).
|
||||
- For warmed cache runs, no Azure embedding calls should occur.
|
||||
- Unit tests cover dedup/prefetch, cache layering, and batch request behavior.
|
||||
- Integration tests (manual/gated) verify real Azure batch API behavior.
|
||||
|
||||
## Rollout / Compatibility
|
||||
|
||||
- No configuration changes are strictly required; the baseline refactor should work with existing `config.toml`.
|
||||
- If new config knobs are added (e.g., batch size), defaults should preserve behavior.
|
||||
|
||||
Reference in New Issue
Block a user