Orchestrator/bahn/teamlandkarte-mcp/docs/architecture.md

Architecture Documentation (Arc42)

1. Introduction and Goals

1.1 Requirements Overview

The Teamlandkarte MCP Server enables AI assistants to match tasks to DB Systel employees with free work capacity, and to match a specific capacity to relevant tasks.

The server:

• queries the DB Systel Open Data Lake (Trino) read-only
• performs BM25+RRF lexical matching for competence similarity
• uses LLM chat completions (Azure OpenAI) for role similarity and role inference
• returns deterministic, table-first Markdown outputs suitable for chat UIs

Availability is displayed and may be applied as an optional date-range filter. It does not influence similarity scoring.

Key Features:

• MCP workflow for DB tasks and ad-hoc matching
• LLM-based role inference (tool: infer_primary_role) using Data Lake role vocabulary
• BM25+RRF competence matching with optional LLM-based auto-tagging pre-expansion
• Two matching methods selectable per call via matching_method:
  • score (default): BM25+RRF competence + LLM role similarity → numeric scores + category
  • llm_fulltext: LLM-based full-text comparison of complete profiles → category + rationale (no numeric scores)
• Capacity browsing + capacity-to-task matching tools:
  • list_free_capacities, get_capacity_details, find_matching_tasks
• Hard confirmation gate before matching runs by default (matching.require_confirmation = true)
• Interactive result exploration with search session management (search_id + filter_id)
• Multi-level caching (DB query cache + search result cache)
• Strict DB schema verification at startup (fail-fast) against required view columns
• Fuzzy filtering on search results (role, competence) via fuzzywuzzy

1.2 Quality Goals

Priority	Quality Goal	Motivation
1	Security	Read-only database access, secure credential management, no data leakage
2	Performance	Fast response times through caching, connection pooling
3	Usability	Simple MCP interface, clear result presentation, deterministic tool outputs
4	Maintainability	Modular architecture, clear separation of concerns, testable components
5	Reliability	Fail-fast schema checks, graceful degradation on LLM failures

Reliability

• The server uses Azure OpenAI for chat completions (role similarity, role inference, auto-tagging).
• There is no embedding dependency -- competence matching is purely lexical (BM25+RRF).
• LLM failures in role similarity return 0.0 (graceful degradation).
• LLM failures in auto-tagging fall back to the unmodified candidate list.
• LLM failures in role inference return None (surfaced to the user).

1.3 Stakeholders

Role	Contact	Expectations
Product Owner	Thomas Handke	Feature delivery, quality, timeline
End Users	DB Systel Employees	Fast, accurate matching
Database Team	DB Systel IT	Minimal database load, read-only access
Security Team	DB Systel Security	Credential protection, audit logging

---

2. Architecture Constraints

2.1 Technical Constraints

Constraint	Background
Python 3.13+	Required for latest language features and type hints
Trino client	Required for Trino/Presto connectivity to the Open Data Lake
MCP Protocol	Must comply with Model Context Protocol specification (FastMCP SDK)
Azure OpenAI chat completions	Required for role similarity, role inference, and optional auto-tagging
Read-only database	No write operations allowed on production database
Strict schema verification	Server fails fast if required view columns are missing

2.2 Organizational Constraints

Constraint	Background
Internal DB Infrastructure	Hosted on Deutsche Bahn Systel GitLab and data lake
Credential Management	Stored in environment variables / .env, not in version control
OpenSpec Workflow	All significant changes require spec proposals

2.3 Conventions

Convention	Details
Code Style	PEP 8, type hints, dataclasses for DTOs, ruff for linting/formatting
Configuration	TOML format (config.toml) for all configuration
API Design	MCP tools with clear parameter schemas, Markdown table outputs
Documentation	Inline docstrings, OpenSpec for architecture decisions
Testing	pytest + pytest-asyncio + hypothesis

---

3. System Scope and Context

3.1 Business Context

End User (DB Systel Employee)
  | natural language
AI Assistant (MCP client)
  | MCP (tool calls over stdio)
Teamlandkarte MCP Server
  | SQL (read-only)
DB Systel Open Data Lake (Trino)

Teamlandkarte MCP Server
  | HTTPS (chat completions)
Azure OpenAI

External Entities:

Entity	Role	Interface
AI Assistant (Claude, GPT, etc.)	Interprets user requests and calls MCP tools	MCP Protocol over stdio
DB Systel Open Data Lake	Stores employee capacity, competence, task, and role data	Trino/Presto SQL
Azure OpenAI	Provides chat completions for role similarity + inference + auto-tagging	HTTPS (Azure OpenAI REST API)
End User	Requests matching through natural conversation	Natural language via AI assistant

3.2 Technical Context

MCP Client (assistant)
  <-> MCP protocol (tool calls, stdio JSON-RPC)
Teamlandkarte MCP Server (Python)
  <-> Trino (read-only SQL, connection pool)
  <-> Azure OpenAI (chat completions)
  <-> Local caches (DB query cache + search session cache)

---

4. Solution Strategy

4.1 Architecture Approach

Layered Architecture with clear separation:

1. MCP Interface Layer (mcp_server.py): Exposes tools to client, formats Markdown output
2. Business Logic Layer (matching/): Matcher, scorer, SimilarityEngine, VocabularyCache, BM25, RRF, AutoTagger, LlmFulltextMatcher (LLM-based full-text matching with rationale)
3. Data Access Layer (database/): TrinoClient, connection pool, query logger, schema verification, read-only guard
4. External Integration Layer (azure/): AzureOpenAIClient, CostTracker
5. Infrastructure (cache/, utils/, config.py): QueryCache, SearchCache, date/markdown utilities, configuration loading

4.2 Key Design Decisions

Decision	Rationale
BM25+RRF competence matching	Lexical matching eliminates embedding false-positives; zero-out rule ensures no token overlap = score 0.0
LLM-based role similarity	Semantic role comparison requires understanding synonyms and hierarchy; cached per-run
LLM-based role inference	Selects from DB vocabulary; constrained to known roles only
LLM-based full-text matching (alternative method)	llm_fulltext mode compares complete capacity/task profiles (description, references, certificates, skills) via Azure OpenAI Chat Completion and returns a category + 1–2 sentence rationale per item; no numeric scores
Optional LLM auto-tagging	Bridges lexical gap for BM25 by expanding candidate competences with covered synonyms
Global BM25 index	One index over all candidates' competences ensures stable IDF weights across the pool
Hard confirmation gating (default)	Prevents accidental matching runs; user must confirm requirements
Markdown table output	Better rendering in LLM clients; table-first outputs for robust parsing
Connection pooling	Bounded pool (default 4) for Trino connections; thread-safe acquire/release
Strict startup schema verification	Fail-fast when upstream views change columns
Two-tier caching	DB query cache (TTL) + search session cache (TTL) for interactive exploration
Fuzzy filtering	fuzzywuzzy for post-hoc filtering of search results by role/competence text

4.3 Technology Stack

Layer	Technology	Justification
Runtime	Python 3.13	Modern features, strong typing
MCP SDK	mcp (FastMCP)	MCP server runtime, stdio transport
Database Client	trino	Trino/Presto SQL connectivity
Caching	cachetools (TTLCache)	In-memory TTL caches for DB queries and search sessions
Configuration	TOML (tomllib)	Human-readable, validated at startup
Competence Matching	rank-bm25 (BM25Okapi)	Lexical ranking, no native extensions
Fuzzy Matching	fuzzywuzzy + python-Levenshtein	Fast fuzzy string matching for filters
LLM Integration	openai (AsyncAzureOpenAI)	Azure OpenAI chat completions
Environment	python-dotenv	Load credentials from .env

---

5. Building Block View

5.1 Level 0: System Context

+-------------------------------------------+
|       Teamlandkarte MCP Server            |
|       (Task <-> Capacity Matching)        |
+-------------------------------------------+

5.2 Level 1: Container View

+------------------------------------------------------------+
|                  Teamlandkarte MCP Server                   |
|                                                            |
|  +-------------------------------------------------------+ |
|  | MCP Interface Layer (mcp_server.py)                    | |
|  | - task browsing + details                              | |
|  | - requirement capture + confirmation gate              | |
|  | - task->capacity matching + search refinement          | |
|  | - capacity browsing + capacity->task matching          | |
|  +----------------------------+---------------------------+ |
|                               |                             |
|  +----------------------------v---------------------------+ |
|  | Business Logic Layer (matching/)                        | |
|  | - Matcher (orchestrates matching runs)                  | |
|  | - SimilarityEngine (BM25+RRF competence, LLM role)     | |
|  | - VocabularyCache (LLM role inference from DB vocab)    | |
|  | - Scorer (weighted overall score + categorization)      | |
|  | - Bm25Index + bm25_rank_competences                    | |
|  | - reciprocal_rank_fusion (RRF normalization)            | |
|  | - AutoTagger (optional LLM competence expansion)        | |
|  | - LlmFulltextMatcher (LLM full-text profile matching)   | |
|  +----------------------------+---------------------------+ |
|                               |                             |
|  +----------------------------v---------------------------+ |
|  | Data Access Layer (database/)                           | |
|  | - TrinoClient (SQL queries, implements DBClient)        | |
|  | - ConnectionPool (bounded, thread-safe)                 | |
|  | - SchemaVerifier (startup fail-fast)                    | |
|  | - ReadOnly guard (SELECT/WITH only)                     | |
|  | - QueryLogger (redacted SQL logging)                    | |
|  +----------------------------+---------------------------+ |
|                               |                             |
|  +----------------------------v---------------------------+ |
|  | External Integration (azure/)                           | |
|  | - AzureOpenAIClient (chat completions, retry/backoff)   | |
|  | - CostTracker (session-level token accounting)          | |
|  +--------------------------------------------------------+ |
|                                                            |
|  +--------------------------------------------------------+ |
|  | Infrastructure (cache/, utils/, config.py)              | |
|  | - QueryCache (TTL, DB results)                          | |
|  | - SearchCache (TTL, search sessions + filters)          | |
|  | - dates.py (ISO parsing, overlap checks)                | |
|  | - markdown.py (table rendering)                         | |
|  | - config.py (TOML loading + validation)                 | |
|  +--------------------------------------------------------+ |
+------------------------------------------------------------+
         |                          |
         v                          v
+-----------------+       +-----------------+
|  Open Data Lake |       |  Azure OpenAI   |
|  (Trino)        |       |  (Chat API)     |
+-----------------+       +-----------------+

5.3 Level 2: Source Module Map

src/teamlandkarte_mcp/
├── __init__.py
├── __main__.py          # CLI entry point, arg parsing, logging setup
├── config.py            # TOML config loading + validation (AppConfig)
├── logging_config.py    # Logging to stderr (MCP stdio safety)
├── mcp_server.py        # FastMCP server, all tool definitions, SessionState
├── models.py            # Frozen dataclasses: Task, Capacity, Requirements, ScoredCapacity
├── azure/
│   ├── openai_client.py # AsyncAzureOpenAI wrapper (chat completions, retry)
│   └── cost_tracker.py  # Session-level token/cost accounting
├── cache/
│   ├── query_cache.py   # Generic TTLCache wrapper for DB queries
│   └── search_cache.py  # Search session store (search_id, filter_id, TTL)
├── database/
│   ├── types.py         # DBClient Protocol definition
│   ├── db_client.py     # Factory: create_db_client() -> TrinoClient
│   ├── trino_client.py  # TrinoClient (implements DBClient, uses pool)
│   ├── pool.py          # ConnectionPool (bounded, thread-safe, LIFO)
│   ├── schema_verifier.py # Startup column verification
│   ├── read_only.py     # SQL guard (SELECT/WITH only)
│   ├── query_logger.py  # Redacted query logging
│   └── hive_client.py   # (empty, reserved)
├── matching/
│   ├── matcher.py       # Matcher: orchestrates matching runs
│   ├── scorer.py        # compute_overall(), categorize()
│   ├── similarity.py    # SimilarityEngine (BM25+RRF competence, LLM role)
│   ├── vocabulary.py    # VocabularyCache (LLM role inference)
│   ├── bm25.py          # Bm25Index, bm25_rank_competences, _tokenize
│   ├── rrf.py           # reciprocal_rank_fusion (zero-out rule)
│   ├── auto_tagger.py   # AutoTagger (LLM competence expansion)
│   ├── llm_fulltext_matcher.py  # LlmFulltextMatcher (LLM full-text profile matching)
│   ├── profiles.py      # CapacityProfile, TaskProfile + serializers
│   ├── task_analyzer.py # (deprecated stub)
│   └── task_helpers.py  # (deprecated stub)
└── utils/
    ├── dates.py         # parse_iso_date, availability_overlaps
    └── markdown.py      # md_table (GFM table rendering)

5.4 Tool Surface (MCP Tools)

Discovery (DB tasks)

Tool	Responsibility	Dependencies
list_open_tasks	List newest published tasks	DB
get_task_details	Table-first task fields + inferred role	DB, LLM (role inference)
validate_task_requirements	Compare DB fields vs LLM-inferred role + competences	DB, LLM
infer_primary_role	Infer closest role from task id or free text	DB vocab, LLM

Requirement capture + confirmation gate

Tool	Responsibility	Dependencies
extract_requirements	LLM-based inference for role + competences from free text	DB vocab, LLM
collect_structured_requirement_data	Accept user-supplied fields and stage requirements	session state
update_requirements	(deprecated stub, directs to collect_structured_requirement_data)	--
start_guided_capture	Start step-by-step guided capture	session state
guided_set_description	Set description (step 1/4)	session state
guided_set_role	Set role (step 2/4)	session state
guided_set_time_range	Set date range (step 3/4)	session state
guided_set_competences	Set competences (step 4/4)	session state
show_pending_requirements	Show review table for pending requirements	session state
request_requirements_confirmation	Mark that assistant asked user to confirm	session state
confirm_requirements	Confirm or reject staged requirements	session state

Task -> capacity matching

Tool	Responsibility	Dependencies
find_matching_capacities	Compute matches and store a search session (search_id). Accepts matching_method ("score" | "llm_fulltext").	DB, BM25, LLM (role) or LlmFulltextMatcher, caches
filter_search_results	Refine stored results (fuzzy role/competence, availability, min_similarity). min_similarity is ignored in llm_fulltext mode and surfaced as a hint in the Applied Filters table.	SearchCache
get_results_by_category	Page through stored results by category. Renders Begründung column instead of score columns when the search was run in llm_fulltext mode.	SearchCache

Capacity browsing + capacity -> task matching

Tool	Responsibility	Dependencies
list_free_capacities	List recently created free capacities	DB
get_capacity_details	Table-first capacity fields + next steps	DB
find_matching_tasks	Find tasks that match a specific capacity. Accepts matching_method ("score" | "llm_fulltext").	DB, BM25, LLM (role) or LlmFulltextMatcher, caches

Team browsing + task -> team matching

Tool	Responsibility	Dependencies
list_teams	List the first N teams (default 20) as a Markdown table (Team Id, Team Name, Schwerpunkt, Anzahl Kompetenzen, Anzahl Referenzen).	DB
get_team_details	Table-first team fields plus sections ## Über uns, ## Leistungen, ## Interessen, ## Kompetenzen (with (Top) marker), ## Referenzen (Partner_Name + Projekte) and ## Next steps.	DB
find_matching_teams	Match a task against team profiles and store a search session (search_id). Accepts matching_method ("score" | "llm_fulltext"). Persists search_type = "team_search" and the chosen matching_method in SearchCache and the META JSON.	DB, BM25, LLM (role) or LlmFulltextMatcher, caches

Profile_Type parameter

Parameter	Allowed values	Default	Selection mechanism	Affected tools
Profile_Type	"capacity", "team"	implicit per tool name	Tool selection (find_matching_capacities ⇒ capacity, find_matching_teams ⇒ team); persisted as search_type ("capacity_search" / "team_search") in SearchCache and META JSON	find_matching_capacities, find_matching_teams, list_teams, get_team_details

• Profile_Type = "capacity" (existing behavior): match a task against employee capacity profiles. find_matching_capacities is the entry point; availability filters apply.
• Profile_Type = "team" (new): match a task against aggregated team profiles. find_matching_teams is the entry point; list_teams and get_team_details browse and inspect team profiles independently of a matching run. Availability filters are not applied for team searches and are surfaced as not-effective hints in the Applied Filters table.

Matching method parameter

Parameter	Allowed values	Default	Affected tools
matching_method	"score", "llm_fulltext"	[matching].default_method (TOML, default "score")	find_matching_capacities, find_matching_tasks, find_matching_teams

• "score": existing BM25+RRF competence + LLM role similarity. Output includes Role Score, Competence Score, Overall Score, Category. For team searches, the team's focus_name is used as the role stand-in and top competences are weighted by matching.team.top_competency_weight (default 1.5).
• "llm_fulltext": LLM-based full-text comparison via LlmFulltextMatcher. Output replaces all score columns with a single Begründung column (1–2 sentence rationale from the LLM). The persisted META JSON contains matching_method so downstream tools (get_results_by_category, filter_search_results) know which schema to render.

For team searches, the result table columns are:

Mode	Columns
team_search × score	Team Name, Schwerpunkt, Top-Kompetenzen, Role Score, Competence Score, Overall Score, Category
team_search × llm_fulltext	Team Name, Schwerpunkt, Top-Kompetenzen, Category, Begründung

5.5 LlmFulltextMatcher (component)

The LlmFulltextMatcher (in matching/llm_fulltext_matcher.py) is the business-logic component behind matching_method = "llm_fulltext".

Aspect	Details
Inputs	A TaskProfile (id, title, description, skills) for match_capacities; a CapacityProfile (id, owner_name, role_name, competences, description, references with partner_name/projects, certificates) for match_tasks. The candidate list (capacities or tasks) is already prefiltered by the MCP tool (same prefilter as in score mode, e.g. availability).
Profile sources	Built from in-memory Capacity/Task objects plus extras loaded from the DB via DBClient.batch_get_capacity_descriptions, batch_get_capacity_certificates, and batch_get_capacity_references (with the partner LEFT JOIN inside the same SQL query).
Outputs	LlmFulltextResult with by_category: dict[str, list[LlmFulltextItem]] (categories Top/Good/Partial/Low/Irrelevant; each item carries category, ungekürzte rationale, raw fields) and errors: list[LlmFulltextError] for items where the LLM call/parse failed.
External dependency	Azure OpenAI Chat Completion via AzureOpenAIClient.chat_completion(...) with response_format=json_object, deterministic German system prompt, JSON schema {"category": "...", "rationale": "..."}. One LLM call per candidate.
Determinism	Profile serialization is field-stable; results within a category are sorted lexicographically by item_id; invalid LLM categories map to Irrelevant with a hint appended to the rationale.
Persistence	Items are stored in SearchCache with category + ungekürzte rationale (no numeric score fields); errors are persisted alongside; META JSON carries matching_method.

---

6. Runtime View (Key Flows)

6.1 Server startup (fail-fast)

1. Parse CLI arguments (--config, --log-level)
2. Configure logging (stderr only, MCP stdio safety)
3. Load and validate config.toml + environment variables
4. Create DB client (TrinoClient with connection pool)
5. Verify required DB view columns (schema verification -> fail-fast on mismatch)
6. Initialize caches (QueryCache, SearchCache)
7. Initialize Azure OpenAI client + CostTracker
8. Optionally construct AutoTagger (if use_auto_tagging = true)
9. Initialize SimilarityEngine, VocabularyCache, Matcher
10. Register all MCP tools
11. Start MCP server (stdio transport via mcp.run())

6.2 Task -> capacity matching

1. Capture requirements (guided capture or extract_requirements or collect_structured_requirement_data)
2. (If require_confirmation = true) confirmation gate:
   • show_pending_requirements -> user reviews
   • request_requirements_confirmation -> assistant marks confirmation requested
   • confirm_requirements(true) -> user confirms
3. Run find_matching_capacities(role_name, competences, date_start?, date_end?, matching_method?):
   • Resolve matching_method (parameter > [matching].default_method > "score")
   • Query all capacities + competences from DB (cached)
   • Filter by date overlap (same prefilter for both methods)
   • Branch by matching_method:
     • "score": build global BM25 index, run BM25+RRF competence similarity + LLM role similarity, compute weighted overall score + categorize
     • "llm_fulltext": batch-load description, certificates, references (with partner LEFT JOIN) for the filtered capacities; build TaskProfile + CapacityProfile; one Azure OpenAI Chat Completion per capacity (response_format=json_object); LLM returns {category, rationale}; invalid categories → Irrelevant with hint; LLM/JSON failures recorded in a separate errors list
   • Store results in SearchCache (payload includes matching_method; LLM mode stores category + ungekürzte rationale per item, no score fields) -> return search_id
4. Browse/filter results:
   • get_results_by_category(search_id, category, page, page_size) (renders Begründung column in LLM mode)
   • filter_search_results(search_id, ...) -> returns filter_id. In LLM mode, min_similarity is ignored and surfaced as a hint; sorting falls back to (category_rank, item_id).

6.3 Capacity -> task matching

1. Browse capacities (list_free_capacities)
2. Inspect (get_capacity_details)
3. Run find_matching_tasks(capacity_id, matching_method?):
   • Resolve matching_method (parameter > [matching].default_method > "score")
   • Query capacity details + competences from DB
   • Query open tasks from DB (same prefilter for both methods)
   • Branch by matching_method:
     • "score": for each task infer role + competences, compute BM25+RRF + LLM role similarity
     • "llm_fulltext": load description, certificates, references (with partner LEFT JOIN) for the capacity; build CapacityProfile once; for each task build a TaskProfile and run one Azure OpenAI Chat Completion; same error/category-normalization handling as in 6.2
   • Store results as a search session (matching_method persisted) -> return search_id
4. Browse/filter results (same tools as task->capacity)

6.4 Task -> team matching

1. Capture requirements as in 6.2 (guided capture, extract_requirements, or collect_structured_requirement_data)
2. Optional confirmation gate (same as in 6.2): show_pending_requirements -> request_requirements_confirmation -> confirm_requirements(true)
3. Run find_matching_teams(role_name, competences, matching_method?):
   • Resolve matching_method (parameter > [matching].default_method > "score"); reject unknown values with an error listing the allowed values, without any DB or LLM call
   • Validate minimum requirements (competences non-empty)
   • Load all teams from the DB-backed cache _get_teams_cached(). The DB layer joins teamlandkarte_v_teams_latest with teamlandkarte_v_teammeter_organizational_units_latest (INNER JOIN over team_id = id for the team name), and aggregates competences (teamlandkarte_v_teammeter_team_competences_latest joined to teamlandkarte_v_competences_latest over competence_id = id) and references (teamlandkarte_v_team_references_latest LEFT JOIN teamlandkarte_v_partners_latest over partner_id = id, exposing name as Partner_Name). Joins between team master data, competences, and references run over ouid.
   • No availability prefilter: team profiles do not carry an availability range; any provided date range is ignored and surfaced as a not-effective hint in the Applied Filters table.
   • Branch by matching_method:
     • "score": build the requirements competence list, derive each team's competence list and parallel top-competence set from Team.competences, and compute competence similarity via the existing SimilarityEngine. Top competences are upweighted by matching.team.top_competency_weight (default 1.5); the per-required score is clamped to [0, 1]. Role similarity uses Team.focus_name as the role stand-in (compute_role_similarity(req.role_name, team.focus_name)). Overall score and category use the same compute_overall(...) / categorize(...) thresholds as for capacities.
     • "llm_fulltext": build a TaskProfile from the requirements and a TeamProfile per team via build_team_profile + serialize_team_profile. The user prompt is === Aufgabe === + === Team === (with an ID: header). The system prompt and JSON schema ({"category": ..., "rationale": ...}) are reused from the capacity LLM path; invalid categories are normalized to Irrelevant with a hint suffix in the rationale; LLM/JSON errors land in a separate errors list.
   • Store results in SearchCache with search_type = "team_search" and matching_method = ... -> return search_id. The META JSON exposes both search_type and matching_method so get_results_by_category / filter_search_results render the team-specific columns.
4. Browse/filter results:
   • get_results_by_category(search_id, category, page, page_size) (renders Begründung column in llm_fulltext mode)
   • filter_search_results(search_id, ...). role_filter matches against Team_Focus_Name; competence_filter matches the competence names of the team and supports a (Top) suffix that restricts the filter to top competences. availability_date_start, availability_date_end, and is_fully_available are ignored for team_search and surfaced as team_search not-effective hints in the Applied Filters table.

6.5 Role inference flow

1. infer_primary_role(task_id=... | task_text=...)
2. Fetch all role names from DB vocabulary view
3. Send task text + role list to LLM (JSON response format)
4. Validate returned role exists in DB vocabulary
5. Return (role_name, confidence) or None on failure

---

7. Deployment View

+-------------------------------------------------+
|  Developer Machine / CI                          |
|                                                  |
|  +--------------------------------------------+ |
|  |  MCP Client (IDE / AI Assistant)           | |
|  |  <-> stdio (JSON-RPC)                      | |
|  |  teamlandkarte-mcp process                 | |
|  |  (Python 3.13, single process)             | |
|  +--------------------------------------------+ |
|                                                  |
|  config.toml + .env (credentials)                |
+-------------------------------------------------+
         |                          |
         v                          v
+-----------------+       +-----------------+
|  Trino cluster  |       |  Azure OpenAI   |
|  (Data Lake)    |       |  (Chat API)     |
+-----------------+       +-----------------+

Requirements:

• Python 3.13+ with project dependencies installed
• Trino connectivity (host/port via config.toml, credentials via env vars)
• Azure OpenAI endpoint + chat deployment + API key (env var AZURE_OPENAI_LLM_API_KEY)
• Environment variables: DATA_LAKE_USERNAME, DATA_LAKE_PASSWORD, AZURE_OPENAI_LLM_API_KEY

---

8. Cross-cutting Concepts

8.1 Deterministic outputs

• Tools produce table-first Markdown output (GFM tables via md_table()).
• Search tools emit deterministic headers with SEARCH_ID=... / FILTER_ID=... for robust client parsing.
• Scoring formula is deterministic: overall = competence_weight * competence_score + role_weight * role_score.

8.2 Caching strategy

Cache	Implementation	TTL	Purpose
DB query cache	QueryCache (cachetools TTLCache)	Configurable (db_ttl_hours, default 12h)	Avoid repeated expensive Trino queries
Search session cache	SearchCache (cachetools TTLCache)	Configurable (search_ttl_minutes, default 60min)	Enable interactive exploration of results
LLM role similarity cache	In-memory dict on SimilarityEngine	Per matching run (cleared between runs)	Avoid duplicate LLM calls for same role pair

8.3 Security

• Read-only DB access: ensure_select_only() guard rejects non-SELECT/WITH queries
• No write operations: Server never modifies the Data Lake
• Credentials via environment: DATA_LAKE_USERNAME, DATA_LAKE_PASSWORD, AZURE_OPENAI_LLM_API_KEY -- never in config files
• Query logging: Parameters are redacted before logging (query_logger.py)
• Logging to stderr: MCP stdio transport requires stdout to remain clean JSON-RPC

8.4 Error handling

• Schema verification failure: Server refuses to start (fail-fast)
• DB connectivity: Lazy validation on first DB-backed tool call
• LLM failures (role similarity): Return 0.0, no caching of failed result
• LLM failures (auto-tagging): Return unmodified candidate list (graceful degradation)
• LLM failures (role inference): Return None, logged as warning
• Connection pool exhaustion: PoolExhaustedError raised immediately (no blocking wait)
• Config errors: ConfigError raised at startup with descriptive message

8.5 Connection pooling

• ConnectionPool (generic, thread-safe, LIFO queue)
• Bounded at pool_size (default 4) connections
• Connections created lazily on demand
• closeall() for graceful shutdown

---

9. Scoring and Matching Algorithm

9.1 Overall scoring formula

overall_score = competence_weight * competence_score + role_weight * role_score

Default weights: competence_weight = 0.8, role_weight = 0.2 (must sum to 1.0).

9.2 Competence scoring (BM25 + RRF)

For each required competence:

1. Query the global BM25 index (built over all filtered candidates' competences)
2. Filter ranked results to the current candidate's competence set
3. Apply Reciprocal Rank Fusion (k=60) to normalize scores to [0.0, 1.0]
4. Zero-out rule: No token overlap -> score 0.0 (eliminates false positives)

Aggregate: competence_score = mean(per_required_competence_scores)

Threshold for "matched": score >= 0.5.

9.3 Role scoring (LLM)

• LLM chat completion compares two role names semantically -> similarity in [0.0, 1.0]
• Identical roles (case-insensitive) -> 1.0 (short-circuit, no LLM call)
• Empty/unknown roles -> 0.0 (short-circuit)
• Results cached symmetrically per matching run

9.4 Categorization

Category	Threshold
Top	overall >= configured top (default 0.8)
Good	overall >= configured good (default 0.65)
Partial	overall >= configured partial (default 0.5)
Low	overall >= configured low (default 0.3)
Irrelevant	overall < configured low

(Thresholds are configurable via [matching.thresholds].)

9.5 Optional: Auto-Tagging (LLM competence expansion)

When matching.similarity.use_auto_tagging = true:

1. Before BM25 scoring, the AutoTagger asks the LLM which required competences are already covered by the candidate's existing competences (via synonym, abbreviation, cross-language equivalence)
2. Covered required-competence names are appended to the candidate's working list
3. Expansion is ephemeral -- never persisted to DB
4. On LLM failure, scoring proceeds with the unmodified list

---

10. Configuration Reference

10.1 config.toml structure

[database]
backend = "trino"           # Only "trino" supported
host = "..."
port = 443
http_scheme = "https"
verify_ssl = true
catalog = "hive"
schema = "tier1_open_lake"
connect_timeout = 10
pool_size = 4

[matching]
competence_weight = 0.8     # Must sum to 1.0 with role_weight
role_weight = 0.2
require_confirmation = true # Hard gate before matching
default_method = "score"    # Default for matching_method when callers omit it.
                            # Allowed: "score" | "llm_fulltext".

[matching.thresholds]
top = 0.8
good = 0.65
partial = 0.5
low = 0.3

[matching.fuzzy]
min_similarity = 0.7       # Fuzzy filter threshold

[matching.similarity]
use_auto_tagging = false   # LLM pre-expansion for BM25

[cache]
db_ttl_hours = 12
search_ttl_minutes = 60
max_size = 100

[azure_openai]
endpoint = "https://..."
api_version = "2024-02-15-preview"
chat_deployment = "gpt-4.1"
show_costs_in_output = false

10.2 Required environment variables

Variable	Purpose
DATA_LAKE_USERNAME	Trino login username
DATA_LAKE_PASSWORD	Trino login password
AZURE_OPENAI_LLM_API_KEY	Azure OpenAI API key for chat completions

---

11. Data Model

11.1 Core domain objects (frozen dataclasses)

Class	Fields	Purpose
Task	id, name, title, description, start_date, end_date, created_date, skills	Published task from DB
Capacity	id, owner_name, role_name, role_level, begin_date, end_date, competences	Employee capacity entry
Requirements	role_name, competences, date_start, date_end, description	Structured matching input
ScoredCapacity	capacity, competence_score, role_score, overall_score, category, matched_competences, missing_competences	Matching result
Team	team_id, ouid, team_name, focus_name, about_us, offerings, interests, competences, references	Aggregated team master data
TeamCompetence	name, top_competency	Single team competence with top marker
TeamReference	partner_name, projects	Single team reference; partner_name may be empty
ScoredTeam	team, competence_score, role_score, overall_score, category, matched_competences, missing_competences	Team matching result (score mode)

11.2 Team Profile

A Team Profile is the second profile type next to the existing capacity profile. It aggregates fields from four read-only views into a single immutable Team instance and is used both for browsing (list_teams, get_team_details) and for matching (find_matching_teams in both score and llm_fulltext modes).

Field	Type	Source	Notes
team_id	str	teamlandkarte_v_teams_latest.team_id	Stable identifier; persisted in SearchCache.
ouid	str	teamlandkarte_v_teams_latest.ouid	Join key for competences and references.
team_name	str	teamlandkarte_v_teammeter_organizational_units_latest.name	Resolved via INNER JOIN teams_latest.team_id = organizational_units_latest.id. Teams without an OU match are excluded.
focus_name	str	teamlandkarte_v_teams_latest.focus_name	NULL → "". Used as role stand-in in score mode.
about_us	str	teamlandkarte_v_teams_latest.about_us	NULL → "".
offerings	str	teamlandkarte_v_teams_latest.offerings	NULL → "".
interests	str	teamlandkarte_v_teams_latest.interests	NULL → "".
competences	list[TeamCompetence]	teamlandkarte_v_teammeter_team_competences_latest joined to teamlandkarte_v_competences_latest over competence_id = id	Joined on ouid. top_competency = COALESCE(top_competency, FALSE). Order: top competences first, then name ascending.
references	list[TeamReference]	teamlandkarte_v_team_references_latest LEFT JOIN teamlandkarte_v_partners_latest over partner_id = id	Joined on ouid. Partner_Name from p.name (COALESCE(..., '')). Whitespace-only projects are filtered. Order: partner_name asc, projects asc.

Profile serialization (build_team_profile + serialize_team_profile in matching/profiles.py) emits a deterministic, German-headed text block in fixed order: Teamname:, Schwerpunkt:, Über uns:, Leistungen:, Interessen:, Kompetenzen:, Referenzen:. Top competences are suffixed with (Top). Reference rows render as Partner: <partner_name> – Projekte: <projects> when the partner is known, and as Projekte: <projects> (no placeholder) when the partner is empty. Empty lists render as Kompetenzen: (keine) / Referenzen: (keine). List ordering matches the DB-provided order.

11.3 Database views (Trino)

The server queries read-only views in the tier1_open_lake schema. Schema verification at startup checks:

• teamlandkarte_v_capacity_roles_latest: requires columns name, active, staffing_board_relevant
• teamlandkarte_v_capacities_latest: requires column creation_date
• teamlandkarte_v_teams_latest: requires columns team_id, ouid, about_us, offerings, interests, focus_name
• teamlandkarte_v_teammeter_organizational_units_latest: requires columns id, name
• teamlandkarte_v_teammeter_team_competences_latest: requires columns ouid, competence_id, top_competency
• teamlandkarte_v_team_references_latest: requires columns ouid, partner_id, projects

Additional views read in llm_fulltext mode

The following columns/views are not part of the startup schema verification but are read at runtime when matching_method = "llm_fulltext":

View	Columns used	Used for
teamlandkarte_v_capacities_latest	description	CapacityProfile.description
teamlandkarte_v_capacity_certificates_latest	capacity_id, description	CapacityProfile.certificates (1:n via capacity_id)
teamlandkarte_v_capacity_references_latest	capacity_id, partner_id, projects	CapacityProfile.references[].projects (1:n via capacity_id)
teamlandkarte_v_partners_latest	id, name	Partner name per reference (CapacityProfile.references[].partner_name)

Reference ↔ Partner join: teamlandkarte_v_capacity_references_latest.partner_id = teamlandkarte_v_partners_latest.id. The partner name column is exposed as partner_name (via COALESCE(p.name, '')) inside the same SQL query that loads the references, so no extra round-trip is required. When partner_id is NULL or no partner matches, partner_name is the empty string and the reference is still returned with its projects text. The LEFT JOIN keeps the reference list complete even for unknown partners.

Views read for team profiles

The following views back the Team data type and are loaded by find_matching_teams, list_teams, and get_team_details:

View	Columns used	Used for
teamlandkarte_v_teams_latest	team_id, ouid, about_us, offerings, interests, focus_name	Team master data and free-text fields
teamlandkarte_v_teammeter_organizational_units_latest	id, name	Team.team_name via INNER JOIN over teams_latest.team_id = organizational_units_latest.id
teamlandkarte_v_teammeter_team_competences_latest	ouid, competence_id, top_competency	Team competences with top markers (joined to teamlandkarte_v_competences_latest over competence_id = id to resolve the competence name)
teamlandkarte_v_team_references_latest	ouid, partner_id, projects	Team references with partner names
teamlandkarte_v_partners_latest	id, name	Partner_Name per team reference

Team_Name join (INNER JOIN): teamlandkarte_v_teams_latest.team_id = teamlandkarte_v_teammeter_organizational_units_latest.id. Teams without a matching OU row are excluded from get_all_teams and get_team_by_id.

Competence and reference joins: Both teamlandkarte_v_teammeter_team_competences_latest and teamlandkarte_v_team_references_latest are joined to the team master data over the shared ouid column.

Reference ↔ Partner join (LEFT JOIN): teamlandkarte_v_team_references_latest.partner_id = teamlandkarte_v_partners_latest.id. The name column from teamlandkarte_v_partners_latest is exposed as Partner_Name via COALESCE(p.name, '') inside the same SQL query that loads the references. When partner_id is NULL or no partner matches, Partner_Name is the empty string and the reference is still returned with its projects text. Whitespace-only projects are filtered out in the DB layer.

---

12. Testing

• Unit tests: pytest + pytest-asyncio
• Property-based tests: hypothesis (for scoring, BM25, RRF invariants)
• Type checking: mypy (strict)
• Linting/formatting: ruff
• No integration test infrastructure: DB and Azure OpenAI are mocked in tests