enhancement
major
minor
defect
critical
major
minor
major
#30053
Agent runs: bound the pool of parallel tool calls (AgentLoop.TOOL_WORKERS) and tell a user whose chat message waits for a free worker
Follow-up to #29767.
Context
The run limits of the AI modules are configured in <capacity> on AgentRuntimeService and <budget> on AgentPolicyService (developer article ai/limits-and-caching, section "Run limits"). Two gaps remain in what an installation can bound and what a user is told.
1. The pool of parallel tool calls has no ceiling
What: AgentLoop.TOOL_WORKERS (tl-ai-core, com.top_logic.tl.ai.core.agent.AgentLoop) is a static Executors.newCachedThreadPool(...). Every READ tool call of a parallel batch (executeConcurrent) takes a thread of its own, across all runs of the node. chat-workers bounds how many runs execute at once, but not how many tool threads those runs start: a model that emits a batch of many READ calls, multiplied by the concurrent runs, creates as many threads.
Fix: add a ceiling to <capacity> (e.g. tool-workers, 0 = no ceiling, the present behaviour), read through RunLimitSettings like the other capacity settings, and build a bounded pool from it in which a call beyond the ceiling waits (as chat-workers does - a READ call must not be refused). Keep the pool's thread name agent-tool-worker and its daemon threads; the bound must not change the order in which results are added to the conversation (collectBatchResults).
Test: a unit test with tool-workers="2" and a batch of five READ calls of a tool that blocks on a latch: at most two calls run at the same time, all five results arrive in batch order; with the default, all five run at once.
2. A queued chat message says nothing
What: With chat-workers set and the default chat-queue="0", a send that finds every worker busy waits in the queue of ChatRunExecutor (tl-chat). The user sees the composer busy and no run starts; nothing tells them that the answer is late because the installation is busy, not because the model is slow.
Fix: when ChatProcessor.sendMessage submits a run that does not start at once (the pool has no idle worker), send an SSE frame that the chat shows as a notice line ("Waiting for a free worker - position N" / "Warte auf einen freien Bearbeiter - Position N"), and clear it when the run starts. The texts are I18NConstants of tl-chat with @en and @de.
Test: a tl-chat test with chat-workers="1": a second send while the first run blocks receives the waiting frame, and the run-started frame of that send follows once the first run ends; a vitest of the chat UI that draws and clears the notice.
Lösung
- Gemeinsamer begrenzter Worker-Pool in tl-ai-core (limits): die bisher in ChatRunExecutor (tl-chat) gebaute Pool-Variante (feste Größe, Warteschlange, Dämon-Threads, allowCoreThreadTimeOut, Neuaufbau bei geänderter Konfiguration) wird zu einer allgemeinen Klasse, die ChatRunExecutor und der Pool der parallelen Tool-Aufrufe (tool-workers, Thread-Name agent-tool-worker) gemeinsam nutzen.
- Keine Verklemmung bei verschachtelten Batches: Startet ein Tool auf einem Tool-Worker selbst einen Agentenlauf (Subagent, Delegation), führt dessen paralleler Batch seine Aufrufe auf dem eigenen Thread nacheinander aus, statt auf freie Worker desselben begrenzten Pools zu warten. Die Obergrenze gilt damit, ohne dass alle Worker auf nie startende Aufgaben warten.
- Hinweis bei wartender Nachricht wie unter 2.; die Position wird aktualisiert, solange die Nachricht wartet.
- Zusätzlicher Test: tool-workers="1" mit einem READ-Tool, das einen verschachtelten parallelen Batch startet, läuft durch.