Fix an agent that could neither explain itself nor stop
CI / test (push) Successful in 54s

Two bugs found while verifying mTLS against real machines.

session cancelled its own derived context and then asked that context
whether the agent was shutting down. The answer was always yes, so every
stream error read as a clean close: the log said err=<nil>, the backoff
reset, and an agent whose certificate the control plane refused hammered it
forever at the base interval with nothing to diagnose. It now keeps the
parent context to answer that question, and a refusal reaches the operator
naming both the certificate and the claim.

Cancelling the context also did not stop the agent. HTTP/2 watches the
context while the request body is being written, and a bidirectional stream
spends its life blocked on the read side instead, so the read never returned.
Measured on a deployed node: `systemctl stop llmgw-agent` took 90 seconds and
ended in SIGKILL, on every restart. Closing the response when the context
ends is what actually unblocks it.

TestAgentStopsWhenContextEnds hangs reliably without the second fix, which is
the only reason to trust that it tests it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 08:03:21 +02:00
1 parent 1e3cafdc86
commit ec1b6b04df
9 files changed
+515 -27

No files matched your search

+15
View File
@@ -7,6 +7,21 @@
node_id: gpu-a-01 # defaults to the hostname
server: "http://llmgw-server.internal:7443"
# The agent's half of mutual TLS. The certificate's common name must be this
# node's id, because the control plane compares the two and refuses the stream
# when they disagree:
#
# llmgw ctl cert issue-client gpu-a-01 --dir /etc/llmgw/pki
#
# With this set, `server` above must be an https:// address — an agent holding
# certificates and dialling http:// is refused at startup rather than connecting
# in the clear without noticing.
#
# tls:
# ca: /etc/llmgw/ca.pem
# cert: /etc/llmgw/agent.pem
# key: /etc/llmgw/agent-key.pem
# The address engines are reachable on, and the address NCCL bootstraps over.
# Never a fabric address: NCCL rendezvouses over TCP before moving to RDMA.
mgmt_addr: "10.0.0.11"