Two bugs found while verifying mTLS against real machines. session cancelled its own derived context and then asked that context whether the agent was shutting down. The answer was always yes, so every stream error read as a clean close: the log said err=<nil>, the backoff reset, and an agent whose certificate the control plane refused hammered it forever at the base interval with nothing to diagnose. It now keeps the parent context to answer that question, and a refusal reaches the operator naming both the certificate and the claim. Cancelling the context also did not stop the agent. HTTP/2 watches the context while the request body is being written, and a bidirectional stream spends its life blocked on the read side instead, so the read never returned. Measured on a deployed node: `systemctl stop llmgw-agent` took 90 seconds and ended in SIGKILL, on every restart. Closing the response when the context ends is what actually unblocks it. TestAgentStopsWhenContextEnds hangs reliably without the second fix, which is the only reason to trust that it tests it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
1e3cafdc86
commit
ec1b6b04df
9 files changed
+515
-27
No files matched your search
@@ -7,6 +7,21 @@
|
||||
node_id: gpu-a-01 # defaults to the hostname
|
||||
server: "http://llmgw-server.internal:7443"
|
||||
|
||||
# The agent's half of mutual TLS. The certificate's common name must be this
|
||||
# node's id, because the control plane compares the two and refuses the stream
|
||||
# when they disagree:
|
||||
#
|
||||
# llmgw ctl cert issue-client gpu-a-01 --dir /etc/llmgw/pki
|
||||
#
|
||||
# With this set, `server` above must be an https:// address — an agent holding
|
||||
# certificates and dialling http:// is refused at startup rather than connecting
|
||||
# in the clear without noticing.
|
||||
#
|
||||
# tls:
|
||||
# ca: /etc/llmgw/ca.pem
|
||||
# cert: /etc/llmgw/agent.pem
|
||||
# key: /etc/llmgw/agent-key.pem
|
||||
|
||||
# The address engines are reachable on, and the address NCCL bootstraps over.
|
||||
# Never a fabric address: NCCL rendezvouses over TCP before moving to RDMA.
|
||||
mgmt_addr: "10.0.0.11"
|
||||
|
||||
@@ -9,10 +9,33 @@
|
||||
listen: ":8080" # client-facing OpenAI API
|
||||
admin_listen: ":9090" # metrics, health, pprof — do not expose this publicly
|
||||
# Where node agents connect. Separate from the client port so the fleet-facing
|
||||
# surface can be firewalled independently. Agents speak plaintext HTTP/2 today,
|
||||
# so this must stay on a trusted network until mTLS lands (plan §5.6).
|
||||
# surface can be firewalled independently.
|
||||
agent_listen: ":7443"
|
||||
|
||||
# Mutual TLS on the agent port (plan §5.6). With it, a node's identity is taken
|
||||
# from its certificate's common name and an agent claiming a different node id
|
||||
# is refused. Without it the port is plaintext HTTP/2 and anyone who can reach
|
||||
# it can claim to be any node, so it must stay on a trusted network.
|
||||
#
|
||||
# These certificates come from llmgw's own authority, not a public one: public
|
||||
# authorities stopped issuing the clientAuth usage, so no Let's Encrypt
|
||||
# certificate can authenticate an agent.
|
||||
#
|
||||
# llmgw ctl cert init-ca --dir /etc/llmgw/pki
|
||||
# llmgw ctl cert issue-server llmgw-ctl --host 10.0.0.140 --dir /etc/llmgw/pki
|
||||
#
|
||||
# Every address an agent might dial has to appear in --host, bare IP addresses
|
||||
# included: hostname verification reads subject alternative names and ignores
|
||||
# the common name entirely.
|
||||
#
|
||||
# This covers the fleet-facing port only. The client-facing API above is public
|
||||
# and belongs behind a publicly trusted certificate.
|
||||
#
|
||||
# agent_tls:
|
||||
# ca: /etc/llmgw/ca.pem
|
||||
# cert: /etc/llmgw/server.pem
|
||||
# key: /etc/llmgw/server-key.pem
|
||||
|
||||
router:
|
||||
scrape_interval: 2s
|
||||
scrape_timeout: 1s
|
||||
|
||||
Reference in new issue
Block a user