Show a node's logs on the admin page
Seeing what a node did meant ssh through a jump host, to a machine whose address nobody remembers, to read a journal only root may open. When an engine install failed on a new node today, the reason was one line — and it was four steps away from the page that said the install had failed. Each node's row on the cluster page now links to its logs: the agent's own, and every engine running there, chosen from the list the node itself sends. The page can follow, refreshing every five seconds, and is on both the admin listener and /admin. `llmgw ctl logs NODE [--source SHARD] [-n N]` reads the same. They travel over the stream the agent already holds, so they need nothing the control plane does not already have: a request goes down, the lines come back, and a reply that does not come in ten seconds is an agent too old to answer. The agent keeps its last 2000 lines in memory as well as writing them wherever it did before; the engines' last few hundred lines were already kept and sent only when one crashed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a821a05840
commit
33e678ab84
18 files changed
+812
-43
No files matched your search
@@ -203,6 +203,7 @@ func newServerCmd() *cobra.Command {
|
||||
api, usage, version.Version, 5*time.Second, true,
|
||||
web.NavExtras{History: history, Window: cfg.Monitoring.Window}, log))
|
||||
adminMux.Handle("GET /_static/", web.Static())
|
||||
adminMux.HandleFunc("GET /nodes/{id}/logs", web.NodeLogsPage(api, true, version.Version))
|
||||
|
||||
// The same page on the client-facing port at /admin, behind a
|
||||
// session and an admin flag. Handing the router the data rather
|
||||
@@ -466,6 +467,21 @@ func (r *dispatcherRef) RemoveWeights(ctx context.Context, nodeID string, paths
|
||||
return rm.RemoveWeights(ctx, nodeID, paths)
|
||||
}
|
||||
|
||||
// Logs forwards a node log request, the same way RemoveWeights does: the
|
||||
// controller finds it by type assertion, so a forwarder without it would make
|
||||
// the capability silently absent at runtime.
|
||||
func (r *dispatcherRef) Logs(ctx context.Context, nodeID, source string, lines int) ([]string, []string, error) {
|
||||
d, err := r.get()
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
lr, ok := d.(control.LogReader)
|
||||
if !ok {
|
||||
return nil, nil, errors.New("this dispatcher cannot read node logs")
|
||||
}
|
||||
return lr.Logs(ctx, nodeID, source, lines)
|
||||
}
|
||||
|
||||
func (r *dispatcherRef) StartShard(ctx context.Context, cmd reconciler.ShardCommand) error {
|
||||
d, err := r.get()
|
||||
if err != nil {
|
||||
|
||||
Reference in new issue
Block a user