Commit Graph
2 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 b986ea180d Fixes from review: setup's memory check, MCP secrets, VS Code, bandwidth
CI / test (push) Successful in 9m47s
- server init run again on a machine already set up is not refused its
  memory (the docs fix a certificate with --host so), and judges count
  only when one will be installed (judgeComes: not without a mirror, nor
  from a bundle without it).
- A server's sealed secrets that it no longer reads are removed when it is
  put or deleted; replacing a credential replaces every header it was
  sent, rather than adding one beside the old.
- llmgw.gatewayUrl is a machine setting and restricted in untrusted
  workspaces: a cloned repository's settings cannot send a sign-in to
  another gateway.
- The bandwidth buffer is at least eight times the CPU's last cache and at
  most a quarter of free memory, else nothing is measured until there is
  room; a registration carries the figure measured so far.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 08:23:00 +01:00
Ilker ManapandClaude Opus 5.5 84444e71c8 How fast a model would answer on a CPU, before it is fetched
CI / test (push) Failing after 6m37s
Each agent measures its node's memory read rate when it starts: every CPU
sums a share of a buffer past any cache, a tenth of free memory at most,
the best of four passes, in a fraction of a second; reported in Host
(mem_bandwidth_gbs) and NodeView. A token on a CPU reads every weight it
uses once, so one reply's speed is about that rate over those bytes:
modelplan.CPUTokensPerSecond, 0.68 of the measured rate plus 5 ms a token,
calibrated on the PoC against llama.cpp's predicted_per_second (node01
24.9 GB/s, qwen2.5-1.5b: 14.5 measured, 14.7 estimated; node05 41.9 GB/s,
qwen2.5-14b: 3.3 and 3.2). A model of experts named …-A3B reads its active
part (ActiveShare).

The explorer shows it as CPU tok/s, at Q4_K_M on the fastest node that
holds the model; a repository's page gives it for each GGUF by its size.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 00:09:47 +01:00