big-model runs on the sim engine, whose whole vocabulary is the Greek alphabet, cycled at a fixed rate until max_tokens. That is what it is for: with no H100s it is the only thing exercising the TP8 two-node path, the fabric invariant and rebuild-after-node-loss. The output is the point. What was wrong is that nothing said so. /v1/models returned it with owned_by "llmgw", identical to the real model, and the chat picker sorted purely by name — so "big-model" led the list, was preselected by the browser, and anyone who typed a question and pressed Send got Greek letters back with no explanation. Reported as the model spitting out crap, which is a fair reading of what it looks like from outside. Simulated is derived from the profiles rather than configured: an operator who adds a sim profile and forgets a flag would be shipping placeholder text under a real model's name. A model counts as simulated only when every profile is, because one real profile means the request can still come back as inference. The marker on /v1/models is owned_by "llmgw-sim", which is already what the sim engine calls itself on its own /v1/models — the gateway was flattening a distinction that existed one layer down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cbf276a41b
commit
72b03a1a1c
8 files changed
+195
-14
No files matched your search
@@ -20,11 +20,16 @@
|
||||
{{if .Models}}
|
||||
<div class="chat-bar">
|
||||
<label class="inline-label">Model
|
||||
<select id="model">{{range .Models}}<option value="{{.}}">{{.}}</option>{{end}}</select>
|
||||
<select id="model">{{range .Models}}<option value="{{.Name}}" data-simulated="{{if .Simulated}}1{{else}}0{{end}}">{{.Name}}{{if .Simulated}} (simulated){{end}}</option>{{end}}</select>
|
||||
</label>
|
||||
<button id="reset" type="button" class="link">New conversation</button>
|
||||
</div>
|
||||
|
||||
<p id="sim-note" class="note" hidden>
|
||||
This model is simulated: it exercises scheduling and reports timings, and its
|
||||
replies are placeholder text, not inference.
|
||||
</p>
|
||||
|
||||
<div id="log" class="chat-log" aria-live="polite"></div>
|
||||
|
||||
<form id="composer" class="chat-composer">
|
||||
@@ -62,6 +67,16 @@ access to it.</p>
|
||||
|
||||
document.getElementById("reset").onclick = () => { messages = []; log.textContent = ""; };
|
||||
|
||||
// The note follows the picker rather than the reply: someone should know what
|
||||
// they are about to get before they spend a minute waiting for it.
|
||||
const simNote = document.getElementById("sim-note");
|
||||
function showSimNote() {
|
||||
const opt = model.selectedOptions[0];
|
||||
simNote.hidden = !opt || opt.dataset.simulated !== "1";
|
||||
}
|
||||
model.addEventListener("change", showSimNote);
|
||||
showSimNote();
|
||||
|
||||
function bubble(role) {
|
||||
const el = document.createElement("div");
|
||||
el.className = "bubble " + role;
|
||||
|
||||
Reference in new issue
Block a user