Jobs are served by the leader, and a moment of fitting nowhere is waited out
CI / test (push) Failing after 8m47s
CI / test (push) Failing after 8m47s
A 3D job leased by the standby control plane, whose router has no endpoints and which a job replayed in the process never leaves, was told "loading" for seven minutes while the leader served the jobs behind it. Jobs and batches are claimed only while leading. Five jobs asked for while cuda01's agent restarted failed for good as "cannot be placed": re-registering, the node had no engines for a few seconds. A job asks again for five minutes before that is its end. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
9b8ec89247
commit
95c61daf1b
6 files changed
+107
-3
No files matched your search
+4
-1
@@ -2086,7 +2086,10 @@ curl -X POST http://gateway:8080/v1/3d/generations/job_…/cancel -H "Authorizat
|
||||
- **How it is served:** the request is kept as it was sent and replayed by
|
||||
whichever gateway holds the job's lease, through the same path a
|
||||
client's request takes — limits, admission in the key's band, guardrails,
|
||||
accounting. It is one usage record under the job's id, tagged
|
||||
accounting — by the leading control plane, since a standby has nothing
|
||||
to send it to. A model said to fit nowhere is asked for again for five
|
||||
minutes before that is the job's error: an agent re-registering says so
|
||||
for a few seconds. It is one usage record under the job's id, tagged
|
||||
`job=<id>`, charged like any request; `GET` shows the usage once it is
|
||||
done. A model still loading, a rate limit or a full queue is waited out
|
||||
for as long as the job has: nobody is holding a connection, which is the
|
||||
|
||||
Reference in new issue
Block a user