This website requires JavaScript.
Explore
Help
Sign In
ilker
/
llmgw
Watch
2
Star
0
Fork
1
Code
Issues
Pull Requests
Actions
Packages
Projects
Releases
96
Wiki
Activity
Files
587d8d5059d2e95cba28934cd049a11f74882264
llmgw
/
internal
/
router
T
History
Ilker Manap
and
Claude Opus 5.5
587d8d5059
CI / test (push)
Failing after 6m30s
Details
A licence left out by its name stays out once its entry gives an id
...
Co-Authored-By: Claude Opus 5.5 (1M context) <
noreply@anthropic.com
>
2026-10-11 16:28:40 +02:00
..
admission
A band may wait longer than the rest; a busy refusal says when to return
2026-10-10 16:11:09 +02:00
breaker
Phase 1 (1/2): scheduler pipeline, endpoint registry, metric scraping
2026-09-16 01:15:21 +02:00
endpoint
Pictures from ComfyUI at /v1/images/generations
2026-10-09 21:15:50 +02:00
kvindex
Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held
2026-10-05 01:21:11 +02:00
prefix
Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held
2026-10-05 01:21:11 +02:00
proxy
A ComfyUI model may make meshes: from a picture, with its licence inside
2026-10-10 18:30:52 +02:00
scheduler
Merge C24 KV-index routing from vLLM's cache events
2026-10-05 01:24:25 +02:00
scrape
Pictures from ComfyUI at /v1/images/generations
2026-10-09 21:15:50 +02:00
activate.go
A job waiting for the card says what holds it and about how long
2026-10-10 22:25:27 +02:00
adapter_test.go
LoRA adapters on vLLM, loaded on demand with replica affinity: max_loaded_adapters
2026-10-05 01:14:38 +02:00
adapterowner_test.go
Trained adapters published, private to their owners like the knowledge bases they come from
2026-10-08 21:30:50 +02:00
admin_test.go
Give metrics a door that does not also open onto pprof
2026-09-17 18:12:10 +02:00
admin.go
Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held
2026-10-05 01:21:11 +02:00
adminauth_test.go
Guard the admin API and pprof with a token only root and the server can read
2026-10-04 10:02:44 +02:00
adminauth.go
Guard the admin API and pprof with a token only root and the server can read
2026-10-04 10:02:44 +02:00
adminguard_test.go
Stop a browser page, or a model definition, from running code on the nodes
2026-09-30 00:29:52 +02:00
adminguard.go
Stop a browser page, or a model definition, from running code on the nodes
2026-09-30 00:29:52 +02:00
audio.go
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
2026-10-04 22:56:25 +02:00
batches_test.go
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
2026-10-04 21:00:01 +02:00
batches.go
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
2026-10-04 21:00:01 +02:00
batchworker.go
Jobs are served by the leader, and a moment of fitting nowhere is waited out
2026-10-10 23:53:33 +02:00
budget_fallback.go
A batch of embeddings is checked text by text against the model's context
2026-10-09 11:43:21 +02:00
cache_test.go
Reusing answers: a response cache per project, exact match, opt-in with a TTL
2026-10-04 23:50:53 +02:00
cache.go
Collections a part of the tree answers from when a request names none
2026-10-08 15:52:43 +02:00
cachestream.go
Reusing answers: a response cache per project, exact match, opt-in with a TTL
2026-10-04 23:50:53 +02:00
canary_test.go
Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared
2026-10-04 19:56:16 +02:00
canary.go
Canary routing: a model's canary takes its share of requests, a session stays on one side, x-llmgw-variant pins one, and the two are compared
2026-10-04 19:56:16 +02:00
chat_load_test.go
Show a model loading in the chat, and send once it is up
2026-09-28 20:52:31 +02:00
chat_load.go
Show a model loading in the chat, and send once it is up
2026-09-28 20:52:31 +02:00
chat_models_test.go
The portal's chat answers from the documents a person picks
2026-10-08 15:59:23 +02:00
chat.go
A person leaves licences out, and no key of theirs reaches a model under one
2026-10-11 16:10:22 +02:00
compat_test.go
Moderations name a judge's categories once, by name
2026-10-04 23:03:35 +02:00
context_limit_test.go
Know how big a model's context is, and what it can do
2026-09-16 23:07:49 +02:00
context_limit.go
Know how big a model's context is, and what it can do
2026-09-16 23:07:49 +02:00
docs_test.go
A docs portal at /docs and a client developer's guide
2026-10-04 23:46:42 +02:00
external_test.go
Serve a model from an external OpenAI-compatible provider
2026-09-30 23:29:01 +02:00
federation_test.go
Federation: another llmgw cluster as a provider, its models as PEER/MODEL
2026-10-05 01:21:30 +02:00
federation.go
Federation: another llmgw cluster as a provider, its models as PEER/MODEL
2026-10-05 01:21:30 +02:00
files.go
Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
2026-10-04 21:00:01 +02:00
finish_test.go
A band may wait longer than the rest; a busy refusal says when to return
2026-10-10 16:11:09 +02:00
guard_test.go
Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
2026-10-06 12:41:08 +02:00
guard.go
Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
2026-10-06 12:41:08 +02:00
guardmetrics.go
Guardrail metrics and alerts: checks, latency, judge failures, fail-open, redactions, stopped streams, tenant blocks
2026-10-04 15:11:50 +02:00
handlers.go
A person leaves licences out, and no key of theirs reaches a model under one
2026-10-11 16:10:22 +02:00
hardware_leader_test.go
Add a standby control plane with server add, and draw the screens as windows
2026-10-03 22:29:12 +02:00
hardware_test.go
Say on the chat page whether a GPU or a CPU answered
2026-10-03 18:57:50 +02:00
images_test.go
G4: a page that makes 3D models, and ctl 3d
2026-10-10 19:16:20 +02:00
images.go
Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
2026-10-11 14:28:15 +02:00
jobs_test.go
Jobs are served by the leader, and a moment of fitting nowhere is waited out
2026-10-10 23:53:33 +02:00
jobs.go
A person leaves licences out, and no key of theirs reaches a model under one
2026-10-11 16:10:22 +02:00
jobworker.go
Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
2026-10-11 14:28:15 +02:00
judge.go
A 3D model is a job: queued with an id, asked after, its result a file
2026-10-10 16:04:11 +02:00
kb_e2e_test.go
llmgw-kb: sync a person's documents into a knowledge base from their own machine
2026-10-08 13:43:47 +02:00
kb_test.go
llmgw-kb collections names each collection's owner as people know it
2026-10-08 15:42:40 +02:00
kb.go
llmgw-kb collections names each collection's owner as people know it
2026-10-08 15:42:40 +02:00
kvroute_test.go
KV routing tests wait for both replicas to be scraped: a replica not yet heard from is passed over, and the test was racing that
2026-10-05 01:27:14 +02:00
kvroute.go
Precise KV-index routing: vLLM KV cache events through the agent, routed by leading blocks held
2026-10-05 01:21:11 +02:00
language_test.go
The portal in English and Turkish: a catalogue per language, a switcher at the foot of every page, landing.language, Turkish numbers and dates
2026-10-05 00:12:53 +02:00
licences_test.go
A licence left out by its name stays out once its entry gives an id
2026-10-11 16:28:40 +02:00
licences.go
A licence left out by its name stays out once its entry gives an id
2026-10-11 16:28:40 +02:00
lora_test.go
LoRA adapters on vLLM, loaded on demand with replica affinity: max_loaded_adapters
2026-10-05 01:14:38 +02:00
lora.go
LoRA adapters on vLLM, loaded on demand with replica affinity: max_loaded_adapters
2026-10-05 01:14:38 +02:00
memory.go
no_store: refuse conversations, notes and models outside the cluster under a policy that keeps nothing
2026-10-04 12:36:02 +02:00
meshes_test.go
A picture is made ready for a 3D model; a model asleep is listed asleep
2026-10-10 20:43:20 +02:00
meshes.go
G4: a page that makes 3D models, and ctl 3d
2026-10-10 19:16:20 +02:00
messages_test.go
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
2026-10-04 22:59:55 +02:00
messages.go
Anthropic's Messages API: /v1/messages translated to chat completions and back, streamed or not, with tools and images; keys in x-api-key
2026-10-04 22:59:55 +02:00
metrics.go
Chat requests answered from knowledge bases
2026-10-08 15:27:50 +02:00
moderations.go
Moderations name a judge's categories once, by name
2026-10-04 23:03:35 +02:00
objglb.go
A job says how far along it is; a ComfyUI model brings its custom nodes
2026-10-10 18:46:44 +02:00
pages_auth_test.go
The client API described in OpenAPI, and both APIs in Swagger UI
2026-10-10 20:22:48 +02:00
pricing_test.go
Publish per-tenant request and budget series, and alert on them
2026-10-03 19:55:24 +02:00
quality_test.go
Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate
2026-10-05 00:04:43 +02:00
rag_test.go
An embedding model's query_prefix, put before questions and never before documents
2026-10-09 18:03:00 +02:00
rag.go
An embedding model's query_prefix, put before questions and never before documents
2026-10-09 18:03:00 +02:00
requestlog_test.go
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
2026-10-04 22:51:21 +02:00
requestlog.go
Reusing answers: a response cache per project, exact match, opt-in with a TTL
2026-10-04 23:50:53 +02:00
residency_test.go
Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster
2026-10-04 23:27:31 +02:00
responses_test.go
Hook a guard into the router: judge requests before dispatch and filter answers as they are relayed
2026-10-04 12:20:24 +02:00
responses.go
More of OpenAI's API: Responses structured output, images and previous_response_id; rerank, score and audio passed through; moderations by the judge
2026-10-04 22:56:25 +02:00
router_listen_test.go
Release v0.10.4: CPU threads, tracing, and the rest of the review
2026-09-30 04:00:35 +02:00
router_test.go
A job waiting for the card says what holds it and about how long
2026-10-10 22:25:27 +02:00
router.go
Jobs are served by the leader, and a moment of fitting nowhere is waited out
2026-10-10 23:53:33 +02:00
tags_test.go
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
2026-10-04 19:45:37 +02:00
tags.go
A job waiting for the card says what holds it and about how long
2026-10-10 22:25:27 +02:00
tenantmetrics.go
Requests can be tagged: metadata, x-llmgw-tags and the user field on the usage record, and counted by the tags router.tag_labels names
2026-10-04 19:45:37 +02:00
threadid_test.go
Guard a GPU-less node's problems, and bound thread ids
2026-10-04 10:49:10 +02:00
threads_test.go
Stop the text-search comment naming one language
2026-09-27 08:09:49 +02:00
threads.go
Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay
2026-10-04 22:51:21 +02:00
tokenmetrics_test.go
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
2026-10-04 23:13:18 +02:00
tokenmetrics.go
Token-level timing and admission by SLO: TPOT, inter-token and tokens a second by model and tenant; a model past its SLO is saturated for the lower bands
2026-10-04 23:13:18 +02:00
trace_test.go
Release v0.10.4: CPU threads, tracing, and the rest of the review
2026-09-30 04:00:35 +02:00
trace.go
Release v0.10.4: CPU threads, tracing, and the rest of the review
2026-09-30 04:00:35 +02:00
workflows_test.go
Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
2026-10-11 14:28:15 +02:00
workflows.go
Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
2026-10-11 14:28:15 +02:00