Ilker ManapandClaude Opus 5
1839832b8b
CI / test (push) Successful in 1m2s
Pasting the gateway's address into a browser returned 404, which looks like a broken endpoint rather than one that tells you how to use it. Two pages, and the split between them is a security boundary rather than a layout choice. The landing page is served without a key — it has to be, because the person reading it is the one who does not have one yet — so it contains no data at all: it says how to connect, not what is here. Model names stay behind /v1/models. It reconstructs the address from the forwarded headers so the copy-pasteable example names the address the client actually used rather than the internal one; those headers are reflected into displayed text and never used for a decision. The dashboard shows nodes, models, replicas, placement failures and usage, and lives on the admin listener only. That listener is loopback-bound and also serves pprof, which can dump the process heap — where API keys sit in plaintext — so reaching the page from elsewhere is an SSH tunnel's job. It is read-only: anything that mints a credential stays in `llmgw keys`, where it needs a shell rather than a browser session this project has no way to authenticate. Mounting the landing page uncovered a hole in the exemption list. An entry ending in "/" was a prefix rule, and "/" ends in "/", so exempting the landing page would have exempted every path there is — authentication still installed, still logging, and letting the whole API through. "/" is now exact, with a test enumerating what must and must not be exempt. Server-rendered templates and one stylesheet, embedded. The single-binary goal rules out a JavaScript toolchain and nothing on either page needs one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
43 lines
1.6 KiB
HTML
43 lines
1.6 KiB
HTML
{{define "landing"}}<!doctype html>
|
|
<html lang="en"><head>
|
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
|
<title>llmgw</title><link rel="stylesheet" href="/_static/base.css">
|
|
</head><body><div class="wrap">
|
|
<header>
|
|
<h1>llmgw</h1>
|
|
<p class="sub">An OpenAI-compatible endpoint in front of self-hosted models.</p>
|
|
</header>
|
|
|
|
<p>Point any OpenAI client at this address. The API is the one you already have a
|
|
client for; only the base URL changes.</p>
|
|
|
|
<h2>Base URL</h2>
|
|
<pre>{{.BaseURL}}/v1</pre>
|
|
|
|
<h2>Example</h2>
|
|
<pre>curl {{.BaseURL}}/v1/chat/completions \
|
|
-H "Authorization: Bearer $LLMGW_API_KEY" \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model": "MODEL", "messages": [{"role":"user","content":"hello"}]}'</pre>
|
|
|
|
<pre>from openai import OpenAI
|
|
client = OpenAI(base_url="{{.BaseURL}}/v1", api_key=os.environ["LLMGW_API_KEY"])</pre>
|
|
|
|
<h2>Which models</h2>
|
|
{{if .AuthEnabled}}
|
|
<p><code>GET /v1/models</code> lists what is available to your key. It needs the key,
|
|
which is why this page does not list them: this page is served to anyone who can
|
|
reach the address.</p>
|
|
{{else}}
|
|
<p><code>GET /v1/models</code> lists what is available.</p>
|
|
<p class="note">This gateway has no API keys configured, so it serves anyone who can
|
|
reach it. That is a deployment choice and only safe on a trusted network.</p>
|
|
{{end}}
|
|
|
|
<h2>Keys</h2>
|
|
<p>Keys are issued by an operator with <code>llmgw keys create</code>, and shown once.
|
|
There is no way to recover one: only its hash is stored.</p>
|
|
|
|
<footer><span>llmgw {{.Version}}</span><span>{{.Now}}</span></footer>
|
|
</div></body></html>{{end}}
|