{{define "landing"}}
An OpenAI-compatible endpoint in front of self-hosted models.
Point any OpenAI client at this address. The API is the one you already have a client for; only the base URL changes.
{{.BaseURL}}/v1
curl {{.BaseURL}}/v1/chat/completions \
-H "Authorization: Bearer $LLMGW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "MODEL", "messages": [{"role":"user","content":"hello"}]}'
from openai import OpenAI
client = OpenAI(base_url="{{.BaseURL}}/v1", api_key=os.environ["LLMGW_API_KEY"])
GET /v1/models lists what is available to your key. It needs the key,
which is why this page does not list them: this page is served to anyone who can
reach the address.
GET /v1/models lists what is available.
This gateway has no API keys configured, so it serves anyone who can reach it. That is a deployment choice and only safe on a trusted network.
{{end}}Keys are issued by an operator with llmgw keys create, and shown once.
There is no way to recover one: only its hash is stored.