{{define "landing"}} llmgw

llmgw

An OpenAI-compatible endpoint in front of self-hosted models.

Point any OpenAI client at this address. The API is the one you already have a client for; only the base URL changes.

Base URL

{{.BaseURL}}/v1

Example

curl {{.BaseURL}}/v1/chat/completions \
  -H "Authorization: Bearer $LLMGW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "MODEL", "messages": [{"role":"user","content":"hello"}]}'
from openai import OpenAI
client = OpenAI(base_url="{{.BaseURL}}/v1", api_key=os.environ["LLMGW_API_KEY"])

Which models

{{if .AuthEnabled}}

GET /v1/models lists what is available to your key. It needs the key, which is why this page does not list them: this page is served to anyone who can reach the address.

{{else}}

GET /v1/models lists what is available.

This gateway has no API keys configured, so it serves anyone who can reach it. That is a deployment choice and only safe on a trusted network.

{{end}}

Keys

Keys are issued by an operator with llmgw keys create, and shown once. There is no way to recover one: only its hash is stored.

{{end}}