{{define "landing"}} llmgw

llmgw

An OpenAI-compatible endpoint in front of self-hosted models.

Point any OpenAI client at this address. The API is the one you already have a client for; only the base URL changes.

Base URL

{{.BaseURL}}/v1

Example

curl {{.BaseURL}}/v1/chat/completions \
  -H "Authorization: Bearer $LLMGW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "<model-name>", "messages": [{"role":"user","content":"hello"}]}'

Replace <model-name> with a name from GET {{.BaseURL}}/v1/models, which needs your key. Sending the placeholder unchanged is answered with the list of models that are available.

from openai import OpenAI
client = OpenAI(base_url="{{.BaseURL}}/v1", api_key=os.environ["LLMGW_API_KEY"])

Which models

{{if .AuthEnabled}}

GET /v1/models lists what is available to your key. It needs the key, which is why this page does not list them: this page is served to anyone who can reach the address.

{{else}}

GET /v1/models lists what is available.

This gateway has no API keys configured, so it serves anyone who can reach it. That is a deployment choice and only safe on a trusted network.

{{end}}

Getting a key

{{if .SignupOpen}}

Create an account. Someone will approve it, and then you can issue your own keys from your profile — as many as you need, revoking them yourself when a laptop is retired or a script is done.

Already have an account? Sign in.

{{else if .Contact}}

Ask {{if .ContactHref}}{{.Contact}}{{else}}{{.Contact}}{{end}} for one. {{if .AuthEnabled}}If you already have an account, sign in.{{end}}

{{else}}

Keys are issued by whoever operates this gateway. If you do not know who that is, the address you reached this page at is the place to start asking.

{{end}}

A key is shown once and never again — only its hash is kept here — so put it in a password manager straight away.

{{if .Notice}}

{{.Notice}}

{{end}}

Once you have one

{{end}}