{{define "landing"}} llmgw
{{if .Notice}}{{end}} {{if not .AuthEnabled}} {{end}}

llmgw

One OpenAI-compatible endpoint in front of your own GPUs.

Point any OpenAI client at this address. The API is the one you already have a client for — only the base URL changes.

{{if .SignupOpen}}Create an account Sign in {{else if .AuthEnabled}}Sign in{{end}}

{{/* A schematic of what this is, not of what is installed here. It names no model, no node and no count: this page is served to anyone who can reach the address. */}}
your client editor agent a script llmgw one address fabric fabric
Clients see one address. Work is placed on GPUs that can actually run it — and a model split across machines never straddles two fabrics.
Base URL {{.BaseURL}}/v1

Three steps

  1. 1

    Get a key

    {{if .SignupOpen}}

    Create an account. Someone approves it, and then you issue your own keys from your profile — as many as you need, revoking them yourself when a laptop is retired or a script is done.

    {{else if .Contact}}

    Ask {{if .ContactHref}}{{.Contact}}{{else}}{{.Contact}}{{end}} for one.{{if .AuthEnabled}} If you already have an account, sign in.{{end}}

    {{else}}

    Keys are issued by whoever operates this gateway. If you do not know who that is, the address you reached this page at is the place to start asking.

    {{end}}

    A key is shown once and never again — only its hash is kept here — so put it in a password manager straight away.

  2. 2

    Point your client at it

    Set the base URL above and your key. Every OpenAI SDK, and every tool built on one, takes both as configuration.

    export OPENAI_BASE_URL={{.BaseURL}}/v1
export OPENAI_API_KEY=$LLMGW_API_KEY
  3. 3

    Pick a model

    {{if .AuthEnabled}}

    GET /v1/models lists what is available to your key. It needs the key, which is why this page does not list them: this page is served to anyone who can reach the address.

    {{else}}

    GET /v1/models lists what is available.

    {{end}}

    Sending the placeholder <model-name> unchanged is answered with the list of models you may use, rather than with an error.

Connect your client

curl {{.BaseURL}}/v1/chat/completions \
  -H "Authorization: Bearer $LLMGW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "<model-name>",
       "messages": [{"role": "user", "content": "hello"}]}'
import os
from openai import OpenAI

client = OpenAI(base_url="{{.BaseURL}}/v1",
                api_key=os.environ["LLMGW_API_KEY"])

reply = client.chat.completions.create(
    model="<model-name>",
    messages=[{"role": "user", "content": "hello"}],
)
print(reply.choices[0].message.content)
// ~/.config/opencode/opencode.json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llmgw": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "llmgw",
      "options": {
        "baseURL": "{{.BaseURL}}/v1",
        "apiKey": "{env:LLMGW_API_KEY}"
      },
      "models": { "<model-name>": { "name": "<model-name>" } }
    }
  }
}

Anything that speaks the chat-completions API works, whether or not it is listed here — there is nothing llmgw-specific to install.

Once you have a key

{{end}}