Ilker ManapandClaude Opus 5.5 06baa38858 Traces say model calls in the OpenTelemetry GenAI conventions ...

An inference request and each attempt at an engine carry gen_ai.*: the
operation, the requested and answering model, max_tokens, temperature and
top_p when the client set them, the session as the conversation id, the
input, output and cached tokens, and on an attempt the engine that answered.
A customer's Grafana, Jaeger or Langfuse then shows llmgw's calls as model
calls. No prompt or answer text is ever put in a span, and a test says so.

BACKLOG gains section L: what is worth taking from Kong's AI Gateway, of
which this is the first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-25 12:59:20 +01:00

..

admin-api.md

Organizations, teams, projects, keys and memberships in the admin API

2026-10-08 22:42:43 +02:00

admin-api.openapi.json

The company's MCP tools under the guardrails, a rate and a breaker

2026-10-20 23:00:22 +02:00

choosing-models.md

Manage people and the tenancy tree, from the web and the CLI

2026-09-28 16:26:39 +02:00

client-api.openapi.json

Pictures, 3D models, a vision model and workflows as tools at /mcp

2026-10-14 13:11:09 +02:00

clients.md

Company MCP servers are registered with ctl mcp, and tools granted with keys limits --tools

2026-10-20 12:50:55 +02:00

console.md

A node's console opens on what the machine is doing

2026-10-19 12:55:27 +02:00

data.md

Merge C18 offline bundles and the Helm chart

2026-10-05 00:15:47 +02:00

development.md

Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install

2026-10-04 17:32:54 +02:00

fabrics.md

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

facility-aware-placement.md

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

guardrails.md

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

install.md

Traces say model calls in the OpenTelemetry GenAI conventions

2026-10-25 12:59:20 +01:00

knowledge.md

A band may wait longer than the rest; a busy refusal says when to return

2026-10-10 16:11:09 +02:00

kubernetes.md

Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

2026-10-05 00:13:39 +02:00

large-models.md

A big model's download no longer runs out its startup timeout

2026-10-20 09:45:31 +02:00

node-onboarding.md

Add a node with one command: llmgw node add, with a terminal screen

2026-09-29 19:33:47 +02:00

onboarding.md

VS Code gets the gateway's models from an extension the gateway serves

2026-10-24 01:39:50 +02:00

quick-install.md

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

reference.md

Sign-in is set up from the console, and a directory tried before it is kept

2026-10-19 12:55:27 +02:00