API docs

The API is OpenAI-compatible. Point any OpenAI SDK at https://api.inference.town/v1, send your inference.town key, and it works.

The town is optional. The API is the product: one endpoint, cost-aware routing, correct streaming, and honest token accounting.

Quickstart

curl https://api.inference.town/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_TOWN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen/qwen3-235b-a22b-2507","messages":[{"role":"user","content":"Hello!"}]}'

Python

from openai import OpenAI

client = OpenAI(
    base_url="https://api.inference.town/v1",
    api_key="itown_sk_...",
)
stream = client.chat.completions.create(
    model="deepseek/deepseek-v3.2",
    messages=[{"role": "user", "content": "Say hi to the town."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inference.town/v1",
  apiKey: process.env.INFERENCE_TOWN_KEY,
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [{ role: "user", content: "Hello from a tiny town." }],
  stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

Authentication

Send your key as a bearer token or in x-api-key. Keys are hashed before storage; we never keep plaintext. Create and revoke keys in the Key office.

Chat completions

POST /v1/chat/completions accepts the usual OpenAI body: messages, temperature, top_p, max_tokens, tools, response_format, stream, and so on.

Three optional inference.town extensions are additive and ignored by a strict OpenAI client:

Field	Meaning
`models`	Ordered fallback models, tried if the primary fails before the first token.
`routing`	`{ strategy, max_cost_micro, min_margin, cache_affinity }`. Strategy is one of cheapest, fastest, balanced, lowest-latency, reliable.
`provider`	Provider preferences (order, only, ignore, sort, max_price), mirroring the routing vocabulary.

Every response carries a routing header and, in the body, an x_inference_town object with the generation id, served model, provider, attempts, TTFT and usage.

Image generation

POST /v1/images/generations returns an OpenAI-shaped image response. Image models are billed per image, not per token.

curl https://api.inference.town/v1/images/generations \
  -H "Authorization: Bearer $INFERENCE_TOWN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"black-forest-labs/flux-schnell",
       "prompt":"a tiny fantasy town at dusk",
       "n":1, "response_format":"url"}'

Available image models include FLUX.1 Schnell and Dev, Stable Diffusion 3.5 Large and Qwen Image. See the market board for live per-image prices.

Streaming

Set "stream": true for Server-Sent Events. We pass chunks through immediately without buffering and add no artificial delay. A final usage frame is always included, then data: [DONE].

We begin streaming the first upstream byte the instant it arrives. Middleware adds only the time to authenticate, pick a route and open the upstream connection.

Models API

GET /v1/models returns an OpenAI-shaped list. GET /v1/models/{"{id}"} returns one model with context length, modalities, capabilities and generated pricing. The market board uses the same source.

Routing & fallbacks

For each model we keep several endpoints. The router scores them on expected cost, latency, reliability and cache affinity, enforces a margin floor, and holds the rest as fallbacks. If a provider fails before generation starts we retry the next candidate. Once tokens have streamed, we never silently switch — the committed response is the committed response.

Strategy	Optimizes
`cheapest`	Lowest expected upstream cost.
`fastest`	Best measured throughput.
`lowest-latency`	Lowest time to first token.
`reliable`	Highest recent success rate.
`balanced` (default)	Weighted price × latency × reliability.

Errors

Errors use the OpenAI envelope: { "error": { message, type, param, code } }. Codes are stable strings such as invalid_request, authentication, insufficient_credits, payment_required, rate_limit_exceeded, no_route, provider_error and timeout.

Payments (x402)

Funding an account is a one-time payment that becomes a credit balance. Inference calls debit that balance locally and settle in batches — we never put a blockchain in your request path.

Request a top-up for an amount: POST /api/topups.
Receive a 402-style requirement (asset, network, payTo, amount) in the body and the PAYMENT-REQUIRED header.
Pay with an x402-capable client and post the proof to /api/topups/{id}/verify (or retry with the PAYMENT-SIGNATURE header).
We verify, settle once, credit the balance idempotently, and mint an API key.

curl -X POST https://api.inference.town/api/topups \
  -H "Content-Type: application/json" \
  -d '{"amount_usd": 10}'

Prefer a familiar flow? Add funds from the Payment booth and use a balance-backed key. Both paths debit the same ledger. Your management token stays in your browser; the API key is shown once.

Providers

We buy inference from multiple providers and route between them. Providers are a modular adapter layer, so adding one is a config change plus, at most, a small normalizer. The public status page shows provider health by name; we never expose internal topology or your traffic.

Security

Keys are hashed at rest and can be scoped and rate-limited.
Upstream provider credentials live only in Cloudflare secrets and never reach a client.
Credit debits are atomic and replay-protected; payment proofs are idempotent by network, transaction, payer and resource.
Payload size and token limits are enforced before we contact a provider.

Privacy

We do not retain prompts or completions by default. Usage records contain token counts, cost, latency and model/provider — not message content. The town shows lagged, quantized, anonymized aggregates and never individual activity.