01 · Throughput
Built around the workload
Concurrency, batch scheduling, context length, and cache policy are tested against interactive and balanced production shapes.
Measured inference infrastructure
More useful tokens.
Less waiting.
Token Labs serves open models through an OpenAI-compatible API engineered for sustained throughput, predictable latency, and transparent capacity—ready for OpenRouter provider integration.
98.68output tokens/s · balanced SLO-safe load
1.24sTTFT p95 · balanced concurrency 4
40.15msstreaming TPOT/ITL p95
429 earlybounded admission, not hidden queues
Why Token Labs
Inference you can measure and trust.
We tune the complete request path, then publish the evidence behind every capacity decision.
02 · Latency
Tail metrics set the limit
Peak token rate is not the launch number. Admission is set by TTFT and streaming TPOT/ITL guardrails.
03 · Transparency
Evidence, not adjectives
Validated exports, exact configurations, rejected experiments, and topology limitations stay available.
Read the methodology →Launch candidate
A focused model offering, tuned deeply.
One model, one clearly measured capacity envelope, and an API surface designed for provider traffic.
Qwen3 30B A3B Instruct 2507 FP8
A sparse mixture-of-experts instruction model selected for capability, memory fit, serving terms, and single-DGX-Spark throughput.
32Kconfigured context
FP8weight quantization
4verified global concurrency
8interactive-only concurrency
Evidence trail
Research lives behind the product story.
Published studies, single-replica controls, partial work, and operational notes now have dedicated destinations.
Trust center
Your prompts are for inference—not training.
Our policies explain what is processed, what metadata may be retained, and how capacity is handled.
Inference data
Prompts and completions are processed transiently, are not intentionally retained after completion, and are not used for training.
Provider documentation
Authentication, bounded admission, model discovery, service terms, and policy documentation are maintained for onboarding.
Measured for the traffic you send.
Talk to Token Labs about OpenRouter provider integration, capacity verification, or a model-serving workload.