OBSERVABILITY
Behind each response.
A closer look at serving performance.
Dashboards open in Grafana and may require sign-in.
Inference
Token throughput, latency, and KV transfer tokens.
GPU metrics
Utilization, temperature, and power consumption.
Service health
Worker scrape availability, request errors, and latency.