Open Issues Need Help
View All on GitHub Roadmap: adaptive output-token budgets (close the reservation vs. actual KV usage gap) about 1 month ago
enhancement help wanted
A token-aware API gateway and load balancer for vLLM and open-source AI infrastructure.
TypeScript
#ai-gateway#fastify#llmops#load-balancer#typescript#vllm
Roadmap: prefix/session-affinity routing (KV-cache reuse) about 1 month ago
enhancement help wanted
A token-aware API gateway and load balancer for vLLM and open-source AI infrastructure.
TypeScript
#ai-gateway#fastify#llmops#load-balancer#typescript#vllm
documentation help wanted good first issue
A token-aware API gateway and load balancer for vLLM and open-source AI infrastructure.
TypeScript
#ai-gateway#fastify#llmops#load-balancer#typescript#vllm
Feature: Add Prometheus /metrics export for the gateway about 1 month ago
enhancement help wanted good first issue
A token-aware API gateway and load balancer for vLLM and open-source AI infrastructure.
TypeScript
#ai-gateway#fastify#llmops#load-balancer#typescript#vllm