llm-d/llm-d-router

Remove deprecated dp_profile_handler 1 day ago

help wanted lifecycle/rotten

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Logs are not correlated with traces: trace_id/span_id never injected into the logger 5 days ago

help wanted needs-triage kind/bug observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Refactor label-based filters 6 days ago

help wanted lifecycle/rotten

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[E2E Tests] Use standalone mode for e2e tests 7 days ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Add span for filter 16 days ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Deprecate load-aware scorer 17 days ago

help wanted lifecycle/stale

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[E2E Tests] Failing on a few PRs 23 days ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Compatibility] Validate and document TurboQuant (KV cache quantization) serving support 25 days ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for mm-embeddings-cache-producer plugin 30 days ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for program-aware-fairness plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for precise-prefix-cache-producer plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Add span for picker about 1 month ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Migrate e2e tests from GAIE about 1 month ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Improve datalayer observability about 1 month ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for approx-prefix-cache-producer plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Add pluginName to the default DumpState about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for predicted-latency-producer plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for file-discovery plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Implement DumpState for no-hit-lru-scorer plugin about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Operations] Tracking: implement DumpState for all in-tree plugins about 1 month ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] An EPP metric that reports inflight tokens and requests about 2 months ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observaibility] epp/metrics: unbounded label cardinality from user-controlled inputs (metrics cardinality DoS) about 2 months ago

help wanted needs-triage kind/bug observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] P/D sidecar observability: add Prometheus /metrics, wire trace propagation, document enablement about 2 months ago

help wanted needs-triage observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

A session-affinity-filter plugin about 2 months ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Observability] Add fairness_id and objective labels to EPP TTFT/TPOT metrics for per-tenant SLO observability about 2 months ago

help wanted observability

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

token-producer: explicit backend selector field 2 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

more granular request-based load-aware scorers 2 months ago

good first issue help wanted needs-triage

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Review and types of fix CRD fields 2 months ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[migration] GAIE changes and migration 2 months ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

EPP should signal invalid plugin config fields via error/warning 2 months ago

good first issue help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Add helm charts builds to ci 3 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration] rename all headers managed by epp to have the prefix x-llm-d 3 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration] Remove the InferencePool CRD definitions. 3 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration]: Migrate the InferenceObjective and InferenceModelRewrite APIs to llm-d.ai group name 3 months ago

help wanted triage/accepted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

[Migration]: rename the docker images 3 months ago

help wanted

llm-d/llm-d-router

236

llm-d Router: The intelligent entry point for inference requests

Go

#ai#gateway-api#inference#kubernetes#networking

Open Issues Need Help