kubernetes-sigs/inference-perf

Improve request-level data collection when only the metrics are needed. 6 days ago

good first issue help wanted priority/important-soon

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Feature Request] Add Anthropic Messages API support about 1 month ago

help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

SGLang support: Incorrect metric mapping about 1 month ago

kind/bug help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

RNG state synchronization issue across worker processes 2 months ago

kind/bug help wanted needs-triage

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Support Custom Multimodal Datasets 2 months ago

help wanted priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add a random interval for session rate 2 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add support for VisionArena Dataset 3 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Surface vLLM prompt-cache stats (cached/uncached tokens) in lifecycle metrics 3 months ago

good first issue help wanted priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Bug] Prometheus metrics not being exported in report 3 months ago

kind/bug help wanted

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

[Feature Request] Add command line flags to run inference-perf without config file 4 months ago

good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

ShareGPT generation hits StopIteration 5 months ago

kind/bug good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add configurable base seed 6 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Comparitive analysis of performance metrics 6 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Consider using typing.NamedTuple instead of TypeAlias 6 months ago

good first issue priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Add tests for worker concurrency 6 months ago

good first issue priority/important-soon

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

support to set an SLO threshold level and then calculate the goodput, rather than measuring the total throughput. 8 months ago

good first issue priority/backlog

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Handle sigint gracefully 10 months ago

kind/feature good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Prometheus stage metric query duration incorrect 12 months ago

kind/bug good first issue

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

need a detail description and guidance for how to setup config.yml about 1 year ago

AI Summary: Create a README file explaining how to configure the `config.yml` file for the Inference Perf benchmarking tool, covering different environment setups (model names, base URLs, API tokens, etc.), and providing clear guidance and examples for successful execution.

Complexity: 4/5

good first issue kind/support

kubernetes-sigs/inference-perf

216

GenAI inference performance benchmarking tool

Python

#k8s-sig-scalability

Open Issues Need Help