llm-d/llm-d-inference-sim

Read environment variables into configuration about 22 hours ago

good first issue

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Align repository with llm-d repo template (12 items) 14 days ago

help wanted

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Generate correct response IDs prefixes 3 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Print real merged configuration to log on simulator initialization 7 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Add retries to connect to ZMQ 7 months ago

good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Support --max-model-len config parameter 8 months ago

AI Summary: Implement a new command-line parameter, `--max-model-len`, in the vLLM simulator. This parameter will define the maximum context window size (in tokens) for the model. Requests exceeding this limit should return a 400 Bad Request error with a specific error message indicating the context length exceeded.

Complexity: 4/5

enhancement good first issue

llm-d/llm-d-inference-sim

102

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Go

#incubating

Open Issues Need Help