vllm-project/vllm-metal

M1 Ultra 128GB RAM fails to load https://huggingface.co/google/gemma-4-31B-it 16 days ago

good first issue

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

[RFC] Redundant KV-cache reads during speculative-decode verification 21 days ago

help wanted

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

[Feature]: Support P/D disaggregation via RDMA over thunderbolt 2 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

vLLM Metal Model Support 3 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

vllm-metal performance vs vllm-swift 3 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

Qwen 3.6 27B model not working 4 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

Server stalls mid-stream for large context requests, never sends [DONE] or closes connection 5 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

[Feature] Support for Mamba-style Linear Attention Layers in Models 7 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

Enhancement: Distributed Inference with RAY + vLLM-Metal on Apple Silicon 7 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

Support for MoE Models (Qwen-MoE, etc.) & Mamba Cache on Apple Silicon 7 months ago

good first issue

vllm-project/vllm-metal

1.4K

Community maintained hardware plugin for vLLM on Apple Silicon

Python

Open Issues Need Help