noonghunna

noonghunna/club-3090

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.

2.3K stars 145 forks 2.3K watchers Python Apache License 2.0

1 Open Issue Need Help Last updated: Sep 20, 2026

Open Issues Need Help

View All on GitHub

fa2-fp8kv: EngineCore dies on TP≥4 — "Sequence bounds exceed this port's supported envelope" (silent, taints quality runs) 2 days ago

bug help wanted

noonghunna/club-3090

2.3K

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, llama.cpp, ik_llama) and model-agnostic. Currently shipping Qwen3.6-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1× and 2× cards.

Python