vllm-project/vllm-ascend

[Contribution] 任务 #26：【社区模型】适配internlm2_20b_chat 4 days ago

help wanted

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #23：【社区模型】适配baichuan-7b 4 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #1：vllm主社区多模态能力加强，视频场景evs能力压缩 8 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #9：Memcache 池化后端高可用模式验证 9 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #6：MooncakeConnector handshake 端口动态分配 9 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #17：310P线上CI稳定性构建 9 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #33：Qwen2.5-1.5B-apeach 离线推理功能测试 10 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Misc]: Discussion on accuracy variance 10 days ago

help wanted stale wait-feedback

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #14：投机解码 11 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #12：AscendStore 池化模块 UT 测试补充 11 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #8：Mooncake 池化场景高可用模式验证 11 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #16：Gemma4 E2B/E4B轻量级图片、语音、文本多模态大模型适配 12 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #15：IQuest-Coder-V1-40B 代码大模型适配 13 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】yxr2_qwen3-32B性能测试 16 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #13：长序列场景下pcp-dcp功能支持 17 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】vLLM-Ascend 支持 Memcache 池化后端验证 18 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #2：FlashComm1 环境变量迁移到 AscendConfig 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #3：MoE通信方式选择代码重构 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #4：Mooncake KV Connector 调度侧/Worker 侧解耦重构 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #5：PD 分离非对等 TP 配置简化 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #7：PD 分离特性叠加场景看护 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #10：A3 25.5 HDK + CANN 9.0 fabric_mem 池化能力验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #11：Mooncake Store standalone 池化模式验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #18：triton算子使用策略重构 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #19：310P支持compressed_tensors_config.py 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #20：310P DeepSeek-OCR2 多模态OCR模型适配优化 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #21：310P DeepSeek-R1蒸馏系列模型适配 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #22：【社区模型】适配qwq-32b 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #24：【社区模型】适配DS-distill-LLaMA-70B 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #25：【社区模型】适配DS-distill-Qwen-32B 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #27：【社区模型】适配Llama-3.2-3B-Instruct 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #28：【社区模型】适配MiniCPM-2B 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #29：【社区模型】适配MiniCPM3-4B 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #30：【社区模型】适配Phi-4-mini 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #31：bge-reranker-v2-m3 离线推理功能测试 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Contribution] 任务 #32：e5-mistral-7b-instruct 离线推理功能测试 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-Coder-30B-A3B 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-235B-A22B-W8A8 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-Omni-30B-A3B-Thinking 部署与性能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-VL-8B-Instruct 部署与性能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-VL-30B-A3B-Instruct 功能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] InternVL3.5-38B-W8A8 部署与性能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3.5-27B-W8A8 端到端功能与精度验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] MiniMax-M2.5-W8A8 端到端功能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-32B-W4A4 Memcache 池化后端验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-32B-W4A4 Mooncake 池化功能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-30B-A3B 性能测试 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】cjw1：Qwen3-Coder-30B-A3B部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】cjw2：Qwen3-235B-A22B-w8a8部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】hjh1-Qwen3-Omni-30B-A3B-Thinking 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】hjh2-Qwen3-VL-8B-Instruct 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】hjh3-Qwen3-VL-30B-A3B-Instruct 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】hjh4-InternVL3.5-38B 部署验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】Qwen3.5-27B-w8a8模型端到端验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】MiniMax-M2.5 W8A8模型端到端验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】Qwen3-32B开启池化功能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】yxr1_qwen3-30B性能测试 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】Qwen3.5-27B 基础部署与服务调用验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

【众测任务】Qwen3.5-27B 并行策略/卡数调优与基线验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3-32B-W4A4 性能测试 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3.5-27B 4卡部署功能验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Community Test] Qwen3.5-27B 8卡部署、性能与精度基线验证 25 days ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Feature]: Some worker interfaces are missing 2 months ago

help wanted feature request

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Help Wanted]: Recover full custom ops test 2 months ago

bug good first issue help wanted wait-feedback

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Bug]: 不支持Ultravox,Key Error:audio 3 months ago

bug help wanted feature request new model

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[RFC]: support sequence parallelism by pass 4 months ago

help wanted RFC

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Feature]: cannot get vllm outputs.metrics 5 months ago

good first issue help wanted feature request

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Misc]: Cleanup for `csrc/torch_binding.cpp` 6 months ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[RFC] Unit test enhancement 7 months ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: https://github.com/vllm-project/vllm-ascend/blob/main/docs/source/tutorials/Qwen2.5-7B.md 7 months ago

documentation help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

cleanup useless patch 8 months ago

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: developer_guide/evaluation/using_lm_eval.html 9 months ago

documentation good first issue

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Test] enable all the skipped test 10 months ago

good first issue help wanted ci/build

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Feature]: Add `get_vit_attn_backend` interface 10 months ago

help wanted feature request

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Feature]: Add import_kernels interface 10 months ago

help wanted feature request

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[RFC]: Unit test coverage improvement 10 months ago

good first issue help wanted feature request

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: a3 image guide is missing 10 months ago

documentation good first issue

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: --task is deprecated 10 months ago

documentation good first issue

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: Make docker image more friendly for devs 11 months ago

AI Summary: This GitHub issue proposes two specific changes to the `vllm-ascend` project's GitHub Actions workflow to improve the developer experience related to its Docker image. The suggested fixes involve removing an `http.https://github.com/.extraheader` configuration and modifying a `git pull` command to use `fetch-depth: 0` instead of `--shallow` in the `image_ubuntu.yml` workflow file.

Complexity: 2/5

help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend

C++

#ascend#inference#llm#llm-serving#llmops#mlops#model-serving#transformer#vllm

[Doc]: 配套软件里，需要添加nnal的安装，同时页面里跳转的昇腾环境安装也需要加上nnal的安装指导 11 months ago

documentation good first issue help wanted

vllm-project/vllm-ascend

2.5K

Community maintained hardware plugin for vLLM on Ascend