waybarrios

waybarrios/vllm-mlx

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

1.6K stars 220 forks 1.6K watchers Python Apache License 2.0
anthropic anthropic-api apple-silicon claude-code continuous-batching inference-server llm local-llm macos mcp mlx multimodal-ai openai openai-api openai-compatible speech-to-text text-to-speech tool-calling vision-language-model vllm
View on GitHub Website
1 Open Issue Need Help Last updated: Sep 2, 2026

Open Issues Need Help

View All on GitHub
Prefix cache: generalize prompt canonicalization beyond Anthropic billing header 19 days ago
enhancement help wanted optimization
waybarrios/vllm-mlx
1.6K

High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.

Python
#anthropic#anthropic-api#apple-silicon#claude-code#continuous-batching#inference-server#llm#local-llm#macos#mcp#mlx#multimodal-ai#openai#openai-api#openai-compatible#speech-to-text#text-to-speech#tool-calling#vision-language-model#vllm