Open Issues Need Help
View All on GitHubCost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Correct a typo in the environment variable name `AIBRIX_Model_GPU_PROFILE_CACHING_FLAG` to `AIBRIX_MODEL_GPU_PROFILE_CACHING_FLAG` within the AIBrix project, likely involving a code change and potentially updating documentation or testing.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Enhance the existing AIBrix Helm chart to meet production-grade requirements. This includes implementing features like high availability, security (RBAC, secret management, network isolation), observability (health checks, metrics, logging), operational excellence (input validation, flexible scheduling, upgrade safety), and potentially enterprise features (multi-tenancy, compliance). The goal is to make the Helm chart the default installation method, replacing Kustomization for production deployments.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Refactor the AIBrix project's quick start documentation page to remove development-related instructions and make it cleaner and more concise, focusing only on essential steps for a quick start. The current page mixes stable and nightly installation instructions, which should be separated.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Extend the AIBrix StormService to support community-standard PodGroup implementations from Volcano and Kueue, in addition to the existing Godel Scheduler integration. This involves creating a generic PodGroup interface, implementing adapters for each scheduler, and allowing users to select their preferred backend via configuration, while maintaining backward compatibility with Godel.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Implement upgrade orchestration features for the Storm service within the AIBrix project, enabling pause, resume, rollback, canary releases, and progress monitoring during upgrades. This mirrors the functionality of Argo Rollouts to improve deployment safety and allow for more controlled releases.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Improve the unit test coverage of the roleset controller within the AIBrix project. This involves writing additional unit tests to increase the overall test coverage, ensuring the reliability and stability of this specific component.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Improve the AIBrix documentation by adding a section detailing the architecture of the storm service controller, explaining its three layers (storm service, roleset, pods), its rolling strategies (Parallel, Sequential, Interleave), update strategies (RollingUpdate vs. InPlace), and the differences between Stateful and Stateless deployments. The goal is to provide users with a clear understanding of the project's P/D orchestration solution.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: The task is to modify the AIBrix controllers to automatically inject the `ROLE_TEMPLATE_HASH` value from pod labels into the environment variables of containers within the pods, eliminating the need for manual configuration. This is a necessary change to support the disaggregation feature and improve user experience.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Improve the unit test coverage for the `stormservice` controller within the AIBrix project. This involves adding more unit tests to enhance internal quality control.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Implement an optional context caching mechanism in AIBrix to improve efficiency in multi-turn conversations. This involves adding API fields for context ID management, integrating with existing KV cache and memory management, ensuring multi-tenant isolation, and implementing metrics tracking.
Cost-efficient and pluggable Infrastructure components for GenAI inference
AI Summary: Refactor the AIBrix project's documentation to separate architectural diagrams and explanations from usage instructions. This involves reorganizing existing content and potentially creating new sections to improve clarity and user understanding of the system's design.
Cost-efficient and pluggable Infrastructure components for GenAI inference