radlab-dev-group

radlab-dev-group/llm-router

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

12 stars 4 forks 12 watchers Python Apache License 2.0
automation cloud genai llm llm-balancing llm-gateway llm-gateway-system llm-router llm-router-models llm-router-plugins load-balancing local-llm local-llm-integration model-management on-prem pii prometheus rest-api security
View on GitHub Website
10 Open Issues Need Help Last updated: Sep 6, 2026

Open Issues Need Help

View All on GitHub
Expand REST API Providers: Add Support for Additional Backends 17 days ago
documentation enhancement help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Add Semantic Response Cache Plugin using Redis 17 days ago
documentation enhancement help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Add compatibility with the OpenAI `/responses` API. 9 months ago
documentation enhancement help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Integration with SIM 9 months ago
documentation enhancement help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Improve streaming for LMStudio 9 months ago
bug help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Integration with Anthropic 10 months ago
documentation help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Add saving and encryption of the audit. 10 months ago
enhancement good first issue
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Build a UI for Managing LLM‑Router Configuration 10 months ago
documentation enhancement help wanted
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Implement Anonymizers and Guardrails Pipelines in `endpoint_i 10 months ago
documentation enhancement good first issue
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security
Verify `_do_masking_if_needed` and masking auditor 10 months ago
bug documentation good first issue
radlab-dev-group/llm-router
12

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

Python
#automation#cloud#genai#llm#llm-balancing#llm-gateway#llm-gateway-system#llm-router#llm-router-models#llm-router-plugins#load-balancing#local-llm#local-llm-integration#model-management#on-prem#pii#prometheus#rest-api#security