Bootstraps a Mule application that intelligently routes prompts to the optimal LLM provider based on semantic classification, with built-in cost-optimization, prompt safety guardrails, and full observability. Ideal for developers who want a production-ready LLM routing gateway and a simple UI to test routing without implementing model selection logic from scratch.
Intelligently routes prompts to the optimal LLM provider based on content semantics — with built-in cost optimization, prompt safety guardrails, and full observability. No model selection logic required from the developer.
Clone or download the project, then import the Mule application into Anypoint Studio or extract and open in a code editor. Review the README for a quick file map and prerequisites such as Mule runtime version and Docker.
Populate src/main/resources/*.properties or the provided properties file with API keys and model preferences for OpenAI, Anthropic, and any local LLM endpoints. Adjust cost thresholds and safety rules in the decision-config YAML or properties file.
Start the project in Anypoint Studio with the configured Mule runtime, or use docker-compose up to launch the Mule app and the UI. Ensure the metrics exporter endpoint is reachable if you run Prometheus or a local observability stack.
Use the included React UI or curl against the HTTP endpoints to submit prompts. Watch logs, metrics, and the audit trail to confirm semantic routing, cost-optimization decisions, and safety rejections. Tweak rules and rerun to iterate.
Modify the DataWeave classifier, cost optimizer rules, or safety flows to reflect your business goals (e.g., latency targets, cost caps, regulatory constraints). Add new provider connectors or model mappings by following the existing connector template.
Choose this template when you need a ready-made LLM gateway with semantic routing, cost controls, and observability — it saves building classification, safety, connector plumbing, and dashboards from scratch.