Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control
Running many AI models without a gateway creates duplicated auth, weak rate limits, and hidden costs. We cover model routing with A/B traffic splitting, token-based rate limiting per team, fallback chains, and per-team cost attribution. "Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control" turns that scenario into a session where Rohit Mishra walks through gateway patterns for routing and control.
Session details
Running multiple AI models on Kubernetes without a unified gateway means duplicated auth, no rate limits, and opaque costs. This talk builds an inference gateway from scratch using Envoy, Kubernetes CRDs, and Prometheus. We cover model routing with A/B traffic splitting, token-based rate limiting per team, fallback chains, and per-team cost attribution. Includes a live demo deploying the gateway and routing traffic across models in real time.
Primary action
Share the public session page with the tracked partner link, then use the copy variants below for LinkedIn, short social posts, and newsletter or email mentions.
Agenda facts
- Format
- Talk / Presentation
- Kind
- Conference Session
- Topic
- Artificial Intelligence and Machine Learning
- Level
- Intermediate
- Time
- Oct 29, 2026, 10:30 AM
- Room
- Room A
- Duration
- 60 min
Speakers
Link partners to speaker kits when available, or to the public speaker profile on mitechcon.net.
Rohit Mishra
Copy to paste
LinkedIn, short social, and newsletter or email copy variants for this session.
LinkedIn
Session LinkedIn Post
Running many AI models without a gateway creates duplicated auth, weak rate limits, and hidden costs. We cover model routing with A/B traffic splitting, token-based rate limiting per team, fallback chains, and per-team cost attribution. "Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control" turns that scenario into a session where Rohit Mishra walks through gateway patterns for routing and control. 📅 October 28–30, 2026 📍 Oakland University, Rochester, MI 🔗 https://www.mitechcon.net/sessions/building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control #MITechCon #Kubernetes #AIInfrastructure #LLM
Short social
Short Session Post
Running many AI models without a gateway creates duplicated auth, weak rate limits, and hidden costs. Learn gateway patterns for routing and control. https://www.mitechcon.net/sessions/building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control #MITechCon #Kubernetes
Newsletter / email
Newsletter Or Email Blurb
Feature this MITechCon 2026 session in your newsletter or email: "Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control". Running many AI models without a gateway creates duplicated auth, weak rate limits, and hidden costs. We cover model routing with A/B traffic splitting, token-based rate limiting per team, fallback chains, and per-team cost attribution. "Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control" turns that scenario into a session where Rohit Mishra walks through gateway patterns for routing and control. Format: Conference Session. Topic: Artificial Intelligence and Machine Learning. Level: Intermediate. Scheduled for Oct 29, 2026, 10:30 AM in Room A. Learn more and share the session: https://www.mitechcon.net/sessions/building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control/?utm_source=partner-kit&utm_medium=partner&utm_campaign=mitechcon-2026&utm_content=session-building-an-ai-inference-gateway-on-kubernetes-model-routing-rate-limiting-and-cost-control
Share image
Generated session card with alt text, dimensions, file size, and download link.
Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control share image
Generated session social card for partner promotion.
- Alt text
- Social share image for Building an AI Inference Gateway on Kubernetes: Model Routing, Rate Limiting & Cost Control at MITechCon 2026
- Dimensions
- 1024 x 1024px
- Size
- 533 KB
- Updated
- 2026-07-08