// case study
Multi-Model AI Orchestration Platform
The AI infrastructure behind my 17 live sites — multi-model routing, RAG, chat assistants, and budget-enforced automation
- Year
- 2026
- Role
- Sole designer and builder — router, RAG pipeline, chat assistants, budget enforcement, and the automation routines that run on top.
- Status
- Live
github.com/ConnorMeador/rag-pipeline-ts
// about this project
What it does and how I built it
Running 17 production sites solo only works because the repetitive work is automated. This platform routes every AI call — site builds, scheduled audits, live chat assistants, bulk content jobs — through a single router that picks the cheapest model capable of the task: local models on my own GPU for the mechanical majority, cheap cloud for overflow, premium models (Claude, GPT) reserved for judgment. A RAG pipeline with locally-run embeddings and an opt-in LLM rerank stage grounds the chat assistants that run on ten of the live sites, and every call appends routing, cost, and trace records so spend and failovers are auditable after the fact.
My role: Sole designer and builder — router, RAG pipeline, chat assistants, budget enforcement, and the automation routines that run on top.
// decisions
One gateway chokepoint instead of per-script API calls
Early scripts each called providers directly — which meant budget caps, failover, and logging had to be reimplemented (or forgotten) in every script. Routing everything through a single OpenAI-compatible gateway put cost enforcement, model fallback, and spend logging at one seam. New scripts inherit all three for free.
Local-first tier ladder with logged escalation
Most daily AI volume is mechanical — transforms, classification, alt text, embeddings — and runs free on local models on my own GPU. Escalation to paid tiers requires a capability reason, and the router records every failover with its direction and estimated cost delta. The discipline is structural, not aspirational: the cheap path is the default path.
Budgets enforced in code, with hard halts
An unattended AI loop with no ceiling is an incident waiting to happen. Every layer carries its own cap — per-call, per-run, and a rolling monthly envelope — with a warning event at 80% and a hard halt on billable calls at 100%. Autonomous loops additionally carry iteration counters and exponential backoff, so a stuck job degrades into a logged skip instead of a runaway bill.
Evidence ledger on every run
Each automated run appends structured records: which tier handled each step and why, what every call cost, and a trace of every action. That turns "why did last night's job spend what it spent" from archaeology into a query — and it is how routing regressions get caught.
// highlights
- Routes across Anthropic, OpenAI, Google, Groq, and local Ollama models by task class and capability
- Automatic failover chains — every reroute logged with direction (up / lateral / degraded) and cost delta
- Hard budget enforcement: warn at 80%, halt billable calls at 100%, per-run and monthly ceilings
- RAG with locally-run embeddings + opt-in LLM rerank second stage that fails open
- Chat assistants grounded on per-site content, live on 10 production directory sites
- GPU lock serializes local inference against image generation on a shared 16 GB card
- Public TypeScript extract of the RAG pipeline on GitHub (rag-pipeline-ts) — full-platform walkthrough on request
// what I’d do differently
The router's architecture was ahead of its observability — when the telemetry pipeline silently broke, calls quietly failed over to the premium tier for weeks before an audit caught it. The dollar caps held (that tier is flat-fee), which is exactly why nothing screamed: the failure cost routing discipline and premium-model throughput, not money. I now treat monitoring as part of the feature — if the ledger isn't writing, the router isn't done, and a watchdog checks that the watchers themselves are alive.
Stack
- TypeScript
- Node.js
- LiteLLM
- Ollama
- n8n
- GitHub Actions
AI Tools
- Claude API
- OpenAI API
- Gemini API
- Groq
- Ollama (local models)
Built with AI as a force multiplier — scaffolding, research, automation, and code review all flow through these tools in my daily loop.
// gallery
Architecture diagrams
Click any diagram to enlarge.
// let’s build
Want something like this?
I design, build, and ship production systems end-to-end — and I’m looking for a remote full-time role where I can do it with a team.