LLM Gateway (optional)¶

An LLM Gateway is a reverse proxy that sits between the Claude CI agent and the Anthropic API. Claude Code routes through it by pointing ANTHROPIC_BASE_URL at the gateway instead of api.anthropic.com. The gateway then gives you two things this project otherwise leaves to the API: prompt-response caching and guardrails.

Claude CI Agent ──(ANTHROPIC_BASE_URL)──▶ LLM Gateway ──▶ Anthropic API
                                          │  cache + guardrails
                                          └─ metrics / budget / audit

It is optional and opt-in: with ANTHROPIC_BASE_URL unset the agent talks to api.anthropic.com directly, exactly as before. See the Architecture diagram for where it fits.

Why a gateway¶

The agent already streams every action through the OTel Collector for secret-scrubbed audit— but that is post-hoc telemetry. A gateway acts in-line on the model traffic itself, so it can cache responses and block bad requests before they reach the API.

Prompt caching¶

Two distinct layers— a gateway uses both:

The gateway passes cache_control through untouched, so native caching keeps working. On top of that, its response cache deduplicates repeated calls— the Advisor re-reviewing an unchanged file, or identical lint-summary prompts across pipelines— and returns the stored completion without spending tokens. This is the big lever in CI, where the same prompts recur run after run.

Guardrails¶

A gateway enforces policy on the wire, in both directions:

• Input screening— detect prompt-injection / jailbreak attempts and redact secrets or PII before the request leaves your trust boundary. This is defense-in-depth with the OTel scrubbing, which only redacts the audit copy.
• Output screening— block disallowed content, enforce response schemas, or strip anything that should never reach a PR comment.
• Budget & rate limits— cap spend per pipeline / project / token, and throttle runaway agents. Pairs with the claude_code.cost.usage metric the agent already emits (see Observability).
• Model allow-listing— restrict the agent to approved models (e.g. only claude-sonnet-4-6 / claude-opus-4-8), rejecting anything else.

Different models— or vendors— for creation vs review¶

The two personalities do opposite jobs, so it makes sense to run them on different models, or even different vendors:

• Agent (creation)— the heavy lift. Give it the strongest coding model (e.g. claude-opus-4-8); it writes the implementation.
• Advisor (review)— an independent check. Route it to a different model or vendor, so the reviewer doesn't share the creator's blind spots. When the same model both writes and grades its own work, a misread requirement or a hallucinated API tends to pass review unnoticed— it made the same mistake twice. A model trained differently de-correlates those failures and catches more.

Why this is safe to do— spec-driven engineering¶

This only works because the work is anchored to a written spec, not to one model's interpretation. The spec— acceptance criteria, the CLAUDE.MD contract, an issue's definition of done— is the shared source of truth:

• The Agent implements to the spec; the Advisor reviews the diff against the same spec. The question is objective— does this meet the spec?— not "would I have written it this way," so a different reviewer model stays grounded instead of bikeshedding style.
• The spec is the contract that lets you swap the model behind either role without changing the pipeline. Creation and review are interchangeable engines bolted to a fixed specification, which is the whole point of spec-driven work.
• Independent review is only meaningful when it's actually independent. Model/vendor diversity is what turns "the AI reviewed itself" into a real second opinion.

Wiring it up¶

The pipelines already set CLAUDE_PERSONALITY (agent / advisor) on each job, so you have two levers:

• Different Claude models— no gateway needed. Set a different CLAUDE_MODEL per personality job (Agent → claude-opus-4-8, Advisor → claude-sonnet-4-6, or vice-versa). Each personality already passes --model "$CLAUDE_MODEL".
• Different vendor— via the gateway. Point both at the gateway and let it route on the CLAUDE_PERSONALITY (or a header you add), sending the Advisor's traffic to a non-Anthropic backend that speaks the /v1/messages shape. LiteLLM and Portkey both support per-route model/vendor mapping. Keep the model allow-list above scoped per route so each personality can only reach its intended backend.

Choosing a gateway¶

Any Anthropic-compatible proxy works— it must expose the /v1/messages API shape. Common options:

Host the gateway inside your trust boundary— it sees full prompts and completions. A central in-cluster service (one gateway, many pipelines) is the usual shape; a per-job sidecar also works.

Enabling it¶

Point the agent at the gateway and (if the gateway authenticates separately) hand it the gateway key. Claude Code reads both from the environment:

env:
  ANTHROPIC_BASE_URL: ${{ vars.ANTHROPIC_BASE_URL }}