This is an architecture decision with cost consequences that are usually discovered afterwards. The gateway question gets framed as latency and reliability; the part that bites later is attribution.
What a gateway actually changes about cost
Start with what it does not change. A gateway does not make tokens cheaper. Whether you call OpenAI directly or through LiteLLM, OpenRouter or Bedrock, the underlying token price is set by the provider.
What changes is four things.
Billing consolidation. One invoice instead of several. Real administrative value, particularly where procurement is slow.
Switching cost. Changing model becomes a config change rather than a code change, which is what makes routing strategies practical at all.
Failover. Cross-provider fallback when one is degraded. Reliability rather than cost, though outage-driven retries are expensive.
Margin, sometimes. Some hosted gateways add a percentage. Self-hosted proxies do not, but you pay in infrastructure and maintenance.
The attribution trade-off
This is the part worth thinking about before you commit.
Calling providers directly means each provider's console attributes spend the way it does natively. OpenAI separates by project and API key. Anthropic reports by key. That detail is granular, provider-maintained, and free.
Behind a gateway, the provider now sees one caller — the gateway. Provider-side attribution collapses to a single line. All of your per-model, per-team, per-feature visibility now depends on what the gateway records and exposes.
Good gateways handle this well, tagging requests and reporting by model, key and virtual team. Others give you a total and a model breakdown and nothing else.
The failure mode is predictable: teams adopt a gateway for switching convenience, and three months later cannot answer which feature drove a cost increase, because the granularity that used to be free now requires instrumentation nobody built.
Ask the attribution question during evaluation, not after. Can it report cost by model, by consuming service, and by team? Can it export that? If not, budget for building it.
When direct is the right answer
You use one or two providers heavily. The consolidation benefit is small and you keep provider-native attribution for free.
You need provider-specific features. Prompt caching, batch endpoints, structured output modes and fine-tuned models are often exposed inconsistently or late through gateways.
Latency is critical. A gateway is a hop. Usually small, occasionally not, and always another thing that can fail.
You want the provider's own usage data. Especially where you are negotiating committed spend, since the provider's own reporting is what the conversation runs on.
When a gateway earns its place
You switch models frequently. If you are genuinely routing between tiers based on task, doing it in config rather than deploys is transformative.
You run many providers. Past three or four, consolidation stops being convenience and becomes the only way anyone holds a total.
You need cross-provider failover. No amount of direct integration gives you this without building it.
Procurement is the constraint. One vendor relationship instead of six is sometimes the whole business case, particularly in regulated environments. This is much of Bedrock's and Vertex's appeal.
The hybrid most teams end up with
In practice many teams route high-volume, well-understood workloads directly — keeping provider-native attribution and features where the money is — and route experimental or multi-model work through a gateway where flexibility matters more than granularity.
That is a reasonable place to land, provided it is a decision rather than an accident. The version that hurts is arriving there by drift, with half the spend in one place and half in another and no single total anywhere.
Whichever you choose, hold the total somewhere
The consistent failure across both architectures is that nobody holds the combined number. Direct means several consoles; a gateway means one console plus whatever did not go through it.
Either way you need one place where all model spend is normalised into the same units with a baseline per provider — otherwise there is nothing for an anomaly to deviate from. StackSpend connects both direct providers and the cloud accounts gateways bill through, so the total survives the architecture decision rather than depending on it.
FAQ
Does an AI gateway reduce token costs?
No. Token pricing is set by the provider and a gateway passes it through; some hosted gateways add a margin on top. Gateways save on switching effort and operational overhead, not unit price.
Do I lose cost attribution behind a gateway?
Provider-side attribution collapses to a single caller, so you depend entirely on the gateway's own reporting. Check during evaluation whether it reports by model, consuming service and team — this is the most commonly missed consequence.
Is Bedrock or Vertex cheaper than calling providers directly?
Generally at or slightly above direct API pricing. The case for managed platforms is procurement, data residency and consolidated billing rather than unit cost.
Can I use a gateway for some workloads and direct calls for others?
Yes, and many teams do — direct for high-volume stable workloads, gateway for experimental or multi-model work. Make it a deliberate split, and keep one consolidated view across both.
