Gateway limits and budgets
Each request's maximum cost is held against workspace and department limits before sending. Rate limits cap requests and tokens per minute. Refusals return 429.
Held before sending#
Before a request goes to a model, the Gateway works out the most it can cost: the prompt it estimates plus the output the request asks for (max_tokens or max_completion_tokens), at the price the request asks for. That amount is held against every limit that applies, together with what is already spent this month and what other requests in flight are holding. If it doesn't fit, the request is refused before it is sent and costs nothing. Requests that fit run in parallel.
When the answer arrives, the held amount is replaced by the measured usage. If the provider never reports usage (the connection broke, for example), the request is settled 15 minutes after it was sent at an estimate: the usage a Messages stream had reported before it stopped, otherwise the held amount. An owner or admin can correct an estimate once, on the request's row under AI Gateway › Usage.
Monthly limits#
| Limit | Counts | Where you see it |
|---|---|---|
| Workspace monthly spend | USD, at the catalog price of each model | AI Gateway › Usage › Monthly limits |
| Monthly tokens and requests | All Gateway requests of the workspace | AI Gateway › Usage › Monthly limits |
| Department ceilings | USD spent by keys charged to that department | Settings › Usage |
| Rolling window | Tokens in the last 6 hours, and each key's share of them | The refusal names it |
| End user, per key | USD spent by one end user of a key this month (optional) | The key's settings |
Monthly limits reset on the first day of the next month, UTC. A key charged to departments counts against each of them and against the workspace; the tightest limit decides.
Rate limits#
Each key and each workspace also has a limit on requests per minute and tokens per minute. The defaults come from the workspace's plan and are set above the model providers' own per-minute limits, so a tool that works when connected directly does not hit ours first. Owners and admins can lower them for a key, for example for a batch job, or for the whole workspace.
The counts are shared by every Gateway server, so a key's limit applies to all its requests together. A request over a rate limit is refused before it is sent and holds nothing.
A key can also have a cap on concurrent requests. There is none unless you set one.
The three kinds of 429#
| Code | What it means | Retry? | What to do |
|---|---|---|---|
provider_rate_limited | The model provider is busy. | Yes. The provider's Retry-After is passed on. | Nothing; the official SDKs wait and retry. |
gateway_rate_limited | This key or workspace sends too fast. | Yes. Retry-After and X-Should-Retry: true are sent. | Wait, or raise the key's or the workspace's rate limit. |
inference_allowance_exceeded | A monthly limit or ceiling has no room left. | No. X-Should-Retry: false is sent. | Ask an owner or admin to raise the limit, or wait for the reset. |
The message names the limit, how much of it is used and, for a monthly limit, the day it resets, for example: This workspace reached its monthly spend limit (US$60.00 of US$60.00). Resets 2026-11-01. Ask a workspace admin to raise it.
Alerts#
When a monthly limit or a department ceiling reaches 80% and 100%, owners and admins see a banner on AI Gateway, and the workspace's notification destinations (email and signed webhook, set in Settings › Notifications) are told once per limit and threshold each month.