What the Gateway stores
The Gateway stores no prompt or response text. It records who sent each request, which model answered, tokens, cost, timing and a request-body fingerprint.
Prompts and responses are not stored#
The hosted Gateway passes a request's prompt and the model's response through memory only. Neither is written to its database, its logs or its traces. An error the provider wrote about your request is passed back to you and not logged.
What is recorded#
For each request attempt, the Gateway records:
| Recorded | Why |
|---|---|
| Request and attempt ids, times, state | To show the request on the usage page and settle it. |
| Key and its departments | To charge the right budgets. |
| End user, when the request names one | To report and limit spend per person. Only the identifier, nothing else. |
| Tags, when the request sends them | To split and filter usage. Only the keys and values you send. |
| Model, and why it was chosen | To show what answered. With auto, the judgement's label, such as analysis/multi_step. |
| Token counts and price tier | To price it. |
| Why it was refused, if it was | The code and the limit's numbers, never the provider's wording. |
| A SHA-256 fingerprint of the body | To tell a resent request (same Idempotency-Key) from a different one. It cannot be turned back into the text. |
The auto judgement reads a bounded excerpt of the request (at most 8 KiB) with a model call of its own; the excerpt is not stored either. Its decision is kept in the serving process's memory for 30 minutes under a hash of the excerpt, so the rest of the same turn uses the same model.
The model provider#
A request still goes to a model provider (Anthropic or OpenAI, or a model in your own network), and that provider's own data terms apply to it. On /v1/responses the Gateway sends store: false, so the provider does not keep the response for later retrieval.
Self-hosted Gateway#
A Gateway you run yourself stores the same records in your own PostgreSQL database. Its trace export is off unless you point it at your own collector, and it then exports the same bounded attributes. The image has a model-exchange recorder for test replays that stores prompts and replies verbatim; it is off unless its operator sets FRANKY_MODEL_TAPE_MODE, and it is never turned on in the hosted Gateway.