> ## Documentation Index
> Fetch the complete documentation index at: https://gomodel-feat-mcp-server.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Rate Limits

> Cap requests per minute, tokens per minute, and in-flight concurrency per user path, provider, or model.

## Overview

Rate limits cap request rate, token rate, and in-flight concurrency. Every
rule has a **scope** that decides what it protects:

* **User path** (consumer control) — e.g. `/team/alpha` can send `100`
  requests per minute, consume `50,000` tokens per minute, and hold `10`
  requests in flight, across everything under that path.
* **Provider** (upstream protection) — e.g. send provider `openai` at most
  `500` requests per minute across all consumers and models, so the gateway
  never blows through your provider quota.
* **Model** — e.g. `openai/gpt-4o` may consume `90,000` tokens per minute; a
  bare id like `gpt-4o` caps the model across every provider.

The scopes behave differently on breach. A **user path** over its limit gets
`429` with an OpenAI-shaped error, an accurate `Retry-After` header, and
`x-ratelimit-*` headers — switching models would not help, the consumer is the
subject. A saturated **provider or model** is instead treated like an
unavailable target: virtual-model load balancing and failover route around it
while another target has capacity, and only when no viable target remains does
the client get `429`.

Limits apply to every model endpoint: chat completions, responses, messages,
embeddings, audio, passthrough, realtime sessions, and batch submission.

Use rate limits for traffic control and budgets for spend control: budgets cap
dollars over hours-to-months from durable usage records, rate limits cap
request and token velocity over seconds-to-a-day with fast in-memory counters.

## Enable rate limits

Rate limits are enabled by default and cost nothing until rules exist:

```env theme={null}
RATE_LIMITS_ENABLED=true
```

Token limits (`max_tokens`) are counted from usage records, so they require
usage tracking:

```env theme={null}
USAGE_ENABLED=true
```

Request and concurrency limits work without usage tracking. If token rules are
configured while usage tracking is off, GoModel starts and logs a warning; the
token limits are not enforced.

## Create rate limits

In the dashboard:

```text theme={null}
Rate Limits -> Create Rate Limit
```

Pick the scope (user path, provider, or model), the subject, a period, and the
caps. Dashboard-created rules are marked `manual`. Rules loaded from YAML or
environment variables are marked `config` and are read-only in the dashboard.

The Models page has a gauge button on every model row and provider header that
opens the same limits for that subject in context: the model's own rules, the
provider rules it shares, and the global (root user-path) rules — each with
live usage — plus add and edit actions.

Editing a rule's scope, subject, or period **moves** it: the rule is recreated
under the new key and its live counters restart. Editing only the caps keeps
the counters.

Seed rules from YAML:

```yaml theme={null}
rate_limits:
  enabled: true
  user_paths:
    - path: "/team/alpha"
      limits:
        - period: "minute"
          max_requests: 100
          max_tokens: 50000
        - period: "day"
          max_requests: 10000
        - period: "concurrent"
          max_requests: 10
  providers:
    - name: "openai"
      limits:
        - period: "minute"
          max_requests: 500
        - period: "concurrent"
          max_requests: 50
  models:
    - model: "openai/gpt-4o"
      limits:
        - period: "minute"
          max_tokens: 90000
```

Or use environment variables. User-path rules use `SET_RATE_LIMIT_<PATH>`
(the suffix maps to a user path exactly like `SET_BUDGET_*`; `__` separates
path segments); provider rules use `SET_PROVIDER_RATE_LIMIT_<NAME>`
(underscores become hyphens, like provider-instance env vars):

```env theme={null}
SET_RATE_LIMIT_TEAM__ALPHA="rpm=100,tpm=50000,rpd=10000,concurrent=10"
SET_RATE_LIMIT_="rpm=1000"
SET_PROVIDER_RATE_LIMIT_OPENAI="rpm=500,concurrent=50"
```

Compact names map to periods: `rpm`/`tpm` per minute, `rph`/`tph` per hour,
`rpd`/`tpd` per day, and `concurrent` for the in-flight cap. A JSON array of
rule objects is also accepted for custom windows:

```env theme={null}
SET_RATE_LIMIT_TEAM__ALPHA='[{"period_seconds":300,"max_requests":50}]'
```

Model rules have no env form (model ids are not env-name safe); declare them
in YAML or through the admin API and dashboard.

Supported named periods:

| Period       | Seconds | Meaning                          |
| ------------ | ------- | -------------------------------- |
| `minute`     | `60`    | requests/tokens per minute       |
| `hour`       | `3600`  | requests/tokens per hour         |
| `day`        | `86400` | requests/tokens per day          |
| `concurrent` | `0`     | max in-flight requests right now |

Custom windows use `period_seconds` directly.

## How matching and counting work

Rule paths apply to the configured path and its descendants, exactly like
budgets: a rule on `/team` matches `/team/app` but not `/team-alpha`. A rule
on `/` applies to all traffic.

Provider rules match the resolved provider instance by name. Model rules
match the executed model: a provider-qualified subject (`openai/gpt-4o`)
covers one provider's model, a bare subject (`gpt-4o`) covers that model on
any provider. Matching is case-insensitive.

**A rule owns one shared counter for its whole subject.** A `100 rpm` rule on
`/team` allows 100 requests per minute across everything under `/team`
combined; a `500 rpm` rule on provider `openai` is shared by every consumer
and model routed there. For per-key limits, bind each managed API key to its
own user path and define rules there.

All matching rules are checked; the first exhausted limit rejects the request.

## Provider and model limits route around saturation

A saturated provider or model behaves like one with stale inventory:

* **Virtual-model load balancing** skips the saturated target and picks the
  next available one, so an alias with several targets keeps serving. When
  every target is saturated, the alias resolves to its first target anyway so
  the client gets the honest `429` with `Retry-After` (or failover takes
  over) rather than an unavailable-model error, and the alias stays listed
  in `/v1/models` throughout.
* **Failover rules** take over for a saturated primary route: the primary
  provider is never called (it would happily serve the request and defeat the
  limit), the sweep starts immediately, and rate-saturated candidates are
  skipped along the way.
* **Direct requests** with no failover configured receive `429` with an
  honest `Retry-After` — the same answer the provider itself would eventually
  give, minus the wasted upstream call. Consumer (user path) breaches always
  return `429`: switching targets cannot relieve them.

Token accounting charges the provider and model that actually executed the
request (recorded on the usage entry), so the windows stay correct under
aliasing and failover. Batch submissions skip provider/model rules — the
batch file can mix models, so only user-path rules apply at submission.

Details worth knowing:

* Request windows use a sliding-window estimate, so bursts cannot double up at
  window boundaries.
* Token usage is counted after each response completes (streams included).
  One request can overshoot a token window before the counter catches up —
  the same behavior as other gateways that count tokens from responses.
* Response-cache hits are served before enforcement and count nothing.
* A realtime session counts as one request and holds one concurrency slot for
  the whole session. A batch submission counts as one request and holds no
  concurrency slot.
* Counters live in memory, per gateway instance: with N replicas the
  effective limit is about N times the configured value, and counters reset
  on restart. Use budgets for durable, cross-instance control.

## Client behavior

Requests over a limit receive:

```json theme={null}
{
  "error": {
    "type": "rate_limit_error",
    "message": "rate limit exceeded for /team/alpha: minute request limit of 100 reached",
    "code": "rate_limit_exceeded",
    "param": null
  }
}
```

with status `429` and a `Retry-After` header holding the seconds until a
retry would actually be admitted under the sliding window (which can extend
past the next window boundary). OpenAI SDKs back off automatically on this
shape.

Successful responses on limited paths carry OpenAI-style headers from the
most-constrained matching rule:

```text theme={null}
x-ratelimit-limit-requests: 100
x-ratelimit-remaining-requests: 97
x-ratelimit-reset-requests: 42
x-ratelimit-limit-tokens: 50000
x-ratelimit-remaining-tokens: 31200
x-ratelimit-reset-tokens: 42
```

## Admin API

```bash theme={null}
curl -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  http://localhost:8080/admin/rate-limits
```

Create or update a rule (`user_path` is shorthand for
`"scope": "user_path", "subject": ...`):

```bash theme={null}
curl -X PUT http://localhost:8080/admin/rate-limits \
  -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "user_path": "/team/alpha",
    "limit_key": {"period": "minute"},
    "max_requests": 100,
    "max_tokens": 50000
  }'
```

Provider and model rules name their subject explicitly:

```bash theme={null}
curl -X PUT http://localhost:8080/admin/rate-limits \
  -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "scope": "provider",
    "subject": "openai",
    "limit_key": {"period": "minute"},
    "max_requests": 500
  }'
```

Delete a rule:

```bash theme={null}
curl -X DELETE http://localhost:8080/admin/rate-limits \
  -H "Authorization: Bearer $GOMODEL_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"scope": "provider", "subject": "openai", "limit_key": {"period": "minute"}}'
```

Reset live counters for one rule (`POST /admin/rate-limits/reset-one` with
`{"scope": ..., "subject": ..., "period": ...}`) or for all rules
(`POST /admin/rate-limits/reset` with `{"confirmation": "reset"}`). Editing a
rule never resets its counters; deleting one does.

See [Admin Endpoints](/advanced/admin-endpoints) for the endpoint list.