[API reference]

Rate limits

Limits are set per team and shared by every key on it. They scale with your plan and are reported on every response.

Limits by plan

PlanRequests / minTokens / minStreamsImages / min
Pay as you go60200k1020
Builder6002M50100
Max3,00010M200500
EnterpriseCustomCustomCustomCustom
Streams are concurrent open streaming responses.

Rate limit headers

HeaderDescription
x-ratelimit-limit-requestsRequests allowed per minute.
x-ratelimit-remaining-requestsRequests left in the current window.
x-ratelimit-reset-requestsTime until the request window resets, such as 1.7s.
x-ratelimit-limit-tokensTokens allowed per minute.
x-ratelimit-remaining-tokensTokens left in the current window.
retry-afterSeconds to wait. Sent only with 429.
response headers
http
HTTP/2 200x-topo-request-id: req_7Hq2kf91x-ratelimit-limit-requests: 600x-ratelimit-remaining-requests: 583x-ratelimit-reset-requests: 1.7sx-ratelimit-limit-tokens: 2000000x-ratelimit-remaining-tokens: 1942310

Staying under the limit

  • Batch embeddings: one request can carry 2,048 inputs.
  • Set max_tokens. Token limits count the requested maximum until the response finishes.
  • Give each agent its own key with a spend limit, so one runaway loop cannot starve the rest of the team.
  • Priority routing on Builder and Max shortens queues at peak, but does not raise the limits.