​
Overview

Our API implements rate limiting to ensure fair usage and maintain service stability. There are three types of rate limits:

  1. Daily rate limit
  2. Burst rate limit
  3. Chat-based rate limit

When a rate limit is exceeded, the API returns a 429 status code along with specific headers that provide information about the limit, remaining requests/tokens, and reset time.

​
Daily Rate Limit

This is a daily limit applied to publish-related API endpoints (new message requests like publish, enqueue, or batch). Other API requests, such as fetching events or messages, do not count toward this limit.

Headers:

  • RateLimit-Limit: Maximum number of requests allowed per day
  • RateLimit-Remaining: Remaining number of requests for the day
  • RateLimit-Reset: Time (in unix timestamp) when the daily limit will reset

​
Burst Rate Limit

This is a short-term limit per second to prevent rapid bursts of requests. This limit applies to all API endpoints.

Headers:

  • Burst-RateLimit-Limit: Maximum number of requests allowed in the burst window (1 second)
  • Burst-RateLimit-Remaining: Remaining number of requests in the burst window (1 second)
  • Burst-RateLimit-Reset: Time (in unix timestamp) when the burst limit will reset

​
Chat-based Rate Limit

This limit is applied to chat-related API endpoints.

Headers:

  • x-ratelimit-limit-requests: Maximum number of requests allowed per day
  • x-ratelimit-limit-tokens: Maximum number of tokens allowed per day
  • x-ratelimit-remaining-requests: Remaining number of requests for the day
  • x-ratelimit-remaining-tokens: Remaining number of tokens for the day
  • x-ratelimit-reset-requests: Time (in unix timestamp) until the request limit resets
  • x-ratelimit-reset-tokens: Time (in unix timestamp) when the token limit will reset

​
Example Rate Limit Error Handling

Handling Daily Rate Limit Error
import { QstashDailyRatelimitError } from "@upstash/qstash";

try {
  // Example of a publish request that could hit the daily rate limit
  const result = await client.publishJSON({
    url: "https://my-api...",
    // or urlGroup: "the name or id of a url group"
    body: {
      hello: "world",
    },
  });
} catch (error) {
  if (error instanceof QstashDailyRatelimitError) {
    console.log("Daily rate limit exceeded. Retry after:", error.reset);
    // Implement retry logic or notify the user
  } else {
    console.error("An unexpected error occurred:", error);
  }
}
Handling Burst Rate Limit Error
import { QstashRatelimitError } from "@upstash/qstash";

try {
  // Example of a request that could hit the burst rate limit
  const result = await client.publishJSON({
    url: "https://my-api...",
    // or urlGroup: "the name or id of a url group"
    body: {
      hello: "world",
    },
  });
} catch (error) {
  if (error instanceof QstashRatelimitError) {
    console.log("Burst rate limit exceeded. Retry after:", error.reset);
    // Implement exponential backoff or delay before retrying
  } else {
    console.error("An unexpected error occurred:", error);
  }
}
Handling Chat-based Rate Limit Error
import { QstashChatRatelimitError, Client, openai } from "@upstash/qstash";

try {
  // Example of a chat-related request that could hit the chat rate limit
  const client = new Client({
    token: "<QSTASH_TOKEN>",
  });

  const result = await client.publishJSON({
    api: {
      name: "llm",
      provider: openai({ token: process.env.OPENAI_API_KEY! }),
    },
    body: {
      model: "gpt-3.5-turbo",
      messages: [
        {
          role: "user",
          content: "Where is the capital of Turkey?",
        },
      ],
    },
    callback: "https://oz.requestcatcher.com/",
  });
} catch (error) {
  if (error instanceof QstashChatRatelimitError) {
    console.log("Chat rate limit exceeded. Retry after:", error.resetRequests);
    // Handle chat-specific rate limiting, perhaps by queueing requests
  } else {
    console.error("An unexpected error occurred:", error);
  }
}

Was this page helpful?

Create Chat CompletionOverview