AppMarketScraper

Rate Limits

API requests are rate-limited based on your subscription plan to ensure fair usage and system stability.

Rate Limit Headers

Every API response includes rate limit headers:

HeaderDescription
RateLimit-LimitYour maximum requests per second (QPS)
RateLimit-RemainingRemaining requests in the current window
RateLimit-ResetUnix timestamp when the limit resets
RateLimit-PolicyRate limit policy (e.g., w=60;limit=10)

Rate Limits by Plan

PlanRequests/Second (QPS)Monthly Credits
Free1200 (one-time)
Pro510,000
Scale1050,000
EnterpriseCustomCustom

Monthly Credit Quotas

Each plan has a monthly credit quota. Credits are deducted for each API request based on the endpoint type.

Endpoint TypeCredits per Request
All endpoints1 credit

When you exceed your monthly quota, you'll receive an error response. Free tier accounts cannot exceed their monthly quota. Paid plans may continue making requests with overage billing (coming soon).

Q: What happens if I exceed my monthly quota?

A: For paid plans, you can continue making requests with overage billing (coming soon). You'll be charged based on your plan's overage rate. Free tier accounts cannot exceed their monthly quota.

Q: Do unused credits roll over?

A: No, unused credits do not roll over to the next month. All quotas reset at the start of each billing cycle.

Q: Can I upgrade my plan mid-month?

A: Yes, you can upgrade your plan at any time. Visit /pricing to view available plans and upgrade. Your new quota will be available immediately, and you'll be prorated for the remainder of your billing cycle.

Handling Rate Limits

When you exceed your rate limit, you'll receive a 429 Too Many Requests response:

{
  "error": "Too Many Requests",
  "message": "Rate limit exceeded. Please retry after 60 seconds."
}

Best Practices

1. Check Rate Limit Headers

const response = await fetch('https://api.appmarketscraper.com/v1/scrape/shopify/apps', {
  headers: { 'x-api-key': 'your_api_key' }
});

const remaining = response.headers.get('RateLimit-Remaining');
const reset = response.headers.get('RateLimit-Reset');
const policy = response.headers.get('RateLimit-Policy');

if (remaining === '0') {
  const waitTime = (parseInt(reset) * 1000) - Date.now();
  await new Promise(resolve => setTimeout(resolve, waitTime));
}

2. Implement Exponential Backoff

async function fetchWithRetry(url, options, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    const response = await fetch(url, options);

    if (response.status !== 429) return response;

    const waitTime = Math.pow(2, i) * 1000; // 1s, 2s, 4s
    await new Promise(resolve => setTimeout(resolve, waitTime));
  }

  throw new Error('Max retries exceeded');
}

3. Use Queuing for Batch Operations

When processing multiple items, use a queue system to control request rate:

class RateLimitedQueue {
  constructor(qps) {
    this.queue = [];
    this.interval = 1000 / qps; // Convert QPS to interval in ms
    this.running = false;
  }

  async add(fn) {
    return new Promise((resolve, reject) => {
      this.queue.push({ fn, resolve, reject });
      if (!this.running) this.process();
    });
  }

  async process() {
    this.running = true;
    while (this.queue.length > 0) {
      const { fn, resolve, reject } = this.queue.shift();
      try {
        const result = await fn();
        resolve(result);
      } catch (error) {
        reject(error);
      }
      await new Promise(r => setTimeout(r, this.interval));
    }
    this.running = false;
  }
}

// Usage
const queue = new RateLimitedQueue(5); // 5 requests per second

const apps = await Promise.all([
  queue.add(() => fetchApp('app1')),
  queue.add(() => fetchApp('app2')),
  queue.add(() => fetchApp('app3')),
]);