API rate limits are a product decision, not only a gateway setting

A rate limit should protect the slowest consequential dependency and distribute capacity fairly among customers. A single global number at the reverse proxy is rarely enough when one tenant can flood a CRM or reporting API used by everyone else.

Inventory the bottlenecks

Separate cheap reads, expensive reports and writes with side effects. Note the upstream provider's published quotas, database connection capacity and business priority. A traffic burst caused by a legitimate import is not an attack, but it can cause the same outage.

Build a capacity budget from the dependency backward

If an upstream CRM accepts only a bounded number of concurrent writes, derive the portal's concurrency limit from that constraint, leaving room for staff operations and retries. A per-tenant request rate protects fairness, while a global concurrency cap protects the provider. They solve different problems and both may be needed.

A queue can absorb a short burst only when its wait time remains acceptable and the operation is safe to defer. Display accepted, processing and completed states rather than pretending the work finished. For interactive search, rejecting quickly may be better than forcing customers to watch a loading spinner for minutes.

Choose admission behavior per operation

Return a clear status and documented retry policy. Clients should use bounded exponential backoff with jitter and respect Retry-After where supplied. Do not turn every rejected write into an invisible queue: users need to know whether the action was accepted.

HTTP/1.1 429 Too Many Requests
Retry-After: 30
Content-Type: application/json

{"error":"rate_limited","scope":"tenant_export"}

Test the client contract

A well-behaved client should distinguish a 429 from a 500, honor Retry-After and stop after a bounded budget. Add jitter so every client does not retry at the same instant. For a write, carry the same idempotency key across retries; otherwise a timeout followed by a new attempt can create two records.

Test one large tenant alongside several small tenants during a simulated provider slowdown. If small tenants stop getting service, the policy is not fair even if total throughput looks healthy. Track rejected legitimate requests and task completion time, not only requests per second.

Do not treat IP address as customer identity. Offices, VPN exits and mobile networks share addresses; a malicious caller can also rotate them. Authenticate before applying the tenant quota when possible, and keep a separate coarse unauthenticated limit for abusive traffic.

Protect fairness and recovery

Measure queue depth, request rejection rate and upstream error rate together. If the dependency slows, lower concurrency before retries multiply load.

A limit that blocks your biggest legitimate customer at month-end without notice is a product defect. Document burst allowances, surface retryable errors and rehearse capacity changes. The cheapest solution may be batching or a better query rather than a more elaborate limiter.