Rate Limiting AI APIs: Keys, Thresholds and False Positives

Short answer: A rate limiting rule is four decisions: which requests match, what you count them by, how many are allowed in what period, and what happens when the limit is reached and for how long. AI endpoints get the second one wrong — counting by IP when what you need to control is the tenant, the API key or the token budget. Free and Pro can only count by IP, Business adds NAT support, and Query, Host, Headers, Cookie, ASN, Country, Path, JA3/JA4, JSON field values, body, form input and custom expressions all require Enterprise. One more thing the docs state plainly: rate limiting rules are “not designed to allow a precise number of requests to reach your origin”, because counters are scoped per data centre and lag by a few seconds. It is a load-shedding tool, not a quota system.

Rate limiting is not a quota system

The rate limiting docs contain a sentence worth reading twice: these rules are “not designed to allow a precise number of requests to reach your origin server”. Counters are scoped per data centre, and there can be a delay of up to a few seconds between Cloudflare detecting a request and updating its counters. During that gap, excess requests still reach your origin.

For an ordinary web API, a few seconds of slop rarely matters. For an AI endpoint it does: one request may hold a GPU for tens of seconds, and the excess that slips through in those few seconds is enough to fill a queue and define that minute’s bill.

So rate limiting belongs in the right slot in your mental model. It reduces abnormal volume and shaves peaks at the edge. Quotas, quota decrementing and billing have to live in your own service. Treating the first as the second is the most common — and most expensive — mistake in this corner of configuration.

One more engine behaviour to keep in mind: rate limiting rules are evaluated in order, and actions such as Block stop the evaluation of later rules. Get the order wrong and a broad rule can shadow every finer rule behind it.

The four parameters of a rule

What matches. A rule expression selects the requests this rule applies to. The fields you can use grow with the plan: Free has Path and Verified Bot, Pro adds Host, URI, Full URI and Query, and Business adds Method, Source IP and User Agent. Enterprise can use general request fields, request header fields, Bot Management fields and request body fields.

What you count by. These are the characteristics, and they decide which requests share a bucket. Free and Pro count by IP only. Business counts by IP with NAT support. Enterprise adds Query, Host, Headers, Cookie, ASN, Country, Path, JA3/JA4 fingerprint, JSON field value, body, form input value and custom expressions.

How many, in how long. Period is the counting window and requests per period is the threshold. Supported values run from 10 seconds to 65,535 seconds, but the ceiling differs by plan: Free is fixed at 10 seconds, Pro allows up to 1 minute, Business up to 10 minutes, Enterprise up to 65,535 seconds.

What happens, and for how long. Duration is the mitigation timeout: 10 seconds on Free, up to 1 hour on Pro, up to 1 day on Business and Enterprise, with Enterprise also able to set a custom timeout through the API. By default the action applies for the whole timeout regardless of whether the rate has dropped; Enterprise can switch to throttling, which blocks only while the rate is above the limit and lets traffic through again once it falls below.

Four ways the counting key goes wrong

Counting by IP blocks a crowd of real users. Behind a corporate egress, a campus network or carrier-grade NAT sit hundreds or thousands of genuine users sharing a handful of addresses. One of them running a batch job gets all of them blocked.

Counting by IP also misses distributed low-and-slow traffic. In a residential proxy pool every address sends very little, so your threshold is never reached. That traffic is only visible through non-IP signals: JA3/JA4 fingerprints, Bot Management fields, behavioural sequences.

The thing you need to control is in the header or the body. Tenant IDs, API keys, model names and max_tokens all live in headers or bodies. Counting “per tenant per minute” means counting on Header, Cookie or JSON field value — all of which are Enterprise counting characteristics.

Counting requests does not control cost. What a request costs depends on the model and the token count, not on the request. At the edge you can only approximate it: split traffic by Path or a JSON field and set one threshold per model tier. Alternatively, use the complexity score model, which counts by request complexity rather than request count and belongs to Advanced Rate Limiting.

A useful order for choosing the key: ask whose money you are protecting. If it is your GPU budget, count by tenant or key. If it is one user misbehaving, count by session or user identifier. Counting by IP is right only when what you are actually defending against is brute-force traffic from a single source.

What each plan can count

	Free	Pro	Business	Enterprise (with app security)	Enterprise + Advanced Rate Limiting
Number of rules	1	2	5	100 (contract-dependent)	100
Counting characteristics	IP	IP	IP, NAT	adds Query, Host, Headers, Cookie, ASN, Country, Path, JA3/JA4, JSON field, body, form input, custom	same, plus custom counting expression
Counting period	10 s	up to 1 min	up to 10 min	up to 65,535 s	up to 65,535 s
Mitigation timeout	10 s	up to 1 h	up to 1 day	up to 1 day, custom via API	up to 1 day
Fields in expression	Path, verified bot	adds Host, URI, Full URI, Query	adds Method, source IP, user agent	general request fields, headers, Bot Management fields, body	same
Also	—	—	cache exclusion, custom counting expression	throttle above the rate	complexity score model

Three caveats: JA3/JA4 and Bot Management fields require a Bot Management purchase; request body fields depend on your WAF plan; and Enterprise customers must have application security on their contract to get rate limiting rules at all, with the rule count set by contract terms.

Deriving a threshold instead of guessing one

Work backwards from cost. Compute the variable cost of one call — token price times average tokens, plus the inference resource it occupies — then decide the largest spend per minute you will tolerate. Divide by the cost per call and you have requests per minute. That number is where the threshold starts.

Work backwards from capacity. Look at what your GPU queue or concurrency ceiling can absorb, then divide it across tenants or plan tiers. The quantity to control is how much one tenant can take, not the global total.

Work backwards from normal behaviour. Take the p99 of legitimate peak traffic and leave headroom above it. Do not pick 100 or 1,000 because they look tidy; round numbers have no relationship to your cost structure.

Observe before you enforce. Run a new rule with a recoverable action such as a challenge and watch who it catches. Tighten to block only once the false positive rate is acceptable. Whether you can log without acting depends on the plan: the Log action on WAF custom rules is Enterprise-only.

Short periods and long timeouts punish normal bursts

AI clients are bursty by nature. An agent handed a task may fire dozens of requests and then go quiet. With a 10-second period and an hour-long mitigation timeout, one legitimate burst locks that user out for an hour.

A better combination is a long period, a short timeout and throttling: stretch the period so bursts have room, shorten the timeout so blocked callers recover quickly, and turn on throttling so traffic is admitted again once the rate drops instead of being blocked for the entire mitigation window. The first two are configurable on any plan; throttling requires Enterprise.

One option is easy to miss: Business and above support cache exclusion, which lets a rule count only requests that miss cache, so repeated cache hits do not inflate the counter.

Who you will block by accident

• Shared egress. Corporate, campus and mobile users share a small number of addresses, with many people behind each one.
• Mobile networks. The address can change mid-session, so counting by IP both over-blocks and under-counts.
• Third-party integrations. A partner calling through one key or one gateway looks like a single user, and blocking it blocks all of them.
• Legitimate automation. The docs warn explicitly that applying rate limits to verified bots can affect SEO. Crawlers, monitoring and the legitimate parts of AI agent traffic need their own path.
• What blocked visitors see. The docs direct blocked visitors to the error 1015 page, so counting 1015s is one usable signal that a limit is actually firing.

The order for handling false positives: challenge before you block; add skip rules for sources you know matter; and keep watching challenge pass rates and error rates on critical paths rather than configuring once and walking away.

Beyond rate limiting: detection, session abuse and an order to follow

Rate limiting controls volume. It does not decide whether a call is legitimate. That part needs detection:

• Schema validation. Schema Validation 2.0 produces an always-on detection result and does not mitigate traffic by itself; you enforce it with a WAF custom rule on cf.schema_validation.uploaded.violated.
• JWT validation. The docs note it does not rotate JWKS automatically, only validates tokens found in headers or cookies, and needs OPTIONS preflight requests to be allowed through.
• Volumetric abuse detection. Enterprise only: buckets requests per session in 10-minute windows over a 7-day period, returning p50/p90/p99 and a confidence score. Rate limiting by session on top of it requires Advanced Rate Limiting.
• Sequence analytics. Reconstructs call sequences per session identifier, with sequences up to 9 operations long and scoring based on the last 24 hours.
• Session identifiers. Up to 10; if none is configured and Authorization appears in more than 1% of eligible sampled requests, Cloudflare configures one automatically.

An order you can actually run: list your endpoints and the cost of one call each; pick a counting key per endpoint, preferring tenant or key over IP; observe with a challenge action for a day or two; set thresholds per plan tier; layer schema and JWT validation to reject malformed calls; then wire false positive rates, 1015 counts and origin error rates into monitoring.

The boundary is worth stating: the edge can shed load using the fields it can see, but it does not know how much quota a tenant has left this month, or whether a given call should be billed. Only your own API layer knows that.

FAQ

Why does IP-only rate limiting keep failing on AI endpoints? It fails in two directions at once. Corporate egress and CGNAT hide large numbers of real users behind few addresses, so they get blocked together; residential proxy pools send too little per address to ever trip a threshold. When the thing you control is a tenant or a key, count on a header or a JSON field value.

What number should the threshold be? Derive it from cost or capacity rather than guessing. Work out the variable cost of one call, set the largest spend per minute you will accept, and divide; or divide GPU queue capacity into per-tenant shares.

Can I rate limit by tokens or by cost? The edge never sees real token consumption. The workable approximations are per-model-tier thresholds keyed on Path or a JSON field value, or the complexity score model, which counts by request complexity and belongs to Advanced Rate Limiting.

Will this hurt real users and AI agents? Yes, especially with a short period and a long timeout. Use a long period, a short mitigation timeout and throttling so you shed peaks instead of cutting traffic off, and add skip rules for verified bots and known partners.

Do I need Enterprise for this to work? It depends what you need to count by. IP-only scenarios work on lower plans. Counting by tenant, header, JSON field or model tier — or throttling, or counting by complexity — requires Enterprise, and complexity scoring additionally requires Advanced Rate Limiting.

How do I know the rule is doing anything? Three places: the Cloudflare error page returned to blocked visitors (the docs point them at the 1015 page), the rule’s hit and action analytics, and whether origin error rates and queue depth actually fall.

Related reading

• Managing API Costs and Abnormal Traffic in AI SaaS
• How to Prevent API Abuse in AI SaaS
• How to Mitigate API Key Abuse and Unauthorized Access
• Cloudflare WAF capabilities

If your inference bill no longer matches your usage and you are unsure which dimension to count on, send us your endpoint list and cost structure — we will work through the rules with you.