Why WAF Rules Miss Bots — and Where Bot Management Starts

Short answer: A WAF asks “does this payload match a known attack pattern?”; bot management asks “does this caller behave like a real user?”. The traffic that survives a WAF is usually well-formed, which is precisely why payload inspection has nothing to match on. The plan tiers differ less in how much they block than in what you are allowed to write rules against: Pro and Business give you bot groupings in analytics, while the numeric bot score, JA3/JA4 fingerprints, bot tags and detection IDs only become available to custom rules on Enterprise with Bot Management. Bot Management is a paid product added to an Enterprise plan by your account team — it is not included by upgrading.

Two different questions: the WAF judges the payload, bot control judges the caller

The two systems inspect the same request and look at different evidence.

	WAF	Bot control
Question	Does this payload match a known attack?	Does this caller behave like a real user?
Evidence	Injection syntax, exploit payloads, scanner fingerprints, request anomalies	TLS and HTTP fingerprinting, session behaviour, network reputation, headless browser signals
Output	A rule match against a known-bad pattern	A score from 1 to 99, plus classifications
Strong against	Exploitation, injection, known vulnerability probes	Automation that is syntactically indistinguishable from a real client
Weak against	Legitimate-looking automation using correct credentials	Attacks that carry a genuine exploit payload

The ordering matters: they are stacked, not swapped. Managed rules keep doing the job they were written for after you turn on bot products, and turning bot products on does not reduce your exposure to injection or exploitation.

What makes the distinction practically useful is what abuse traffic actually looks like in logs. A credential-stuffing run sends correct JSON to the right endpoint with a plausible rate. A price-scraping job sends well-formed GET requests with a rotating user agent. A promo-abuse script walks your signup flow the way a user would. In none of these is there an attack payload for a WAF signature to match — the distinguishing evidence is about who is calling, not what they sent.

Why rule-based controls fail against distributed, low-and-slow traffic

Three structural reasons, none of which is fixed by adding more rules:

1. The IP is no longer a stable identity. Residential proxy pools, mobile carrier NAT and cheap cloud hosting rotation mean one operator can present thousands of source addresses. Any control keyed on source IP — including rate limiting — can be spread underneath.
2. The request itself is valid. Correct headers, correct JSON, correct method, plausible interval. There is nothing malformed to match.
3. Thresholds cannot cover low-and-slow. If an attacker needs 20,000 requests per hour and has 5,000 addresses, that is four requests per address per hour. Setting the global threshold low enough to catch it blocks real users behind corporate NAT first.

That produces a predictable pattern of what rules catch and what they miss:

Kind of automation	What rule-based controls catch	What leaks through
Simple scripts (curl, requests)	Header and TLS anomalies; missing JavaScript execution	Little, if your baseline checks are on
Headless browsers (Playwright, Puppeteer)	Partly — JavaScript Detections target these explicitly	Well-configured drivers with patched fingerprints
Residential proxy rotation	Almost nothing — the IP rotates faster than any threshold	Nearly all of it; frequency looks normal per address
Real-device farms and fraud farms	Almost nothing — genuine browser fingerprints, genuine devices	Nearly all of it; only behavioural and business signals distinguish them

The bottom two rows are where the escalation decision gets made. If your incident traffic is row one or two, more rules will help. If it is row three or four, rules are the wrong instrument, because the evidence you need is not in the payload or the address.

What the tiers actually include

Availability is stated per plan in Cloudflare’s bot documentation; the console remains authoritative for your account.

	Free	Pro	Business	Enterprise without Bot Management	Enterprise with Bot Management
Product	Bot Fight Mode	Super Bot Fight Mode	Super Bot Fight Mode	Super Bot Fight Mode	Bot Management
Documented scope	Simple bots from cloud hosting providers and headless browsers	Simple bots and headless browsers	Simple bots and headless browsers	Simple bots and headless browsers	Heuristics, machine learning and JavaScript Detections
Control	Cloudflare issues a computationally expensive challenge, applied to all traffic on the domain	You choose allow, block or challenge	You choose allow, block or challenge	You choose allow, block or challenge	Full rule expression fields
Analytics	—	Limited analytics in a Bot Report	Limited analytics in a Bot Report	Limited analytics in a Bot Report	Granular score, bot tags, detection IDs

Two statements in that documentation are worth quoting because they decide architecture:

• Bot Fight Mode is applied to all traffic across a domain. On Free there is no per-path control; the mitigation is a computationally expensive challenge issued by Cloudflare. That is a reasonable default and a poor fit for an API that receives machine traffic.
• Handling likely automated traffic in bot settings is not available on Pro. Cloudflare’s own table marks that capability as requiring Bot Management. Free, Pro and Business customers get groupings — automated, likely automated, likely human, verified bot — in Bot Analytics, and the finer instrument is the paid product.

The difference that matters: whether you can write rules

Teams compare these tiers by asking “which one blocks more”. That is the wrong axis. The axis is what fields are available to your rules.

Bot settings in Security Settings handle a set of cases automatically, and Cloudflare documents explicitly which of them need custom rules instead:

Capability	Handled by bot settings	Needs custom rules
Block or challenge definitely automated traffic	Yes	Only for path-specific or threshold-tuned rules
Block or challenge likely automated traffic	Not available on Pro	Yes, with Bot Management
Allow or block verified bots	Yes	No
Block AI crawlers	Yes	Only to target individual AI crawlers
Protect static resources	Yes	No

So the practical upgrade trigger is not “we are being attacked” — it is “we need different behaviour on different paths, and we need a threshold we chose”. Domain-wide toggles cannot express /login being stricter than /catalogue, and they cannot express “challenge between scores 2 and 29, block at 1, allow verified crawlers on /public/*”.

The fields that unlock those rules are gated. Cloudflare states that granular bot scores are only available to Enterprise customers who have purchased Bot Management, and that all other customers access the information through bot groupings in Bot Analytics. The same gating applies to Bot Management fields in rate limiting rules: the fields column lists Bot Management fields as Enterprise-with-Bot-Management only.

How the score is produced, and why it is not a verdict

The bot score runs from 1 (Cloudflare is quite certain the request was automated) to 99 (quite certain it came from a human). The documented groupings are:

Grouping	Range
Not computed	0
Automated	1
Likely automated	2–29
Likely human	30–99
Verified bot	Non-malicious automated traffic

Four details decide whether your rules behave the way you expect:

• A score of 0 does not mean safe. It means Bot Management did not evaluate the request — internal Cloudflare service requests, or requests redirected or handled by another feature (such as Redirect Rules) before Bot Management ran. Treat 0 as “no evidence”, not as “human”.
• The engines are Enterprise Bot Management. Heuristics matches against a database of known malicious fingerprints and assigns 1 for high-confidence detections, occasionally 29 while Cloudflare is still assessing traffic overlap. Adaptive Intelligence is the machine learning model behind most detections and retrains frequently rather than on a fixed schedule. JavaScript Detections targets headless browsers. Anomaly Detection exists but Cloudflare is deprecating it and not onboarding new customers.
• Scores move. Model updates do not change your configuration, but they do change scores, so rules keyed on a score threshold should be reviewed periodically.
• The __cf_bm cookie smooths the score across a session to reduce false positives for real users. Rules that key off a single request rather than a session will be noisier.

That is why the recommended response ladder starts with a challenge rather than a block: a challenge converts a probabilistic judgement into an observable outcome, at the cost of one round trip for the caller.

Five signals that rule-based defence has run out

Escalation is worth discussing when two or more of these are true:

1. Your rate limits are being spread. You raise the threshold, the traffic follows; you lower it, real users behind carrier NAT or corporate egress complain.
2. The traffic is distributed and slow. Aggregate volume is significant, per-address volume is below every threshold you can set.
3. Specific endpoints are targeted. Login, signup, password reset, coupon redemption, search and pricing pages attract automation because each call has value. General-purpose rules cannot price that difference.
4. Blanket rules are already costing conversions. You are blocking a whole country, ASN or user-agent family because it is the only granularity you have.
5. You need per-path, graduated responses and the only controls available to you are domain-wide.

If none of these apply, the honest answer is that better-configured existing controls will do more for you than a new product.

A rollout order that starts with bot settings

Step 1: turn on the baseline in Security Settings. Definitely automated traffic, verified bot handling, AI crawler controls and static resource protection are handled automatically and need no custom rules. Do this before writing anything.

Step 2: add custom rules only where the baseline is not expressive enough. Cloudflare’s own guidance is that custom rules are for path-specific or threshold-tuned cases. A rule scoped to /login with a challenge action is a different instrument from a domain-wide block.

Step 3: walk the response ladder, do not jump to the top. Allow → log → challenge → block. Run new rules in Log first where the plan supports it, read what they would have caught, then escalate the action. Note that on Free, Pro and Business, WAF custom rules do not have a Log action — it is Enterprise-only — which makes an observe-then-enforce rollout harder below Enterprise and is worth knowing before you plan one.

Step 4: put rate limiting where the value is, and respect how counters work. Rate limiting rules count per data center, so they are a per-point-of-presence control rather than a global ceiling, and Cloudflare notes a delay of up to a few seconds before a mitigation takes effect. Counting characteristics expand with the plan: IP on Free and Pro; IP with NAT support from Business; and — with Advanced Rate Limiting — query, header, cookie, ASN, country, path, JA3/JA4 fingerprint, JSON field value and body.

Step 5: keep the application-layer controls. Bot control decides whether a caller looks human. It does not decide whether that caller may read a given object, how much a tenant may consume this month, or whether a signup produced a real customer. Those stay in your code — see how to distinguish legitimate users from malicious bots for the evidence layers that sit above the score.

What bot management does not solve

• Business fraud. Promo abuse, fake transactions, referral farming and collusion are business-logic problems: the requests can come from real devices and even real humans, and the evidence sits in account history, fund flows and campaign rules — edge signals are only inputs to that decision.
• Application-layer identity and quotas. Object-level authorisation, per-tenant ceilings and cost caps belong to your stack; a rate limit counter per data center is not a global quota.
• Zero false positives. Every behavioural judgement carries a confidence. Challenges and Turnstile are the buffer, not a guarantee.
• SEO side effects of your own rules. Cloudflare explicitly warns that applying rate limiting rules to verified bots may affect search engine optimisation.
• Origin exposure. If your origin IP is publicly reachable, automation can go around the edge entirely — that is a separate problem with a separate fix.

FAQ

Is Bot Management included when I upgrade to Enterprise? No. Bot Management for Enterprise is added to an Enterprise plan by your Cloudflare account team; upgrading the zone plan does not include it. Enterprise without Bot Management receives Super Bot Fight Mode, the same product Business receives.

Do I still need the WAF if I have Bot Management? Yes. They answer different questions and are evaluated on the same request. Managed rules continue to cover injection, exploitation and known vulnerability probes, none of which bot detection is designed to replace.

What bot score should I block at? There is no universal threshold. The documented groupings give you a starting point — 1 for automated, 2–29 for likely automated, 30–99 for likely human — and the safe pattern is to challenge the 2–29 band and block at 1, then tune per path. Two cautions: a score of 0 means the request was not evaluated, not that it is safe; and scores shift as the machine learning model retrains, so review score-based rules periodically.

Can I use bot score in rate limiting rules on Business? Bot Management fields in rate limiting rules are documented as available to Enterprise customers who have purchased Bot Management. On Business the counting characteristics are IP and IP with NAT support, and the available expression fields are host, URI, path, full URI, query, method, source IP, user agent and verified bot.

Will blocking bots hurt my search rankings? It can. Cloudflare warns that applying rate limiting rules to verified bots may affect SEO, and verified bots include the crawlers that power search engines. Handle verified bots through bot settings rather than a blanket block, and use custom rules only when you need to target an individual crawler.

When is Bot Management not worth buying? When your traffic is simple scripts and headless browsers that JavaScript Detections and Super Bot Fight Mode already handle, when no specific high-value endpoint is being targeted, and when your losses come from business abuse that no traffic classification will detect. In those cases better-scoped custom rules, a challenge ladder and per-path rate limiting deliver more than the product does.

Related reading

• How to Distinguish Legitimate Users from Malicious Bots
• How to Prevent Malicious Web Scraping
• Preventing Automated Signups and Free-Tier Abuse in AI Apps
• How to Mitigate HTTP Flood and Layer 7 Attacks

If you are deciding whether behavioural detection is justified on your traffic, the useful starting point is your own logs: which endpoints are targeted, at what per-address rate, and what your current rules already catch — review the architecture with us.