AWS WAF Can Now Charge AI Bots for Your Content: How Pay-Per-Crawl Works at the Edge
For most content businesses, AI crawlers have quietly become the largest single source of traffic, and not one penny of it shows up as revenue. The bots scrape articles, product data, documentation, and pricing pages to train and feed AI models, they consume real bandwidth and origin capacity, and they give nothing back. On 15 June 2026 AWS changed the economics of that exchange. AWS WAF now includes an AI traffic monetization capability that lets you charge AI bots and agents for access to your content, and it does it at the network edge before the request ever reaches your servers. For any business that publishes valuable content online, this turns an unpriced cost centre into something you can meter, gate, or sell.
In plain terms, AWS WAF AI traffic monetization allows website owners to charge AI bots for accessing their content directly through AWS WAF. The service can return an HTTP 402 Payment Required response to an AI bot, quote a price for the requested page, verify payment, and then deliver the content within a single request. You can configure per-request pricing based on content paths, bot categories, or verification tiers, while deciding which AI agents should pay, be blocked, or access content for free. Payments are collected in stablecoins and sent directly to a wallet you control, with AWS taking no share of the content revenue and charging no additional fees beyond standard AWS WAF pricing. Available today for Amazon CloudFront customers, this feature requires no changes to your origin infrastructure and no application code modifications.
Why This Matters Right Now
The scale of the problem is the reason AWS built this. AI bot traffic now makes up more than half of all web traffic for many content providers, and AI-specific crawlers have grown more than 300 percent year over year. If you run a media site, a marketplace, a knowledge base, a documentation portal, or any product where the content itself is the asset, you are already paying to serve these bots. You pay in CloudFront egress, in origin compute, in database load, and in the slow erosion of the commercial value of content that took real money to produce.
Until now the only realistic defences were blunt. You could block known AI user agents and hope they did not rotate, or you could publish a robots.txt file and trust crawlers to honour it, which the aggressive ones increasingly do not. Neither option lets you capture value from bots that genuinely want your content and would pay for it. The new capability gives you a third path that sits between block everything and give everything away for free.
How Pay-Per-Crawl Actually Works
The mechanism is built on the x402 open protocol for machine-to-machine payments, and it reuses an HTTP status code that has been reserved for exactly this purpose for decades. When an AI bot requests a page you have chosen to monetize, AWS WAF responds with a 402 Payment Required. The body of that response is a machine-readable price manifest in JSON, and it spells out everything the bot needs to pay: the price of the content in USDC, the blockchain networks you accept such as Base and Solana, the destination wallet address, the maximum time allowed to complete payment, and the payment scheme.
The bot then submits a signed payment authorization on its chosen network. AWS WAF verifies that authorization at the edge, works with third-party facilitator services to settle the payment on chain, fetches the content, and serves it back. From the bot operator’s point of view this all happens in one normal request and response cycle, which is what makes it practical at machine speed. There is no login, no account creation, and no human in the loop, which is the entire point when the customer is itself a piece of software.
Settlement stays in your hands. For each network you provide your own wallet address and set a base price per page in USDC, and disbursement is either self-managed or handled by your wallet provider. AWS does not process the payment or skim a percentage of your content revenue, so the commercial relationship is between you and the bot operator with AWS acting as the gatekeeper rather than the merchant.
You Are Not Limited to Charging
The feature is more flexible than a paywall, and that flexibility is where the real operational value sits. Within each agent verification tier you can assign one of six actions, and only one of them involves taking payment. You can Monetize, which returns the 402 with your pricing. You can Allow, which grants free access, which is sensible for partners or for search engines you actually want indexing you. You can Block, which denies access outright to bots you never want near your content. You can Count, which logs the traffic without charging, so you can measure who is hitting you before you decide on a price. And you can fall back to CAPTCHA or a silent Challenge to confirm a request is coming from a real human browser rather than an agent at all.
That mix means you can run a tiered policy that mirrors how you actually feel about different visitors. Reward the crawlers that send you traffic, charge the ones that only take, block the abusive ones, and keep the human experience untouched. Because the verification and the decision happen at the edge on CloudFront, none of this adds latency to your real users or load to your origin.
The Catch Worth Understanding Before You Switch It On
This is genuinely new ground, so go in with clear eyes. The payment rails are crypto-first today, settling in stablecoins such as USDC on Base and Solana, which means you need a wallet and a basic comfort with on-chain settlement before any money moves. AWS has said that integration with Stripe for direct account payments and support for the Machine Payments Protocol are coming, so if card-based settlement matters to your finance team it may be worth waiting for that. There is a test mode that still enforces the x402 flow but uses testnets such as Base Sepolia and Solana Devnet with faucet funds, so you can prove the whole path end to end without touching real money.
The other thing to weigh is adoption. This only earns revenue if the bots hitting you actually speak x402 and choose to pay rather than walk away. Many will simply leave when they see a 402, which is itself a perfectly good outcome if your real goal is to stop the free scraping. So treat the revenue as upside and the traffic control as the dependable benefit, at least while the ecosystem matures.
What to Do About It
Start by turning on Count mode, not Monetize, and just watch. You cannot price content sensibly until you know which bots are hitting which paths and how often, and Count gives you that picture from the same dashboard without changing anyone’s experience or charging a thing. Give it a week or two and the heavy crawlers and the valuable paths will make themselves obvious.
From there, build a policy that matches your business rather than a blanket rule. Allow the crawlers that genuinely send you traffic, set a price on the high-value paths that AI firms clearly want, block the bad actors, and leave your human-facing routes on Challenge so real visitors never notice. Run the whole thing in test mode against a testnet first so you can confirm the 402 flow, the manifest, and the settlement behave the way you expect before a single live payment moves. Then start small, on one content type or one section, and expand once you trust the numbers on the dashboard.
If you would rather not learn the x402 protocol, wallet settlement, and AWS WAF policy design on production traffic, this is exactly the kind of edge configuration HAZERCLOUD sets up every day. We will audit your current bot traffic on CloudFront, design a tiered allow, block, and monetize policy that fits your content and your risk appetite, and stand the whole thing up in test mode before it ever touches real money. Book a free consultation at https://hazercloud.com/contact/ and we will tell you what your AI bot traffic is really costing you and what it could be worth.
