Cloudflare Announces New Initiatives to Advance the "Agentic Internet"

(June 30, 2026, 6:00 AM PDT)

Cloudflare, Inc. (NYSE: NET), the leading connectivity cloud company that powers frictionless connectivity for everyone, today announced new initiatives to advance the "agentic internet." At the heart of these efforts are a new classification system to bridge website owners and AI companies, enhanced analytics, and industry-defining partnerships. These will help owners and AI companies make websites more discoverable, while optimizing efficiency and profitability. By building this new foundation, Cloudflare aims to foster a healthy and harmonious "agentic economy" that benefits website owners, AI companies, and society as a whole.

The nature of internet traffic has fundamentally changed. Today, more than half of all web requests are made by automated agents and bots. AI is becoming the new interface for accessing information and conducting commerce online, and the rules for content discovery are beginning to change. Most website owners want their content to be incorporated into AI systems. However, for businesses that rely on advertising and subscriptions for revenue, the free use of their content to train AI systems is a threat.

Cloudflare believes that if AI is to be fed content, the process should be as simple and efficient as possible. Conversely, if it is not desired, tools should be provided to restrict AI access. To this end, Cloudflare is testing new default classifications, providing deeper insights, accelerating AI search, and ensuring creators are compensated when their content is used to generate AI responses. The goal is a level playing field. By optimizing web crawling efficiency while protecting intellectual property, Cloudflare aims to achieve an "agentic economy" where creators and AI can thrive together.

"Last year, Cloudflare gave website owners the transparency and control to understand which bots were accessing their content, and we were thrilled to see the positive impact it had on the ecosystem," said Matthew Prince, co-founder and CEO of Cloudflare. "With non-humans now generating the majority of internet traffic, we must choose to act quickly and decisively to build a sustainable ecosystem. Cloudflare's new tools and partnerships will provide website owners with increased visibility and new business opportunities, while also benefiting AI companies that operate with clear intent and transparent bots. We expect that the default setting changes Cloudflare is proposing will help curb the proliferation of unconstrained, multi-purpose crawlers and clarify their usage."

The Day of the "Multi-Purpose Crawler" That Uses Content for Free

One thing has become clear through conversations with AI companies and website owners: most websites want to be discovered by AI. However, for sites that generate revenue through advertising or subscriptions, increasing discoverability by AI cannot come at the cost of providing their intellectual property for free. Transparent AI companies are currently addressing this need by clearly distinguishing bot intentions into search, agent usage, and training. However, multi-purpose bots also exist, crawling with a mix of intentions. The presence of these bots is highly disruptive to both site owners and transparent AI companies. Content owners face a dilemma: to increase their chances of being discovered in the age of agentic AI, they must offer their most valuable content to crawlers for free.

Over the next two months, Cloudflare will engage with the ecosystem, analyze feedback, and conduct testing to finalize new default settings and classifications by September 15, 2026.

For new Cloudflare customers and new sites for existing customers, the default setting from September 15, 2026, will be: "Allow search for pages with ads, but block training and agent usage." Multi-purpose crawlers will be blocked on all pages with ads unless they present website owners with a choice of whether to crawl for search, agent usage, or training. These settings can be changed at any time in the dashboard.

Existing customers on the free plan who do not change their settings in the dashboard by September 15, 2026, will also see these changes applied on September 15, 2026. The purpose of these changes is to create a level playing field for AI companies and traditional search engines that are redefining discoverability in the agentic AI era in a fair and transparent manner.

Enhanced Analytics to Increase Visibility and Drive Content Monetization

Cloudflare is introducing a new "Attribution Business Insights" dashboard. This dashboard will allow any stakeholder, not just security or IT teams, to clearly understand how their content is being consumed by AI bots and track how much human traffic each AI company is actually generating. As a result, the data gap that has forced many content owners to negotiate for years with insufficient information will be resolved.

However, understanding how AI consumes content is only the first step. As the landscape of content discovery shifts from search engines to AI-powered answer engines, a new discipline is emerging: "Answer Engine Optimization" (AEO). AEO is the practice of understanding not only who is crawling, but also how, how often, and how prominently your content is cited in AI-generated answers across all AI models. Just as web analytics became essential infrastructure in the search era, Cloudflare aims to provide the definitive measurement and optimization layer for the AI era.

Smarter Search and Fairer Compensation

Cloudflare is advancing two initiatives to support both creators and AI companies: streamlining AI search and automating compensation when creators' work is used. Currently, AI models and search engines crawl millions of un-updated web pages millions of times to find fresh data to answer user queries. Cloudflare data shows that over 50% of crawl traffic from AI crawlers is spent re-fetching un-updated pages. This results in wasted bandwidth and compute resources, without necessarily yielding better answers. Cloudflare's capabilities allow it to understand whether a web page has been updated and what has actually changed. This allows it to signal whether a page is worth re-fetching, thereby reducing unnecessary crawling and improving answer accuracy. Cloudflare is working with leading AI companies to test these signals, measure improvements in data freshness and reductions in wasted crawling, with general availability planned for late 2026.

Cloudflare announced the first version of "Pay Per Crawl" on Content Independence Day 2025, providing a mechanism for content owners to charge AI companies for crawling their content. Today, "Pay Per Crawl" evolves to "Pay Per Use." Owners will be compensated not just when their content is acquired, but when it actually generates value. Cloudflare is working with top partners like Ceramic.ai and You.com to bring this principle to market. With Ceramic.ai, opted-in owners will be compensated each time their content appears in Ceramic's AI search results, with queries, citations, and search rankings provided. Other AI companies may bring their own models tailored to their business. For example, You.com has adopted a model where they pay on-demand for specific premium content required by each agent, and Cloudflare provides the common underlying layer.

One Year of "Content Independence Day" and Features Released to Date

Content Independence Day 2025 was primarily about "defense" – focusing on providing tools for creators to regain control of their data. Over the past 12 months, Cloudflare has been working diligently to build the infrastructure needed to move from mere protection to proactive support. Today's announcement culminates this foundational work, delivering on three pillars that support a sustainable ecosystem: control, transparency, and monetization. Last year, Cloudflare launched "AI Crawl Control," a permission-based AI crawling model. To democratize this protection, Cloudflare partnered with platforms like beehiiv to provide a detailed dashboard that allows everyone from independent creators to major media outlets to easily track, block, or permit specific AI models. Cloudflare also pioneered "Web Bot Auth" to fundamentally protect interactions with agents. This is an open framework for AI bots and autonomous shopping agents to prove their identity and legitimacy to creators and merchants. Cloudflare has also partnered with leading payment service providers to build the payment infrastructure underpinning the emerging agentic commerce economy. Building on "Web Bot Auth," this supports AI shopping agents in transacting securely with millions of merchants worldwide without exposing raw credentials. These efforts ultimately contribute to building the foundation for a sustainable agentic economy, ensuring the next generation of the internet is built on the principles of transparency, discoverability, and fair compensation.

For more details on Cloudflare's vision, please see the following resources:

Content Independence Day, one year on: building the business model for the agentic Internet

https://blog.cloudflare.com/agentic-internet-bot-report/

Your site, your rules: new AI traffic options for all customers

https://blog.cloudflare.com/content-independence-day-ai-options/

Unmasking the crawls with Attribution Business Insights

https://blog.cloudflare.com/attribution-business-insights/

Making AI search smarter

https://blog.cloudflare.com/making-ai-search-smarter

About Cloudflare

Cloudflare, Inc. (NYSE: NET / https://www.cloudflare.com/ja-jp/ ) is the leading connectivity cloud company on a mission to help build a better Internet. Cloudflare's connectivity cloud provides a unified, built-in platform of security, performance, and zero-trust services that protects and accelerates everything organizations deliver online. Cloudflare delivers this integrated platform through its global network, which is the largest and most interconnected of its kind.

More than 265 million Internet properties rely on Cloudflare for their security, performance, and reliability. Cloudflare's connectivity cloud is designed to help organizations of all sizes become more agile, more efficient, and more innovative.

For more information about Cloudflare's connectivity cloud, visit https://www.cloudflare.com/ja-jp/connectivity-cloud/ , and for the latest trends and insights on the Internet, visit https://radar.cloudflare.com .

Follow us: Blog | X | LinkedIn | Facebook | Instagram

Forward-Looking Statements

This press release contains forward-looking statements within the meaning of the Private Securities Litigation Reform Act of 1995 (and the general legal principles applicable in the jurisdictions where Cloudflare operates) that involve risks and uncertainties. These forward-looking statements are identified by their use of terms and phrases such as "may," "will," "should," "expects," "plans," "anticipates," "could," "intends," "target," "projects," "contemplates," "believes," "estimates," "predicts," "potential," "ongoing," or other similar expressions that predict or indicate future events or trends or that are not statements of historical matters. The achievement of the events described in this press release depends on the following factors, among others: the features and effectiveness of Bot Defaults, the Attribution Business Insights dashboard, Pay Per Use, and other Cloudflare products and technologies; the benefits customers will gain from using Bot Defaults, the Attribution Business Insights dashboard, Pay Per Use, and other Cloudflare products and technologies, and integrating them with products from Ceramic.ai and You.com; Cloudflare's ability to acquire new customers and increase sales to existing customers by integrating its products with products from Ceramic.ai and You.com; the timing of the general availability of Bot Defaults, the Attribution Business Insights dashboard, Pay Per Use, or related features to all of Cloudflare's existing and potential customers; the timing of the development, beta, and general availability of Bot Defaults, the Attribution Business Insights dashboard, Pay Per Use, or related features to all of Cloudflare's existing and potential customers; Cloudflare's technological development, future business prospects, growth, initiatives, or strategies; and statements made by Cloudflare's CEO and others. Actual results could differ materially from those predicted or indicated in any forward-looking statements, as a result of various factors, including, but not limited to, the risks described in Cloudflare's filings with the SEC, including those described in its quarterly report on Form 10-Q filed with the SEC on May 8, 2026, as well as Cloudflare's other filings with the SEC.

Cloudflare does not undertake any obligation to update the forward-looking statements contained in this press release to reflect events that occur or circumstances that exist after the date on which they were made. Such forward-looking statements are based on Cloudflare's beliefs and assumptions as of the date of this press release. In addition, any forward-looking statements contained in this press release are made only as of the date hereof. Cloudflare may update the forward-looking statements included in this press release in the future, but does not promise or guarantee to do so. Cloudflare's plans, intentions, and expectations are based on current assumptions and may change. You should not rely on forward-looking statements because they relate to matters that have not yet occurred.

© 2026 Cloudflare, Inc. All rights reserved. Cloudflare, the Cloudflare logo, and other Cloudflare marks are trademarks or registered trademarks of Cloudflare, Inc. in the United States and other jurisdictions. Other marks and names mentioned herein may be trademarks of their respective owners.