The sentence saying Meta may use content from your domains to “train” its products is missing from the US-English edition of Meta’s updated Commercial Terms. We loaded the preview on October 17 at about 12:30 UTC. The UK-English edition of the same preview, loaded the same hour, still carries it. We found no announcement of the edit, and the terms still take effect on October 30, 2026.
What did Meta change in the Commercial Terms crawling clause?
In the US-English edition, Meta’s separate sentence saying it “may use any content from such domains” for purposes that include “train” is gone. The crawl permission now ends in a purpose clause tied to “advertising and business products” and “the integrity and security of Meta Company Products”. Captures from Oct 5 and Oct 6, and the text Nieman Lab quoted on Oct 15, carry the old sentence; the US-English page we loaded on Oct 17 does not.
The clause sits in the section “Licenses & Rights You Provide Meta”. Here is the old US wording, from the Wayback capture of October 6:
Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms.
The new US wording, from the Wayback capture of October 17 and the live US-English preview, is the end of the single crawl sentence:
…and to periodically pull updates from such domains, web pages, or URLs in order to provide, develop, improve, research, and maintain advertising and business products (including business, commercial and related products to help you create and deliver ads and other commercial content, provide users information about your products and services, and connect you with users who may be interested in your products and services) and maintain the integrity and security of Meta Company Products.
The UK edition, from the Wayback capture of October 17:
Meta may use any content from such domains, web pages or URLs to provide, develop, improve, research, train and maintain the integrity of, the Meta Company Products, services and features provided under these Commercial Terms and any other applicable supplemental terms.
The terms we read carry no clause saying which edition applies to whom, and we cannot say when or why the US text changed.
What Stayed the Same
Comparing the October 6 capture with the October 17 US page, we found substantive edits in two places: the crawl paragraph (the removed sentence and the new purpose clause) and the content-licence sentence earlier in the same section, which now ends at “maintain Meta Company Products”. On October 6 it continued with “services, and features provided under these Commercial Terms” and was followed by a sentence that the new crawl clause now partly reuses. Unchanged in the US edition:
- Meta may create “test credentials” and “periodically pull updates”.
- The scope still includes pages “made available through your or users’ use of Meta Company Products and services”.
- Consent comes from “continuing any access or use” after notice; the exit is that “you may stop all access or use of our Products for business or commercial purposes”.
- The effective date is October 30, 2026.
The new purpose list still says “develop, improve, research”. It does not say training is excluded, and we draw no legal conclusion. Legal experts quoted by Nieman Lab were commenting on the old wording only.
What the Terms Say About Crawlers and robots.txt
The terms name no crawler and do not mention robots.txt. PPC Land made the same point on October 16: they “neither carve it out nor override it”. Andy Stone, who handles communications at Meta, wrote on X on October 16: “There’s no change to how Meta uses web crawlers. As always, content owners can use robots.txt flags to indicate their preferences.” According to Nieman Lab, a Meta spokesperson said in an email that crawling language had previously been scattered across different terms of use and that the update consolidates it in one place.
Meta’s crawler page (updated May 21, 2026) lists five agents. It does not mention the Commercial Terms or map any clause to an agent. How Meta describes four of them:
- Meta-ExternalAgent “crawls the web for use cases such as training foundation AI models or improving products by indexing content directly.”
- Meta-ExternalAds “crawls the web for use cases such as improving advertising and other business-related products and services.”
- Meta-ExternalFetcher “fetches individual links at a user’s request” and, Meta writes, “this crawler may bypass robots.txt rules.”
- FacebookExternalHit crawls content shared on Meta apps for previews, and “might bypass robots.txt when performing security or integrity checks”.
robots.txt rules apply per user-agent token, so a rule for one agent says nothing about the others. Meta’s page shows example blocks for meta-externalagent and meta-externalads only. Our robots.txt generator has a “Block AI bots” preset whose list of AI user agents includes meta-externalagent; other Meta agents can be typed into the User-agent field. We make no recommendation on any of them. Our earlier piece on Cloudflare’s Disallow AI Training option covers how these controls interact.
The Self-Serve Ad Terms Show the Same Split
The US-English preview of the Self-Serve Ad Terms carries the new paragraph, and its UK-English edition still has “train”. The Self-Serve Ad Terms in force now (effective August 22, 2025) already say that by placing an order an advertiser grants Meta the right to crawl “the content available through your domains, webpages or URLs” for uses that include “research and training”.
What to Check Before October 30
- Reload both editions of the Commercial Terms and Self-Serve Ad Terms previews. The page can change again, and the two may converge or stay apart.
- Look for an announcement of the edit. We found none.
