Andy Garcia · freelance dev
FRENES
Back to the blog
AI & Terms

Meta Deleted the AI Training Sentence From the English Text of Its New Business Terms. The Permission to Crawl Your Site Stayed.

October 19, 2026 7 min read

On Friday, October 16, Meta removed one sentence from the new version of its Commercial Terms, the terms that apply whenever a business uses Meta's products, Facebook and Instagram included: a Page, ads, a product catalog, marketing messages, Meta Business Suite. The new terms take effect on October 30, two weeks later. The sentence read:

"Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms."

"Such domains" meant the websites associated with your business. The sentence is gone from the English text. The paragraph that authorizes Meta to crawl those websites, with test credentials, is still there.

To be clear about what was in force on October 19: nothing in this story. Until the new version takes effect on October 30, the Commercial Terms dated March 6, 2024 apply, and they contain no crawling paragraph. A narrower permission already exists for advertisers who use Meta's generative AI ad tools: the Ad Creative Generative AI Terms, in force since May 6, 2024, let Meta crawl the URLs you provide "for your use in connection with your ads." The question is what you accept on October 30 by continuing to use Meta's business tools.

The arc, with dates

Meta announced the update in early October, in a blog post and in Meta Business Suite, according to Nieman Lab. The Internet Archive holds a copy of the preview page from October 5 with the full clause and the October 30 effective date.

On October 15, Nieman Lab's Andrew Deck and Hanaa' Tameez published the reading of three legal experts: as written, the terms could let Meta scrape sites associated with business accounts, news publishers included, and train its AI on the content. "For news publishers, this clearly represents permission to Meta to train AI on any content from their sites," said James Grimmelmann, a professor at Cornell Law School. On the "test credentials" in the clause, he said they would technically allow Meta to "sign up as a fake user" to get past registration walls. Benjamin Sobel, a law professor at the University of Wisconsin, said that tethering access to the business tools to broad scraping permissions echoes "tying", a concept in antitrust law. And there was no way to opt out before October 30, other than to stop using Meta's products for business.

On October 16, Meta pushed back. Spokesperson Daniel Roberts said, in a statement The Verge published: "There is no change to how Meta uses web crawlers; any suggestion to the contrary is false." Andy Stone, Meta's vice president of communications, wrote on X that the update was "designed to make sure we're able to pull accurate information about products for sale on third-party websites when making on-platform ad and content recommendations, not about scraping news sites." Publishers' groups went further: Danielle Coffey, president and CEO of the News/Media Alliance, called the terms "outrageous and probably illegal," Nieman Lab reported.

The rewrite came the same day, US time. Nieman Lab says Meta published its revision on Friday afternoon, and the Internet Archive's copies of the preview page bracket it: the sentence is there at 16:20 UTC on October 16 and gone at 03:19 UTC on October 17.

On October 17, The Verge reported that the training sentence no longer appeared, with a new statement from Roberts: "We've heard you and want to clarify: the crawling terms we consolidated only applied to our ads and business products, and we've clarified that in the terms themselves. It does not change how Meta uses web crawlers. As always, content owners can use robots.txt to signal their preferences." TechCrunch reported the removal the same day, citing The Verge.

The three legal experts Nieman Lab consulted remained skeptical. In a follow-up on October 17, Matt Topic, a media and intellectual property attorney at Loevy & Loevy, said: "I think 'develop' is broad enough to include 'train.'" Grimmelmann said the "develop…advertising and business products" wording "is still broad enough to drive a truck through."

What the October 30 text says now

I read the preview page on October 19 and compared it with the October 5 copy. The authorization survives almost word for word. You authorize Meta, "or any entity acting on our behalf", to use "web scrapers, bots, robots, spiders, crawlers, and user agents" and to "create test credentials" to access, index, cache, analyze, collect, retrieve or crawl the domains, pages and URLs "associated with your products or services", or "made available through your or users' use of Meta Company Products and services", plus "any content available through them", and to "periodically pull updates" from them.

What changed is the purpose. "Use any content" and "train" are gone. The crawling is now "in order to provide, develop, improve, research, and maintain advertising and business products", with a parenthesis listing ads, product information for users and matching you with interested users, "and maintain the integrity and security of Meta Company Products."

That is a real narrowing. It matches what Stone described: reading product pages for ads. It is not a removal of the crawling, and how narrow it is depends on words the contract does not define: Topic pointed out that "business product" does not seem to be a defined term.

Two smaller things only show up in a diff. The sentence on how Meta may use content you post was shortened too: the October 5 version tied it to the business services provided under the Commercial Terms and listed them, the new one says "Meta Company Products". Both versions point back to the main Terms, so I would not read much into it, but it changed between the two copies. And neither version mentions robots.txt. On October 17, a Meta spokesperson told Nieman Lab the terms were revised because "it wasn't clear in the initial consolidation of terms that robots.txt preferences are respected." That commitment lives in press statements and in Meta's developer documentation, not in the contract you accept.

The French page had not caught up

Meta publishes the terms in several languages. On the evening of October 19, Paris time, the US English, UK English and Spanish versions of the preview showed the narrowed clause. The UK edition had caught up only that day: an Internet Archive copy taken at 04:38 UTC on October 19 still carried the original sentence. The French and German versions still showed the original. In French: "Meta peut utiliser tout contenu provenant de ces domaines, pages Web ou URL pour fournir, développer, améliorer, rechercher, entraîner et maintenir l'intégrité des Produits, services et fonctionnalités des Entités Meta fournis en vertu des présentes Conditions commerciales et de toute autre condition supplémentaire applicable."

Translations lag, and this one may be fixed by the time you read this. But if your business account runs in French, the text in front of you on October 19 still carried the sentence Meta took out of the English one. On October 11, I wrote about how AI vendors' contracts answer the liability question their executives, under oath, did not answer with a yes. Same habit here: I read the text I am shown, with its date, not the statement about it.

What robots.txt can still do

Meta's crawler documentation names separate user agents, and the split matters here:

  • Meta-ExternalAgent crawls "for use cases such as training foundation AI models or improving products by indexing content directly."
  • Meta-ExternalAds crawls "for use cases such as improving advertising and other business-related products and services." That is almost the purpose the revised clause now states.
  • Meta-WebIndexer works "to improve Meta AI search result quality", and allowing it helps Meta cite and link to your content in Meta AI answers.
  • FacebookExternalHit crawls pages shared on Meta's apps and "might bypass robots.txt when performing security or integrity checks."
  • Meta-ExternalFetcher fetches links at a user's request and "may bypass robots.txt rules."

So the training question and the ads question have different switches. Blocking meta-externalagent is the documented signal against training crawls. Blocking meta-externalads is an ads decision, and its stated purpose is close to the one the revised clause now gives. The documentation says changes can take up to 24 hours, because crawlers cache robots.txt.

What I would check before October 30

As a CTO who builds on Claude and ships iOS apps on the side, this is the list I would run for any product that buys Meta ads or has a Page.

List the domains. The clause does not define "associated with your products or services", so take the broad reading: the website on the Page, every ad destination, the domains in a catalog feed, the domains where a Meta pixel runs, the support site. Each subdomain serves its own robots.txt.

Look behind the login. The clause covers content available through those domains and authorizes test credentials. If your product has a free signup, assume an account you see may not be a prospect, keep it out of your conversion numbers, and check that nothing reachable with a free account is something you would not want an ad system to read.

Decide robots.txt per user agent, on purpose, and write the decision down next to the date.

Keep the text you accept. Save a dated copy of the preview in the language your account uses, and check again on October 29. If the French version still carries "entraîner" then, ask Meta in writing which text applies to you.

Know the exit. The terms say that continued use after notice is acceptance, and that if you do not agree you "may stop all access or use of our Products for business or commercial purposes." There is no middle option in the contract.

The sentence that made the headlines is gone. The permission to send a crawler, with test credentials, to your website is still in the text, and it starts on October 30.

Sources

A project like this one?

I design and deploy products like this. Let's talk.

Let's talk