Apple’s support documentation for Applebot now carries a sentence it didn’t have in late August: “Site rules for Applebot-Extended are not considered in ranking for Search.” The page, titled “About Applebot,” is dated September 4; Wayback Machine snapshots from August 9, 18 and 21 carry no such line. The addition closes out two questions that have attached to Apple’s AI-training opt-out since it shipped: whether blocking it costs a site anything in Apple Search ranking, and whether it costs a site its place in Apple Search results at all. Apple answered the second question when Applebot-Extended launched. Now it has answered the first, in writing, the same way Google’s crawler documentation already answers it for Google-Extended.
Inclusion Was the Old Promise, Ranking Is the New One
Apple added Applebot-Extended to its documentation on June 11, 2024, the day after its WWDC keynote introduced Apple Intelligence. That original text already said: “Applebot-Extended does not crawl webpages. Webpages that disallow Applebot-Extended can still be included in search results.” That was an inclusion promise only: disallow Applebot-Extended, and a page still shows up in Apple Search, with no word on where it would land. Apple’s support page closes that gap now. Between the August 21 snapshot and the live version, dated September 4, the “About search rankings” section gained one line: “Site rules for Applebot-Extended are not considered in ranking for Search.” Search Engine Roundtable flagged the addition. The rest of the section, including the ranking factors listed above the new sentence, is unchanged.
Does blocking Applebot-Extended affect Apple Search ranking?
No, according to Apple’s own documentation. As of September 4, the Applebot support page states that “Site rules for Applebot-Extended are not considered in ranking for Search.” Disallowing Applebot-Extended in robots.txt opts a site out of having its crawled content used to train Apple’s generative AI models; it does not change how that content is scored once it appears in results. The same page confirms pages blocking Applebot-Extended “can still be included in search results,” so the training opt-out costs a site neither inclusion nor ranking.
Apple’s Five Ranking Factors, Verbatim
The same section lists what actually does move a result. Apple states: “Apple Search may take the following factors into account when ranking web search results”: “Aggregated user engagement with search results,” “Relevancy and matching of search terms to webpage topics and content,” “Number and quality of links from other pages on the web,” “User location based signals (approximate data),” and “Webpage design characteristics.” The paragraph below the list adds a caveat: “Search results may use the factors above with no (pre-determined) importance of ranking.” Applebot-Extended sits outside that list entirely, now stated outright.
Four Crawlers, Two Kinds of Cost
Apple isn’t the first to draw this line; not every AI crawler behaves the same way once it’s drawn. Google’s crawler documentation carries a near-identical disclaimer for Google-Extended: it “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search.” OpenAI’s crawler documentation draws no such line for GPTBot, which its doc describes as a training crawler. The crawler that does move ChatGPT search standing is a separate one, OAI-SearchBot, and blocking it carries a real cost by OpenAI’s own account.
| Crawler | What it governs | What blocking costs in search (per the vendor’s own doc) | Source doc |
|---|---|---|---|
| Google-Extended | Whether content Google crawls may train future Gemini models | None: “does not impact a site’s inclusion in Google Search nor is it used as a ranking signal in Google Search” | Google Search Central, common crawlers doc |
| Applebot-Extended | Whether crawled content trains Apple’s generative AI models | None: pages “can still be included in search results,” and as of Sept. 4, “Site rules for Applebot-Extended are not considered in ranking for Search” | Apple Support, About Applebot |
| GPTBot | Whether content trains OpenAI’s generative AI foundation models | Training opt-out; OpenAI’s crawler doc does not tie it to ChatGPT search results | OpenAI, Overview of OpenAI Crawlers |
| OAI-SearchBot | Whether a site surfaces in ChatGPT’s search feature | Removal from ChatGPT search answers: “will not be shown in ChatGPT search answers, though can still appear as navigational links” | OpenAI, Overview of OpenAI Crawlers |
What the New Sentence Doesn’t Cover
The new line is scoped narrowly, to ranking within Search alone. Apple’s own page says the data crawled by Applebot powers features including Spotlight, Siri and Safari, but the ranking sentence names only Search. On being found at all, the document does give a first-party answer. In the overview at the top of the page it notes that “Even if you disallow Applebot-Extended and tag website content with the nosnippet meta tag, your website instructions may still allow Applebot to crawl your webpages,” and adds: “Your content will remain discoverable through Spotlight, Siri, and Safari, as well as other system-wide features on Apple devices.” What the document does not address is ordering: whether the Applebot-Extended rule also stays out of how results are ranked inside Siri or Apple Intelligence answers, Spotlight suggestions, or Safari’s suggestions. A publisher weighing the block for those surfaces has Apple’s word on discoverability, and no statement yet on position.
Auditing Your Own Block
Anyone using elsop’s own robots.txt generator and its “Block AI bots” preset should check what got bundled in. The preset writes a disallow rule for Applebot-Extended, now confirmed free of ranking cost, but it also writes one for OAI-SearchBot, and that one carries a visibility cost. Per OpenAI’s own documentation, sites disallowing OAI-SearchBot “will not be shown in ChatGPT search answers, though can still appear as navigational links.” Anyone who wants ChatGPT search visibility while still opting out of AI training should remove that one block. The pattern is the same one covered when Google’s Search AI controls went worldwide: search operators are writing opt-out mechanics into their own documentation, crawler by crawler, rather than leaving publishers to guess.
