Google’s John Mueller says that scripted pages serve a particular purpose for developer’s directory sites but won’t help most websites, as search becomes more sophisticated.
Mueller laid out his reasoning for the Bluesky thread. He was responding to a question from Lily Ray about why Google publishes LLMs.txt files and markup pages, even though they are not needed for search functionality.
His answer focuses more on developer documentation markup versions, not llms.txt as a standalone file.
Mueller wrote:
“The short answer is that it’s not for search. There’s more to websites than just SEO :-).”
Mueller’s Discovery vs. Working Framework
His thinking focused on two different goals for the website.
He called the first “discovery,” or discovery by a search engine, and the second “operation,” which helps users complete tasks on a page.
Mueller acknowledged that the term was inaccurate. “There may be a more accurate term for this,” he wrote in the song.
He compared the difference to calls to action on traditional pages, saying:
“You don’t do it” to get SEO (to be found), but if you are responsible for the website as a whole, ensuring a high ‘discovery rate’ (SEO) and a high conversion rate helps to justify your work.”
Why Developer Documentation Is Different
On developers.google.com, he noted, the markdown versions make sense.
Mueller said;
“AI coding has become very popular, and these coding systems can be efficient and accurate in the code they produce if they can easily read/classify reference materials, such as developer documentation.”
He added that bookmarks can help AI programs “understand the context of the documents they’re viewing, as well as a simplified version of the reference page.”
Mueller called this a long-term necessity, adding:
“OF COURSE they can read HTML well, so this is just a temporary solution, maybe to save some tokens.”
Non-Developer Sites Should Be Skipped
To everyone, Mueller was specific, saying:
“For non-engineering sites, I don’t think this makes much sense, even with a lot of agencies in the future. Making a markdown version of the shoe specs won’t get you more sales (competitors appreciate that).
He went further in the next post, going back to the idea that sites should prepare for a future where agents drive more traffic.
Mueller added:
“And (I know, no one reads this far), if you think this is important to prepare when agents are everywhere: your site (all sites) have more important things to do in SEO than preparing for a future situation that may or may not come. Put needs before dreams.”
Why This Matters
Mueller’s comments reflect a more detailed position than his previous statements on the topic.
In February, Mueller called the idea of handing over pages of text to bots a “stupid idea.” His Bluesky comments highlight the differences in developer documentation while holding the line for all other site types.
The thread also came on the same day we reported that Google’s guidance on llms.txt now depends on which product you’re asking about. AI Productivity’s guide to developing AI says to skip llms.txt, while Lighthouse 13.3 adds a test that checks the file as part of the agent’s browser readiness.
Looking Forward
Mueller’s distinction between discovery and page performance can help you evaluate whether agent development is worth your time. To test whether building agents are currently producing measurable results for a particular site.
The line “put needs before dreams” captures the tension in the industry right now. Marketers have been promoting llms.txt and markup optimization as emerging practices, but Google search documents or independent data to support investing in these sites are not developers.
Featured Image: kirill_makarov/Shutterstock
