What the Proposal Says
The idea itself is concise: put an llms.txt in the site root that explains, in a model-friendly format, what the site is about, where the important content is, and how you'd like the AI to cite you. Unlike robots.txt, which governs "whether it can crawl," this governs "how to understand it once crawled." Supporters see it as going with reality—the model is reading your site anyway, so you might as well proactively hand it a manual and save it from guessing out of messy HTML.
Controversy and the Current State
There's no shortage of opposing voices either. The most practical: there's no evidence that mainstream crawlers and models actually prioritize reading this file, and right now it looks more like a wishful gentleman's agreement. Others worry it will become a new SEO-cheating ground—showing one story to the AI and another to humans. For now it sits in the "cheap enough to be worth a try" stage—adding a file is a few minutes' work, and if it happens to work, it's a free win. But treating it as an actual means of controlling AI crawling is overthinking it—control still has to rely on firewalls and the law.
via: Hacker News