llms.txt is a community proposal for a Markdown file that gives AI agents a concise overview of a website and links to useful resources. It is not a crawler access rule, search ranking factor or guarantee that an AI product will read or cite a page.
Google’s current guidance says site owners do not need new machine-readable files, AI text files or special markup to appear in AI Overviews or AI Mode. The same foundational SEO practices apply. Other tools may choose to read llms.txt, so a site owner can publish it as a small, maintained index when there is a clear use for it.
What the proposal contains
The current proposal allows a site or a section of a site to publish an llms.txt file. It uses Markdown. A file can contain:
- An H1 with the site or project name. This is the only required section.
- An optional blockquote summarising the site.
- Optional explanatory paragraphs or lists, without headings.
- Optional H2 sections with lists of links to relevant pages or Markdown resources.
The proposal suggests keeping the overview concise and linking to detailed source pages. A section titled Optional is a convention for secondary material, not an access-control rule.
How it differs from robots.txt and a sitemap
These files have different purposes:
- robots.txt gives crawling instructions to compatible automated crawlers. It is not a reliable way to keep private information secure or guarantee that a URL will not be indexed.
- sitemap.xml lists URLs that a site owner wants search engines to discover. It does not control access or ranking.
- llms.txt is a proposed curated guide for agents that choose to read it. It does not grant or deny access and does not change what search engines can crawl.
Do not use llms.txt instead of authentication, server permissions, robots directives or noindex controls. Do not put confidential information or claims that are absent from the public site in the file.
When it may be useful
A concise index can help a compatible agent find official documentation, product details, policies or other canonical resources. It may be useful for a documentation-heavy site where the owner can keep the file and linked resources current.
That is a possible discovery aid, not an established route to search visibility, AI citations or fewer inaccurate answers. Google explicitly says its Search AI features do not require the file. Check the requirements of the particular tool before assuming it uses the format.
Create and maintain the file
- Choose a clear scope, such as the whole site or a documentation section.
- Write a short, accurate summary and select canonical public pages.
- Link to current pages that answer likely questions. Use stable URLs and descriptive labels.
- Check every link, confirm that the pages are publicly accessible and ensure the file does not imply access controls.
- Assign an owner and review the file when key pages, products or policies change.
For example, a small file could look like this:
# Example Organisation
> A short, factual description of the organisation and its public website.
## Main resources
- [Services](https://example.com/services/): Current service scope
- [Documentation](https://example.com/docs/): Technical referenceReplace the example details with information that is accurate and visible on the linked pages. Do not present the example as a required format for Google Search.
Publish without disrupting the site
Publish the file as plain text or Markdown at the path your chosen tools expect, commonly /llms.txt. The proposal also allows files at a subpath to describe resources under that part of a site. Test the public response, content type, access permissions and links after publishing. If using a plugin or build process, include the file in deployment and assign someone to maintain it.
The proposal also discusses Markdown versions of linked pages, such as an alternate .md URL. This is optional. Check for duplicate maintenance work and ensure the HTML page remains complete, crawlable and authoritative. Do not make essential facts available only in an AI-oriented file.
How to evaluate the result
Check that the file is available, accurate and useful to a tool that supports it. You can measure referrals or requests in server analytics when they can be identified reliably, but absence of a visible visit does not prove whether a model used the file. Do not claim an SEO or AI visibility gain without evidence.
Frequently asked questions
Does llms.txt improve Google rankings?
Google says AI Search does not require new AI text files or special markup. There is no basis to promise that adding llms.txt will improve Google rankings.
Does it control AI crawlers?
No. It is a proposed information file, not an access-control mechanism. Use authentication, server controls and the relevant crawler directives for access decisions.
Will ChatGPT, Gemini or another AI system read it?
Do not assume so. Support varies by product and can change. Verify the current documentation for each tool you care about.
Should every business publish one?
No. Publish it only if a clear user or agent workflow benefits from a curated index and someone can maintain it. Well-structured, accessible source pages remain the priority.
Conclusion
Treat llms.txt as an optional, evolving convention. It can organise links for agents that choose to use it, but it cannot replace good pages, search controls or evidence-led technical SEO, and it does not guarantee discovery or citation.
Sources checked on 20 October 2026: the llms.txt proposal, version 2 and Google’s AI features and your website.