llms.txt Explained: What It Is and What Google Says About It
On this page
llms.txt is a proposed plain-text file, written in Markdown, that gives AI agents a short summary of a website and links to the pages that matter most. It usually sits at yoursite.com/llms.txt. It's a proposal by one author, not a formal web standard. Google Search says it ignores the file, so it won't change how you appear in Google, including AI Overviews. This explainer covers what the file is and what the companies involved have actually said. It's part of our AI search optimization guide.
Key takeaways
- llms.txt is a proposed Markdown file that guides AI agents to a site's key content.
- It is a proposal by one author, open for community input, not a formal web standard.
- Google Search says it ignores llms.txt: it neither helps nor harms visibility.
- Its clearest use today is software documentation read by coding agents.
What llms.txt is
The proposal comes from Jeremy Howard, first published on September 3, 2024 and updated to version 2 on August 10, 2026. It describes itself as "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website."
The reasoning is practical. Web pages are built for people, wrapped in navigation, scripts and design. An AI agent that only needs the facts has to strip all that away, and long pages eat into the limited context it can read at once. A short, curated file pointing to clean versions of key pages saves the agent that work.
The proposal also suggests offering a Markdown copy of important pages at the same address with `.md` added. The llms.txt file then links to those clean copies.
It is still open for community input. No standards body such as the W3C or IETF has adopted it.
What goes in an llms.txt file
The format is deliberately simple. According to the proposal, a file contains these parts, in this order:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail, without headings.
- Optional sections under H2 headings, each a list of links with a short note.
Here is a minimal example for a service business:
# Example Studio
> Example Studio is a design agency. This file lists our key pages.
## Services
- [Web design](https://example.com/web-design.md): what we build
- [Pricing](https://example.com/pricing.md): how quotes work
## Optional
- [Blog](https://example.com/blog.md): articles and guidesBy convention, the "Optional" section holds links an agent can skip when it needs a shorter read. Version 2 also allows a file at a subpath, such as /docs/llms.txt, covering just that section of a site.
How llms.txt compares with robots.txt and sitemaps
People often confuse llms.txt with two files that already sit at the root of most sites. The proposal itself explains the difference. The table below sums it up.
| File | Main job | Who reads it |
|---|---|---|
| robots.txt | Says which crawlers may access which paths | Search and AI crawlers |
| sitemap.xml | Lists all indexable pages on a site | Search engines |
| llms.txt | Curated summary and links for agents | AI agents, on demand |
The key point: llms.txt doesn't grant or block access. If you want to control which AI crawlers can reach your pages, that's robots.txt and your firewall, covered in robots.txt for AI crawlers. The proposal says llms.txt is read when an agent needs information to help a user, rather than in a background crawl.
What Google says about llms.txt
Google's position is the clearest of any company, and it's on the record in its own documentation. Its guide to generative AI features says you don't need "new machine readable files, AI text files, markup, or Markdown" to appear in Google Search, "as Google Search itself doesn't use them."
The same guide goes further: keeping an llms.txt file "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them." Google adds that it's completely fine to maintain one for other services that use it.
Two other Google signals point the same way. Search Engine Journal reported in May 2026 that Chrome's Lighthouse 13.3 checks for llms.txt in a separate Agentic Browsing category, not in its SEO audit. And in June 2026, SEJ reported Google's John Mueller saying llms.txt can't help systems choose between websites, though it may help an agent that is already on a site.
What other AI companies have said
Here the honest answer is: very little. OpenAI's and Perplexity's crawler documentation explains how to let their search bots in through robots.txt and published IP ranges. Neither mentions llms.txt as something their crawlers read or use to choose sources. Anthropic's help page on its crawlers doesn't mention it either.
What is true is that the AI labs publish llms.txt files for their own developer documentation. The proposal names OpenAI, Anthropic and Google's Gemini. That shows they find the format useful for coding agents reading their docs. It doesn't show their search products use your file to decide whom to cite.
Treat any claim that llms.txt "boosts AI visibility" with caution unless it links to an official source. No such source exists today.
Should your business have an llms.txt file?
It depends on what you want from it. The proposal says the file is used most heavily for software documentation, where coding agents follow it to find API references. If you publish docs, a knowledge base or a large product catalog, it can be a sensible convenience for agents.
For a typical service business, the honest case is modest. A small, accurate file costs little and can't hurt your Google visibility. But it won't replace the work that the engines do document: crawler access, indexable pages, clear answers and accurate facts about your business.
If you publish one, keep it short and keep it current. List only pages you're happy for an agent to quote, and update it when services or pages change. An out-of-date file is worse than none.
For the documented basics, start with crawler access, then structured data for AI search, which Google does use for rich results. Our in-house SEO team covers all of this end to end, including reporting on where clients are mentioned in Google AI Overviews and ChatGPT.
Common questions
Will adding an llms.txt file get my business cited by ChatGPT or Perplexity?
There is no evidence that it will. OpenAI's and Perplexity's crawler documentation explains how to allow their search bots through robots.txt and firewalls, and neither mentions llms.txt. Nobody has published proof that the file changes which sites get cited. If citations are the goal, crawler access and clear, useful pages are the documented route.
Does an llms.txt file hurt my SEO?
No. Google's guide says keeping an llms.txt file will neither harm nor help your visibility or rankings in Google Search, because Google Search ignores it. The only real cost is upkeep. An outdated file that lists old services, prices or pages could mislead any agent that does read it, so update it whenever key pages change.
Where does the llms.txt file go?
The proposal puts it at the root of the site, at /llms.txt, though version 2 also allows a file at a subpath, such as /docs/llms.txt, covering only the pages under that path. Where more than one file applies, agents are meant to use the most specific one. It is a plain Markdown text file, served like any other page.
Is llms.txt the same as blocking AI crawlers?
No. llms.txt doesn't control access at all. It is a guide that points agents to content. Access rules live in robots.txt, where you can allow or block crawlers such as GPTBot or PerplexityBot by name, and in your firewall. A site can publish an llms.txt file and still block some AI crawlers, or the reverse.