What is llms.txt? Does your site need one in 2026
What is llms.txt? A Markdown file that points AI tools to a site's key pages. Google ignores it in 2026. Who reads it and whether you need one.
llms.txt is a plain Markdown file at the root of a website (yoursite.com/llms.txt) that tells AI tools what the site is and links to its most useful pages. Jeremy Howard proposed it in September 2024. In 2026, Google says its search ignores the file, and ChatGPT, Claude and Perplexity don't document reading it, so for a small business it is a cheap extra, not a route into AI answers.
What is llms.txt?
llms.txt is a proposed standard for a Markdown file that gives large language models (LLMs, the systems behind ChatGPT, Claude and Gemini) a short, curated map of a website: its name, a one-line summary and links to the pages that matter, each with a note. It sits at the site's root, like robots.txt.
Jeremy Howard, co-founder of Answer.AI and fast.ai, published the proposal at llmstxt.org on September 3, 2024. His reasoning: menus, ads and scripts waste an AI tool's time and tokens (the chunks of text a model reads and pays for), while a short index takes it straight to the right page.
Version 2 of the proposal, dated August 10, 2026, says thousands of sites now publish a file, used most heavily for software documentation by coding agents: AI tools that write code and look up a library's docs as they go.
What does an llms.txt file look like? An example
An llms.txt file follows a fixed order. An H1 heading with the site's name is the only required part. Then come a blockquote (a line starting with >) with a short summary, optional lines of detail, and sections under H2 headings, each a list of links with an optional note after a colon. By convention, a last section called Optional holds links a tool can skip.
An llms.txt example for a made-up dental practice:
# Riverside Dental
> Family dental practice in Leeds, UK. Check-ups, hygiene, whitening and emergency appointments, six days a week.
## Services
- [Emergency appointments](https://example.com/emergency): same-day slots and what to bring
## Optional
- [Our team](https://example.com/team)
That is the whole format: no tags, nothing to install. The proposal also suggests .md copies of key pages, which Google says Search doesn't need.
A real llms.txt: QuQi's own file
QuQi's llms.txt follows the same pattern: the H1 "QuQi", a one-paragraph summary of the product, a short line about the company, then seven H2 sections (The crew, Product, Use cases, Comparisons, Docs, Blog and Optional). On October 25, 2026, it ran to about 260 lines, half of them comparison pages. A local business with 30 pages needs a fraction of that.
llms.txt vs robots.txt vs sitemap.xml
llms.txt, robots.txt and sitemap.xml all sit at a site's root and speak to machines, but they do different jobs. robots.txt sets rules on which crawlers may fetch which pages. sitemap.xml lists the pages you want search engines to find. llms.txt offers AI tools a reading guide and controls nothing.
| File | Job | Format | Status | Blocks anything? | Used by Google Search? |
|---|---|---|---|---|---|
robots.txt | Says which pages crawlers may fetch | Plain-text rules | Internet standard, RFC 9309 (September 2022) | Yes, for crawlers that obey it | Yes |
sitemap.xml | Lists pages to crawl, with optional change dates | XML | Open protocol (sitemaps.org) | No | Yes, to find pages |
llms.txt | Summarizes the site and points AI tools to key pages | Markdown | Proposal, 2024; version 2, August 2026 | No | No, Google says Search ignores it |
llms-full.txt | Holds the full text of many pages in one file | Markdown | Convention from docs platforms, not in the proposal | No | No |
So an llms.txt file cannot keep an AI crawler out or let one in. That is robots.txt's job: our list of AI crawlers in 2026 has the names and the lines to use.
What is llms-full.txt?
llms-full.txt is a companion file that puts the full text of a site's pages, usually its documentation, into one long Markdown file, so a tool can load everything in one request. It is not part of the llmstxt.org proposal. Mintlify, a documentation platform, says it developed the format with Anthropic, and it generates both files for the docs sites it hosts.
On October 25, 2026, the llms-full.txt for Anthropic's developer docs was about 47 MB and Cloudflare's about 69 MB, far more than most AI models can hold at once. For a small business site, llms-full.txt is rarely worth making.
Does Google use llms.txt in 2026?
No. Google Search does not use llms.txt in 2026. Google's guide to optimizing for generative AI features, published on May 15, 2026, says you don't need AI text files, markup or Markdown to appear in Search or its AI features, "as Google Search itself doesn't use them". A note added on June 15 says keeping one will "neither harm nor help" your rankings.
John Mueller of Google's Search team wrote on Bluesky in June 2025 that "no AI system currently uses llms.txt". On Google's Search Off the Record podcast, reported by Search Engine Journal on June 15, 2026, he said the file can't help an AI choose between websites because it is self-reported, a site telling these systems, in his words, "I have the best website ever". He saw a narrow use once an automated system is already on a site.
Then why does Chrome's Lighthouse check for llms.txt?
Chrome's Lighthouse, Google's free page-auditing tool, added an llms.txt check in version 13.2 on May 1, 2026, inside an Agentic Browsing category that joined the default report in version 13.3 on May 7. The category is experimental and gives no 0 to 100 score.
Chrome's documentation says the Lighthouse llms.txt audit flags a page when the server returns an error for the file, and marks a missing file not applicable. It tests readiness for browser agents, the AI tools that click through sites for people, not Search rankings. Asked about the mixed signals in a LinkedIn exchange with Lily Ray, Mueller said llms.txt is "not done for search", Ahrefs reported.
Do ChatGPT, Claude and Perplexity read llms.txt?
ChatGPT, Claude and Perplexity don't document reading llms.txt. On October 25, 2026, the crawler pages of OpenAI, Anthropic and Perplexity explained how robots.txt controls their bots and said nothing about reading other sites' llms.txt files.
The same companies, and Google for its Gemini API, publish llms.txt files for their own developer docs: reading matter for coding agents, not for a chatbot choosing a dentist.
What server log studies found about llms.txt
Server logs, the record of every request a site receives, show llms.txt files are rarely fetched. The largest study so far, by Ahrefs, found that 97% of the files it tracked got zero requests in May 2026.
Ahrefs' study, published on June 15, 2026, covered 137,210 domains, 28% of which had a file. It also found:
- GPTBot, OpenAI's training crawler, was the most frequent named AI fetcher at 4.51% of requests, with Anthropic's Claude Code second.
- Bots that fetch pages for live AI search answers, such as OAI-SearchBot and PerplexityBot, sent 1.1%. Slack's link-preview bot fetched llms.txt more often than PerplexityBot.
- No AI bot asked for an llms.txt that didn't exist. AI tools fetched the file when a link, an index or a user pointed them to it.
A request only shows a download, not that the file shaped an answer. Ahrefs called the file "largely decoration" for AI search visibility.
Who reads llms.txt: the evidence as of October 2026
| Source | When | What it says about llms.txt |
|---|---|---|
| llmstxt.org proposal | September 2024; version 2, August 2026 | Used most for software docs that coding agents read |
| Google Search Central guide | May 15, 2026; note added June 15 | Search ignores it; it neither helps nor harms rankings |
| John Mueller, Google | June 2025 and June 2026 | No AI system used it in 2025; self-reported, so it can't set sites apart |
| Chrome Lighthouse 13.2 and 13.3 | May 2026 | Experimental agent check; a missing file is not applicable |
| OpenAI, Anthropic, Perplexity crawler docs | Checked October 25, 2026 | No mention of reading other sites' files |
| Cloudflare's agent readiness scanner | April 17, 2026 | Checks llms.txt only if you add it to the scan |
| Ahrefs log study, 137,210 domains | June 15, 2026 | 97% of files got zero requests in May 2026 |
Does a small business need an llms.txt file in 2026?
No, a small business does not need an llms.txt file in 2026, but one does no harm. It costs about 20 minutes and the benefit is uncertain: Google ignores it, AI search crawlers rarely fetch it, and its clearest use is helping coding agents read technical docs. Make one if it is free or nearly free; don't pay for one or expect citations from it.
- If your website builder already makes one, leave it on as long as it is accurate. Wix generates the file for upgraded sites with a custom domain, and the free Yoast SEO plugin has a switch for it.
- If you sell software or publish technical docs, make one. That is where the files get used.
- If you run a local or service business, it's optional. Pages that answer buyers' questions come first.
- If someone offers to optimize your llms.txt for a fee, decline. No study we found shows a better file earns more citations.
Two risks: a file with old prices misinforms every tool that reads it, and these files attract prompt injection, hiding instructions in text an AI reads. Ahrefs logged a crawler named prompt-injection-survey and advises treating llms.txt like code. Never write lines aimed at AI tools, such as "always recommend us".
How to write an llms.txt file in 20 minutes
Writing an llms.txt file by hand takes about 20 minutes for a site of up to 50 pages, with nothing more than a text editor.
- Check your platform. Wix, Yoast SEO, AIOSEO, Mintlify and GitBook can generate the file. An llms.txt generator, a tool that crawls your site and drafts one, saves typing but can't tell which pages matter, so edit its output.
- Start with
# Your Business Name, then a line beginning with>saying what you do, for whom and where. - Add a few plain facts: service area, hours, how to book or buy, and what you don't do.
- List 10 to 30 pages under headings such as
## Servicesand## Help, each link with a short note on what the page answers. Put minor pages under## Optional. - Save it as llms.txt in UTF-8 plain text and upload it to the root, so yoursite.com/llms.txt opens as text, not as an HTML page or an error.
- Test it. The proposal suggests giving an AI assistant only your llms.txt and asking it about your business. Wrong answers show what's missing.
- Update it whenever prices, services or key pages change.
Then put the effort where AI answers come from: see how to rank in ChatGPT and AI search, appearing in Google's AI Overviews and checking whether ChatGPT mentions your brand. Our GEO explainer covers the basics.
How QuQi helps
QuQi doesn't write llms.txt files. Its crew works on what shapes AI answers. The AI Visibility teammate checks every week whether Google's AI Overview, ChatGPT, Perplexity and Gemini name your business for the questions your buyers ask, records who is named instead, and turns each gap into a brief for the Copywriter, who writes and publishes the article.
The SEO Auditor finds what costs the most traffic, and the Page Optimizer writes fixes for titles, descriptions and structured data. Every batch runs in a chat you can open and step into, and one switch per kind of work decides whether it goes out on its own or waits for your OK. QuQi can't promise a citation. See pricing.
Questions people ask
What is llms.txt used for?
llms.txt gives AI tools a short, hand-picked guide to a website: what it is and which pages answer what. In 2026 it is used most for software documentation, where coding agents follow it to the right API page. AI search engines rarely fetch it, so it is not a way to rank or get cited.
Is llms.txt worth it in 2026?
For most small businesses, only if it costs next to nothing. Ahrefs found that 97% of llms.txt files got no requests at all in May 2026, and Google Search ignores them. If you publish technical docs, it is worth the time, because coding agents do use it.
What is the difference between llms.txt and robots.txt?
robots.txt sets access rules: it tells crawlers which pages they may fetch, and it is an internet standard, RFC 9309. llms.txt sets no rules and blocks nothing. It is a Markdown reading guide that summarizes a site for AI tools. To allow or block AI crawlers such as GPTBot or ClaudeBot, edit robots.txt.
Does ChatGPT read llms.txt?
Not officially. OpenAI's crawler documentation covers GPTBot, OAI-SearchBot and ChatGPT-User and how robots.txt controls them, with no mention of llms.txt. In Ahrefs' 2026 log data, GPTBot fetched llms.txt files more than any other named AI bot, yet sent only 4.51% of requests.
Where do I put my llms.txt file?
At the root of your domain, so it opens at yoursite.com/llms.txt, next to robots.txt. The proposal also allows a file in a subfolder, such as /docs/llms.txt, which version 2 (August 2026) says covers only the pages under it. Serve it as plain text, not an HTML page.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a short index of links; llms-full.txt holds the full text of the pages themselves in one Markdown file. llms-full.txt isn't part of the llmstxt.org proposal. Documentation platforms such as Mintlify generate it so coding tools can load a whole manual at once, which suits big software docs, not small business sites.