Short answer: llms.txt is a proposed plain-text file, written in Markdown and placed at the root of a website, that gives AI assistants a concise summary of the site and links to its most important pages. It is not an official standard and AI companies have not all confirmed that they use it, but it is cheap to add and it encourages a useful exercise: describing your site clearly for machines.
Where llms.txt came from
The format was proposed in September 2024 by Jeremy Howard of Answer.AI. The idea is simple: websites are built for human visitors, with menus, scripts and layouts that make them hard for language models to read in a limited context window. A short Markdown file can point AI tools straight to the content that matters.
What goes in the file
The proposal describes a simple structure:
- A heading with the name of the site or business.
- A short summary in a blockquote: who you are and what you do.
- Optional paragraphs with extra context.
- Sections of links to key pages, each with a one-line description.
A companion file, llms-full.txt, can contain the full text of key pages in one document. Our own site publishes both at /llms.txt.
llms.txt vs robots.txt vs sitemaps
| File | Purpose |
|---|---|
| robots.txt | Tells crawlers, including AI crawlers, what they may and may not access. |
| sitemap.xml | Lists every page you want search engines to discover and index. |
| llms.txt | Summarises the site and highlights the most important pages for AI assistants. |
They work together. A common mistake is listing llms.txt as a sitemap in robots.txt; it is not a sitemap, and should simply live at the root of the site.
What matters more than llms.txt
If you want AI assistants to mention and cite your business, these steps have more impact today:
- Let AI search crawlers in. Check robots.txt does not block crawlers such as OAI-SearchBot and PerplexityBot if you want to appear in their answers.
- Answer questions directly. Put a clear, self-contained answer near the top of each page, then the detail.
- Use structured data. Organization, Service, FAQ and Article schema help machines understand who you are and what each page covers.
- Show evidence. Specific facts, case studies and named experts are easier to cite than vague marketing claims.
- Make content crawlable. Important text should be in the HTML, not hidden behind scripts or pop-ups.
Should you add one?
Yes, if it takes little effort, and with modern SEO plugins it usually does. Just do not expect it alone to change your visibility. Treat it as one part of making your site easy for both people and machines to understand.
Read our UpRender case study on AI search visibility, or see our WordPress, Joomla and Shopify services.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal from September 2024. Some tools and AI products read it, but major AI companies have not all confirmed that they use it for ranking or citations.
Does llms.txt replace robots.txt or my sitemap?
No. robots.txt controls crawler access, the sitemap lists pages for indexing, and llms.txt summarises the site for AI assistants. They serve different purposes and work together.
Will llms.txt help my Google rankings?
There is no evidence that it affects Google rankings. Its purpose is to help AI assistants understand your site. For Google, focus on helpful content, structured data, speed and links from other sites.
