What Is llms.txt? Help AI Engines Discover Your Brand
A complete guide to llms.txt — the emerging standard that helps AI language models understand your brand. Learn the spec, see real examples, and implement it for better GEO performance.
What Is llms.txt?
llms.txt is a proposed web standard created by Jeremy Howard of Answer.AI in 2024. It is a plain-text Markdown file placed in your website's root directory that serves as a navigation guide for AI language models, helping them quickly find your most important content.
Unlike search engines that systematically crawl and index entire websites, LLMs fetch information in real-time when users ask questions. They do not permanently index or "remember" your site structure. llms.txt gives AI a concise map of your highest-value pages.
How It Differs from robots.txt and sitemap.xml
| Aspect | llms.txt | robots.txt | sitemap.xml |
|---|---|---|---|
| Purpose | Guides AI to key content | Controls crawler access | Lists all URLs for indexing |
| Audience | AI language models | Search engine crawlers | Search engine crawlers |
| Format | Markdown | Plain text directives | XML |
| Can block access | No | Yes | No |
Critically, llms.txt contains no directives. It cannot grant or deny access -- it purely guides AI to your most valuable pages.
Format Specification
Per the official spec, the only required element is an H1 heading with your site name. Recommended elements include a blockquote summary, detailed information sections, and link lists organized under H2 headings using the format [Name](URL): Description.
A special ## Optional section marks secondary resources that can be skipped when shorter context is needed.
Example: hogiah's llms.txt
As an early adopter, hogiah maintains its own llms.txt. Here is a simplified version:
# hogiah
> hogiah is built for Taiwan's Traditional Chinese market, tracking brand visibility across 6 standard AI engines: ChatGPT, Perplexity, Google AI Overview, Gemini, Claude, and Grok.
## Product
- [AI Visibility Monitoring](/v2-preview/features/visibility): Track brand mentions across 6 standard AI engines (Starter 1 / Growth 3 / Pro 6)
- [Citation Analysis](/v2-preview/features/citations): Analyze AI citation sources and trends
## Resources
- [What is GEO?](/blog/what-is-geo): Complete guide to Generative Engine Optimization
- [GEO vs SEO vs AEO](/blog/geo-vs-seo-vs-aeo): Comparison of search optimization strategies
Implementation Steps
- Create a file named
llms.txt(case-sensitive) in UTF-8 encoding - Write Markdown content with 15-30 curated high-value pages
- Upload to your root directory at
https://yourdomain.com/llms.txt - Verify: HTTP 200 status,
text/plaincontent type, HTTPS protocol - Review quarterly or after major site updates
Current AI Engine Support (Mid-2026)
OpenAI's GPTBot, Microsoft's crawlers, and Anthropic's ClaudeBot have been confirmed to fetch llms.txt files. However, no major AI company has publicly committed to using it in production systems. Research shows no statistically significant correlation with AI citation frequency yet.
Despite this, implementation takes under 30 minutes and positions your brand for future AI discoverability. Combined with hogiah's visibility monitoring, you can track whether llms.txt impacts your brand's AI mention rate over time.
FAQ
Will llms.txt replace robots.txt? No. They serve entirely different purposes and your site should have both.
Will AI cite me more after adding llms.txt? Not immediately. Think of it as a long-term investment in AI discoverability.
Can I write llms.txt in Chinese? Yes. AI models support multilingual understanding. Write in whichever language serves your target market.
How large should the file be? Under 5,000 tokens (about 15-30 curated links). For extensive documentation, create an additional llms-full.txt file.
Ready to optimize your AI visibility?
Get your free 6-dimension GEO score and see where your brand stands in AI search.
Start Free Analysis