Robots.txt Checker
Test if robots.txt blocks a URL for Googlebot, Bingbot or AI crawlers like GPTBot and ClaudeBot. See the rule that matches and the sitemaps listed.
Create an llms.txt file for your website in a minute: name, summary, details and sections of links in the right Markdown format. Free, with an example.
One link per line: address | title | short note. Start a new section with ## Section name.
Runs in your browser. Your data never leaves your device.
llms.txt is a short Markdown file at the root of a website that gives AI assistants a summary of the site and links to its most useful pages. Fill in your site name, address and a one-line summary, add a few lines of details, then list your links, one per line, grouped under ## section names. Links can be full addresses or paths such as /pricing/. The generator writes the file in the format proposed at llmstxt.org, ready to save as llms.txt and upload next to your robots.txt.
The file starts with the site name as a heading and the summary as a quote, followed by sections of links. This is the format the generator writes:
# Example Store
> Handmade leather bags and wallets, shipped worldwide.
We sell bags, wallets and belts made to order. Every product page lists sizes, materials and delivery times.
Website: https://www.example.com
## Products
- [Bags](https://www.example.com/bags/): all bags with prices
- [Wallets](https://www.example.com/wallets/): wallets and card holders
## Help
- [Shipping](https://www.example.com/shipping/): countries, costs and times
- [Returns](https://www.example.com/returns/): 30-day returns Be clear about what it does. llms.txt is a proposal from 2024, not a standard, and Google has said it doesn't use it. Some AI tools and coding assistants read it when they're pointed at a site, and it takes a few minutes to make, so it's a cheap extra. Being found in AI search still depends on the basics: crawlable pages, a robots.txt that lets AI search crawlers in, and clear content.
On WordPress, Yoast SEO and Rank Math can create llms.txt for you, or you can upload the file made here to the folder that holds wp-config.php. On Drupal or any PHP site, put it in the web root next to index.php. Shopify doesn't let you upload files to the store root, so stores use an llms.txt app from the Shopify App Store.
After uploading, open yoursite.com/llms.txt: it should load as plain text. Our HTTP header checker confirms it returns status 200 with a text content type. Developers will find the spec and parsers on GitHub and npm by searching for llms-txt; they turn the file into context for a model.
GuideWhat Is llms.txt? What It Does, What It Doesn't, and How to Make One
A Markdown file at /llms.txt that summarises a website for AI assistants and lists its key pages, so a model can find the right content without reading every page.
Yes, with no sign-up. The file is made in your browser.
To the root of your site, the same folder as robots.txt, so it opens at https://yoursite.com/llms.txt.
No. robots.txt tells crawlers what they may fetch. llms.txt is a guide to your best content and blocks nothing.
No. Google doesn't use it for Search. Treat it as a small extra for AI tools, not an SEO fix.
Test if robots.txt blocks a URL for Googlebot, Bingbot or AI crawlers like GPTBot and ClaudeBot. See the rule that matches and the sitemaps listed.
Create a robots.txt file: block folders, add your sitemap, and allow or block AI crawlers such as GPTBot, ClaudeBot and PerplexityBot.
Generate HTML meta tags for SEO and sharing: title, description, robots, canonical, Open Graph and Twitter cards, with a Google preview. Free.
Preview how a link looks when shared on Facebook, WhatsApp, LinkedIn and X, and check every Open Graph and Twitter Card tag.