Shopify robots.txt generator
Build a robots.txt that keeps Shopify's rules and adds yours.
Three steps, no account.
- 1
Choose your rules
Block filtered pages, AI crawlers or any path you list, and add extra sitemaps.
- 2
Copy the template
Get a robots.txt.liquid that keeps Shopify's defaults and adds only your rules.
- 3
Add it to your theme
Create the robots.txt template in your theme's code editor, paste, and save.
What it checks
Choose what to keep crawlers out of, from filtered collection pages to AI training bots, and get a robots.txt.liquid ready to paste into your theme. It keeps Shopify's default rules, which Shopify updates for you, and adds yours on top. A plain robots.txt version is included for sites on other platforms.
- Keeps Shopify's default rules, updated by Shopify
- Blocks filtered and vendor collection URLs
- Blocks AI training or AI search crawlers, separately
- Loads a store's current robots.txt to compare
What each number is telling you.
A report you cannot interpret is just a longer way of saying something is wrong.
The Liquid template
Shopify builds robots.txt from a template. Keeping its default groups means you keep every rule Shopify adds later; your rules sit on top.
Rules for every crawler
Disallow lines added to the group for all crawlers (User-agent: *). Paths can use * as a wildcard.
AI crawler groups
Each blocked AI crawler gets its own group with Disallow: /. Training and search crawlers are separate, because blocking search ones also hides you from AI answers.
Your current file
Load any store's live robots.txt to see whether it blocks everything, lists a sitemap, or already blocks AI crawlers.
The mechanism, not the platitude.
- 01
Filters multiply your pages
Every combination of colour, size and price is a new URL for Google to crawl. On a big catalog that crowds out the pages you want found.
- 02
AI crawlers are a choice now
Training crawlers take your product copy for models; search crawlers bring shoppers from AI answers. robots.txt lets you choose each.
- 03
Editing Shopify's file is easy to get wrong
Replacing the template with plain text drops Shopify's rules and their future updates. Adding to it is the safe way.
What this usually turns up, and the fix.
Four problems that account for most of what this tool reports on a Shopify store.
The finding
The whole store blocked
Why it happens
A leftover Disallow: / from before launch, or a password-protected store.
The fix
Remove the rule, and turn off the store password in Online Store, Preferences.
The finding
Shopify's defaults missing
Why it happens
The template was replaced with plain text instead of added to.
The fix
Start from this generator's template so the default groups come back.
The finding
Blocked pages still in Google
Why it happens
robots.txt stops crawling, not indexing; pages already known can stay listed.
The fix
Use a noindex tag for pages that must leave search, and keep them crawlable until they drop out.
What this will not tell you.
Every tool has a boundary. Knowing where this one ends is what keeps you from over-reading it.
- robots.txt is a request. Well-behaved crawlers follow it; others don't, and it is not a way to hide private pages.
- It controls crawling, not indexing. To remove a page from search results, use noindex.
More free tools.
All toolsRun this continuously instead.
Install ShopSEO free, run your first audit in under a minute, and roll back anything you do not like. No credit card, no theme edits, no lock-in.