Robots.txt Editor

Control which pages search engines are allowed to crawl.

Back to dashboard

Rules editor

*
Googlebot

Live preview

User-agent: *
Disallow: /admin
Disallow: /thank-you
Disallow: /cart

User-agent: Googlebot
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

Validation

No validation issues found.

AI crawler presets

GPTBotClaudeBotPerplexityBotGoogle-Extended