Skip to content
AI Tools

Robots.txt & Sitemap Generator

Generates robots.txt and sitemap configurations

Format: robots.txt and XML sitemap (sample structure). Context: You configure crawlability for optimal indexing. Task: Generate robots.txt and sitemap structure for {{SITE_DOMAIN}}, {{NUM_PAGES}} pages. Constraints: Block unnecessary dirs, allow important pages, XML sitemap with priorities, standard format. Do NOT: Block important content, create overly restrictive rules, miss priority pages.
0 copies

Variables

Replace these variables with your own values before using:

{SITE_DOMAIN}{NUM_PAGES}{EXCLUDED_DIRECTORIES}{PRIORITY_PAGES}

Example output

robots.txt: User-agent: * Disallow: /admin/ Disallow: /private/ Allow: /public/ Sitemap: {{SITEMAP_URL}} XML Sitemap: 350 URLs, priorities set