ToolBoxBase
All tools
About
🧰 Tool
Robots.txt Generator
Generate a robots.txt file for your site (allow or block crawlers).
Mode
Allow all (recommended)
Block all
Custom
Sitemap URL (optional)
Disallow paths (one per line)
Add basic AI crawler blocks
Add Host: (optional)
Host (only if checked above)
Generate
Copy
Clear
robots.txt output
Ad placeholder
Optional inline ad spot.
Tips
Tip: “Allow all” is usually best unless you have private areas you want to block (like /admin).
Not all bots respect robots.txt.
Ad placeholder
Put your ad code here (sidebar).
← All tools