Robots.txt Generator

Use the form below to generate the content of robots.txt file, when you place that file on your server you'll be able to control access to specific search engine bots, whether you want to allow them access or deny it and even control their crawl speeds.

Robots access (All robots) Crawl delay
Search engine Access Bot name
Google googlebot
Yahoo yahoo-slurp
Bing (MSN Search) msnbot
Baidu baiduspider
Alexa (Wayback) ia_archiver
DMOZ robozilla
Google Image googlebot-image
Yahoo Blogs yahoo-blogs/v3.9
Yahoo MM yahoo-mmcrawler
Bing Pic-Search psbot
Deny access to a specific directory
You must add a trailing "/" because path is relative to root. Example: /cgi-bin/
Restricted Directory:
Restricted Directory:
Restricted Directory:
Restricted Directory:
Related Links
SEO Analysis
Meta Tags Generator
Browser Addon
Firefox Addon    Chrome Extension