Robots.txt Generator

Leave blank if you don't have.
Google
Google Image
Google Mobile
MSN Search
Yahoo
Yahoo MM
Yahoo Blogs
Ask/Teoma
GigaBlast
DMOZ Checker
Nutch
Alexa/Wayback
Baidu
Naver
MSN PicSearch
The path is relative to the root and must contain a trailing slash "/".

Robots.txt is a text file that tells web robots (often called spiders) which pages on your website to crawl and which to ignore. This file, which must be named "robots.txt", is placed in the root directory of your website. It is possible to use the "User-agent" and "Disallow" lines to specify which web robots should access your site and which areas they should avoid.

The Robots.txt Generator is an online tool that helps you create a robots.txt file for your website. You simply need to enter your website's URL and the tool will generate a default robots.txt file for you. You can then edit this file to add or remove rules as needed.