Free Robots.txt Generator

Search Engine Optimization

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

What is robots.txt Generator?

Now, produce ‘robots.txt’ file at your root directory. Copy higher than text and paste into the document.

Robots.txt Generator generates a file that's pretty much opposite of the sitemap that indicates the pages to be enclosed, therefore, robots.txt syntax is of nice significance for any web site. Whenever a look engine crawls any web site, it continuously 1st appearance for the robots.txt file that's set at the domain root level. once known, crawler can browse the file, and so determine the files and directories that will be blocked.

How to use this Tool?

Using our wonderful tool, you'll be able to generate robots.txt file for your web site by following these few simple and easy steps:

By default, all robots are allowed to access your site’s files, you'll be able to opt for the robots you wish to permit or refuse the access.
Choose crawl-delay that tells what proportion delay ought to be there within the crawls, permitting you to settle on between your most well-liked delay period from five to one hundred twenty seconds. it's set to ‘no delay’ by default.
If there already could be a sitemap for your web site, you'll be able to paste it within the text box. On the opposite hand, you'll be able to leave it blank, if you don’t have.
List of search robots is given, you'll be able to choose those you wish to crawl your website and you can refuse the robots you don’t want to crawl your files.
Last step is to limit directories. the trail should contain a trailing slash “/”, because the path is relative to root.
At the end, after you are done generating Googlebot friendly robots.txt file with the assistance of our Robots .txt Generator Tool, you'll be able to currently transfer it to the basis directory of the web site.

If you want to explore our friendly tool before victimisation it then be happy to play with it and generate a automaton.txt example.