Tagged: robots txt file

robots txt file for wordpress site, what is robots.txt file


What is robots.txt?

robots.txt file for WordPress is a text file which contains instructions for the Web Crawlers (especially search engine robots) about how to crawl the pages on a particular site. You can see this site robots.txt here....