Robotstxt vs relnofollow vs meta robots nofollow Guide
Today, I am going to share with you the most important files and tags in SEO i.e. Robots.txt and robot tags.
Suppose, you designed a website and uploaded on the server and you do not let your pages crawled in Search Engines. What will you do? You have many options available with you like you can give instructions to search engine that do not crawl the particular pages of your website or do not crawl this contents or do not follow the particular links to pass page rank juice. This presentation will help you out.
What is the role of robots.txt file in SEO?
Makes sure that you will use wildcards in robots.txt very carefully. A Single line of code can harm your website ranking and traffic. One case study I would like to share with you. Before some time, when I was new to this, have used wildcards incorrectly in robots.txt. Our web pages have not been crawled by Search Engines due to that wildcard(*). Then I got understand about it. So, you should also take care about this. Have a look this presentation based on Robots.txt and some robots tags.
* Robotstxt vs relnofollow vs meta robots nofollow Guide