What is a robot meta file?
A robots meta file is simply a file that tell what pages for the google crawler to index or not to index. By indexing we mean what files you would like to show up in the search engine and what files you do not.
Now how does this effect my website?
Well it can effect your website in various ways. For example if you have a page or post that is written in java script or embeded code then it’s a good chance that the crawler can not read the page. If it can not read the page it will then leave and then come back some other time whenever that may be. Now if this keeps happening your pages will not be indexed into the search engine. So all the extra post and articles that you do for people to find you will go to waste unless you fix it.
What can I do to fix the indexing problem?
You can download robots meta and configure it to the proper settings. It will also give you the option to choose whether you want some pages indexed or not. You generally want your contact page that is written in either all java script or iframes to not be read by the crawler. Or any pages that are heavy in java script or iframes.
article from: http://hillsproduction.com/
Related posts: