July 5, 2007

Robots.txt and THE MITHOLOGY

One can charge anything to search engines with the help of special meta tags. Namely more wrong nothing!
You can not charge the return of search engine to your site in 2 days, neither also - near the use of metatags - forbid her visiting all of the pages of the service. And this is these supposedly working meta tags:

in meta "revisit-after" content ="1 days"


First line of the above mentioned code orders - rather, would like to be able to order - the robot indexing the site his behaviour. This does not work. Second line orders again to visit the side in 1 day. Obviously also useless.About how much second metatag is entirely useless, then one can replace first with the suitable file. The name of file is 'robots.txt'. He should be placed in the main catalog on the server, that is how the URL of this file looks: http://your-web.com/robots.txt. Search engine reads contained information in this file first, and just begins then to index the side.One can create the file ' robots.txt' in the usual textual editor, e.g. the Notebook. And it looks so:

User-agent: *
Disallow: /


The above mentioned code will block the possibility of indexing the whole site, for all search engines [the star near the User-agent].

If you want to exclude the single brochure from indexing, or the even single file, you should so do this:

User-agent: *
Disallow: /folder
Disallow: /folder2/mysecret.htm


There are no meta tags "strengthening" search engines optimization (obviously keywords, or description have in some browsers the influence on results but they do not order anything to search engine). There is only such, so you can exclude certain fragments of the service from the process of positioning.

No comments: