|
Robots (spiders) are used by operating systems like Google to automatically
explore and index web sites on the World Wide Web. Robots usually
follow web site links (hyperlinks) from one web site page to another.
Met tags can be placed in <head> section of a HTML file to
allow robots to be banned from indexing web pages or allowed to
index all.
Robot Meta Tag Examples
There are four directives that can be placed in a robots meta tag.
The content section of the meta tag can contain:
- index,
- noindex,
- follow,
- nofollow
The index directive tells the robot it is ok to index the page.
The follow directive tells the robot it is ok to follow the links
found on this page.
Robots Meta Tag Examples:
<META NAME="ROBOTS" CONTENT="INDEX,FOLLOW">
<META NAME="ROBOTS" CONTENT="NOINDEX,FOLLOW">
<META NAME="ROBOTS" CONTENT="INDEX,NOFOLLOW">
<META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">
|
