All I put in my pages is the following;
<meta name="robots" content="ALL">
<meta name="robots" content="index,follow,all">
<meta name="revisit-after" content="1 days">
for this to work you need to create a TXT file in notepad called "robots.txt" with the following text; then put it in the same directory your index.html is located.
# intellihouse robots.txt (intellihouse is my domain, this is where you enter your own)
User-agent: *
Disallow: /cgi-bin/
This has worked for me ever since I created my site, pretty much get all the bots hitting the site.
Bookmarks