Use noindex Meta Tag in WordPress to Prevent Search Engine Indexing Categories, Tags and Archives

WordPress has the content duplication problem which may confuse the search engine. Our sites may get search engine penalty because of content duplication. Actually we only want the search engine to index the posts and pages instead of categories, tags or archives.

A noindex meta tag will prevent search engine such as google from indexing the page. Google’s rule about noindex tag is like this [1]:

“When we see a noindex meta tag on a page, Google will completely drop the page from our search results, even if other pages link to it. If the content is currently in our index, we will remove it after the next time we crawl it. The meta tag allows you to control access on a page-by-page basis, which is useful if you don’t have root access to your server. (You’ll need to be able to edit the source HTML of your page.)”

We can add noindex tag to the head of archives, categories, tags and search results in WordPress. And add a index to the posts, pages, etc.

The method is simple. First find the header.php file in the theme directory. Then insert these codes into the area between <head> and </head>:

<?php
// Do not index author, date archive, category, tag and search result
if( is_author() | is_date() | is_search() | is_category() | is_tag() ) {
  _e('<meta name="robots" content="noindex,follow" />');
} else {
  _e('<meta name="robots" content="index,follow" />');
} ?>

Referrence:

[1] http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156412

Update history
Aug. 9, 2010. Change head.php to header.php which should be right name.
Updated on 16 Mar. 2010.