Joomla WordPress and Drupal Training
Home / Joomla Tips & Tricks / Add Search to Your Robots.txt File 
Joomla Tips & Tricks
Jan
13
2008
Add Search to Your Robots.txt File
Written by Steve Burge   
Avatar

I've not been able to find the reason yet, but search engines been indexing a lot of search results on Joomla sites. It seems to be an issue with the component itself, rather than any particular URL setup. I've been able to identify the issue with several different setups:

  • Default Joomla URLs : /index.php?option=com_search&searchword=pursuing
  • Default SEF URLs: /component/option,com_search/Itemid,38/index.php?searchword=ctores
  • sh404SEF: /index.php/Search/newest-first.html?searchphrase=any&searchword=trip

I've not been able to find the reason yet, but its worth adding the search component to your robots.txt file. With the examples above, you would use this code:

  • Default Joomla URLs: Disallow: /*com_search*/
  • Default SEF URLs: Disallow: /*com_search*/
  • sh404SEF: Disallow: /index.php/Search/

UPDATE: the answer is in the blog post. It's a Google bug.


 

Add comment


  Joomla WordPress and Drupal Training