Jan
13
2008
| Add Search to Your Robots.txt File |
|
| Sunday, 13 January 2008 | |
|
I've not been able to find the reason yet, but search engines been indexing a lot of search results on Joomla sites. It seems to be an issue with the component itself, rather than any particular URL setup. I've been able to identify the issue with several different setups:
I've not been able to find the reason yet, but its worth adding the search component to your robots.txt file. With the examples above, you would use this code:
Comments (10)
  written by Yannick Gaultier, January 15, 2008
Hi Steve,
I can't find these (sample?) links in Google, whether directly or using allinurl. Did you find them in Y! or live ? ANyway, could the reason they show is that there are such links found in forum discussions ? I'm sure there are several such links in my forum, where users put them to show a specific problem... Regards  written by Steve Burge, January 15, 2008
Hi Yannick
Sure thing - try this as an example: http://www.google.com/search?h...php/Search Its a new site so no-one has been linking to search results and I've also seen it too often. written by Joomla Blog Insider, January 18, 2008
Hi there and tnx for your hint concerning these indexed search pages. I still don't have any prove that these pages could be bad for google ranking. I mean: have a look at these tag modules
 They produce massive amounts of search result pages which then usually become indexed. If these indexed search pages would have negative results on google there wouldn't be any need of these tag modules - I'd love to switch them off  What do you think? Do you have any prove? Tnx in advance! written by Steve Burge, January 18, 2008
Hi
This post is probably a good start: http://www.alledia.com/blog/search-engine-optimisation-(seo)/joomla-seo--why-less-is-often-more/ The key idea is that your site only has a certain amount of Page Rank and link juice. The more you give to useless extra pages, the less you have for your important pages. written by Joomla Blog Insider, January 20, 2008
Hi Steve, just want to say thank you for your help and this very useful information! Crazy enough that many joomla webmaster use these tag module which produce massive amounts of quite useless search pages. I edited my robots.txt and now I'm really curious about what happens the next months. Thanks again!
written by agung, January 29, 2008
thanks, nice articles you have...
my google index increase more and more after i installed the APF bridge component on my site. it is 15.000 now...  is it bad for seo? cause i think google will read it as a duplicate content. am i right? i've modified the robots.txt and i don't seem any changes. do you have a suggestion for me? thanks agung http://www.vetclinics.net written by Steve Burge, January 31, 2008
Hi agung
Yes - the APF Bridge is one component that can do a lot of SEO damage: http://www.alledia.com/blog/search-engine-optimisation-(seo)/joomla-seo--why-less-is-often-more/ Remove its pages using robots.txt. written by iougs, May 02, 2008
Been using sh404SEF for a while now .. it seems google is indexing a lot of search results instead of actual pages. I have a site map (submitted to google via webmaster tools) for as long as i've been using sh404sef (about a year, i'd say) so i don't see why it keeps doing it.
If I search for a keyword in google, results from my site typically appears like this: http://scoangers.iougs.com/index.php/Search.html?searchword=oite Note: it's hosted on a subdomain. Is there a way to get rid of this since it makes no sense especially when I have a joomla category on the site for that very keyword. It does this for lots of keywords. Is this a common thing ? What can I do ? Write comment
|
Show/Hide comments
Show/Hide comment form
I think the best one is using sef urls and adding
Disallow: /index.php/
Disallow: /content/
Disallow: /component/
Disallow: /*?*/
It's a part of my robots.txt file. I'm using urls without ? , content and component. my all urls are with real words, not like /content/52/96/ etc.