Nov 12 2007
How to Solve Common SEO Problems with JReviews, Fireboard and MosTree Print
Monday, 12 November 2007

Robots.txt I've talked before about how having less pages indexed is often better and after doing SEO audits on several sites last week I was reminded of how important this was. Sites with only 200 or 300 genuine pages of content had over 1000 pages indexed in Google.


Lots of Joomla components produce 100s and sometimes 1000s of unnecessary extra URLs that are then indexed by Google. A simple example is Mostree which is a very robust component but because it was built in pre-Ajax days, it creates separate 4 extra pages for each listing: recommend, rate, review and report listing.


In this post we're using JReviews, Fireboard and MosTree as examples. You can cut-and-paste the rules from this post into your robots.txt file. However, please read the "important notes" section first.

Important notes

1) If you add a rule that contains wildcards (*) then you are removing all the URLs from Google that contain that word. For example:

  • If you use JReviews for a mouthwash review site, then Disallow: /*list*/ will remove all the unnecessary lists but also your opinion on Listerine.
  • If you use Mosets Tree for file listings, then Disallow: /*contact*/ will remove some surplus contact pages but also your information about Jodie's Fosters so-so Sci-Fi Movie called Contact.


2) These rules are for sites with Search Engine Friendly URLs turned on. Change the rules to fit your own uppercase / lowercase settings.


3) Be careful. Robots.txt files can have a big impact on your site.

Mosets Tree

Disallow: /*review*/
Disallow: /*rate*/
Disallow: /*recommend*/
Disallow: /*report*/
Disallow: /*Owner*/
Disallow: /*Contact*/
Disallow: /*List_Alpha*/
Disallow: /*New_Listing*/
Disallow: /*Featured_Listing*/
Disallow: /*Popular_Listing*/
Disallow: /*Most_Reviewed*/
Disallow: /*Most_Favoured*/
Disallow: /*Top_Rated*/
Disallow: /*Most_Rated*/
Disallow: /*Recently_Updated*/
Disallow: /*Add_Category*/
Disallow: /*Add_Listing*/

JReviews

Disallow: /*list*/
Disallow: /*myreviews*/
Disallow: /*reviewrank*/
Disallow: /*rss*/
Disallow: /*order*/
Disallow: /option=com_jreviews/

Fireboard

Disallow: /profiles/
Disallow: /*action=reminder*/
Disallow: /*action=stats*/
Disallow: /*;msg*/

Any Kind of Calendar or Photo Gallery

Disallow: /calendar/
Disallow: /photogallery/


For SEO Club members we're building a longer list covering more components. Click here to view.


Comments (7)Add Comment
Calendar/gallery ?
written by Brian Teeman, November 12, 2007
Why do you want to disallow the calendar and photo gallery as a whole?
...
written by trichnosis, November 13, 2007
@Brian Teeman: As i know , google and the other search engines is avoiding the calendar crawls. calender crawls will not bring you anythink smilies/smiley.gif
...
written by Tom, November 13, 2007
Nice post! I run the latest version of m tree and it now has rel='nofollow' on many of the links such as recommend etc. I'll still be applying a few of the other rules to my robots.txt file though!

I'm also looking into modding fireboard to include rel='nofollow' on all external links posted in forum and sigs, I'll report back if successful!
...
written by Steve Burge, November 13, 2007
Hi Brian

Sorry - I should have explained that. Trichnosis thinks like me. Google has a habit of crawling 1000s of pages in most calendars like this poor guy: http://forum.joomla.org/index....294.0.html

Photo galleries are normally text-free or text-light and offer little SEO value.

Hi Tom

Thats great news - I hadn't checked out the Mostree update.

That would be a nice Fireboard touch and would certainly help cut spam. I know some advanced forums (VBulletin etc) are looking into adding nofollow to normal signature links and removing it from trusted users.
Is mine Robot.txt look right???
written by Igor Dernov, November 15, 2007
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /components/
Disallow: /editor/
Disallow: /help/
Disallow: /includes/
Disallow: /language/
Disallow: /mambots/
Disallow: /media/
Disallow: /modules/
Disallow: /templates/
Disallow: /installation/
Disallow: /*review*/
Disallow: /*rate*/
Disallow: /*recommend*/
Disallow: /*report*/
Disallow: /*Owner*/
Disallow: /*Contact*/
Disallow: /*List_Alpha*/
Disallow: /*New_Listing*/
Disallow: /*Featured_Listing*/
Disallow: /*Popular_Listing*/
Disallow: /*Most_Reviewed*/
Disallow: /*Most_Favoured*/
Disallow: /*Top_Rated*/
Disallow: /*Most_Rated*/
Disallow: /*Recently_Updated*/
Disallow: /*Add_Category*/
Disallow: /*Add_Listing*/
...
written by Steve Burge, November 15, 2007
Hi Igor

Yes - thats right for older versions of Mostree.

As Tom points out above, the new version adds no-follow for you in some situations
SEO by the Hour
written by SEO GUY, February 19, 2008
Great Info on Mosets Tree!

Write comment
quote
bold
italicize
underline
strike
url
image
quote
quote
smile
wink
laugh
grin
angry
sad
shocked
cool
tongue
kiss
cry
smaller | bigger

busy
 
Joomla SEO Club and Book Logo
Search
Login
Blog Details

Creative Commons License All blog articles are licensed under a Creative Commons Attribution 3.0 United States License.
Top Comment Posters
Good Web Practices
(141 comments)
Brian Teeman
(105 comments)
Klaus
(83 comments)
türkçe sözlük
(52 comments)
Cory Webb
(49 comments)
Hummerbie
(42 comments)
Amy Stephen
(36 comments)
guido
(36 comments)
ILJ
(35 comments)
Yannick Gaultier
(33 comments)
Blog Categories
Accessibility
Affiliates
Alledia News
Analytics
Book Reviews
Business
Design
Domain Names
Domain Tips & Tricks
Drupal
E-Commerce
Extensions of the Month
General CMS Issues
Interviews
Joomla Affiliates
Joomla 1.5
Joomla Blogs
Joomla Hacks
Joomla Hosting
Joomla News
Joomla People
Joomla SEO eBook
Joomla SEO Rankings
Joomla Sites
Joomla South East
Joomla Templates
Joomla Tips & Tricks
Joomla URLs
Open Questions
PHP
Pay Per Click
Product Reviews
Rants
Scams
Recommended Sites
Search Engine Optimization
Security
SEO
SEO Club
SEO Competition
Site Reviews
Template Clubs
Yellow Pages
Virtuemart
Vlogs
Wordpress
right