Make Your Site Search Engine Friendly: mod_rewrite

[iGuest 3]

One of the most desired features of the modern websites are that the web-pages must be search engine friendly. The crawler must crawl through your webpages smoothly and can reach all the pages of your website as per your desire.

 

Creating a static link like

<a href="https://m.secureserver.net/?ci=101147&prog_id=domainspricedright&mrk=sales.none&mrf=%2fsc_origin-b0000272%2f%3fci%3d101147%26prog_id%3ddomainspricedright&pl_id=1592 Home Page </a>

is still the best options as search engine bots can quickly pick-up such links. And you will definitely have a better chance of getting pages from your website indexed in major search engines.

 

But many of us are gradually moving from the days of static HTML pages to the dynamic pages and we have started using link like:

 

<a href="http://forums.xisto.com/no_longer_exists/ here to search for Jewelry </a>

 

Drawbacks of using dynamic pages:

 

Using dynamic pages, instead of age-old static HTML one, may give the impression to the search engine that the same page is being re-loaded. This fact has been mentioned in the Google's guide for webmasters.

Here is an abstrat of what Google says about it:

 

....... our crawlers may suspect that a URL with many dynamic parameters might be the same page as another URL with different parameters. For that reason, we recommend using fewer parameters if possible. Typically, URLs with 1-2 parameters are more easily crawlable than those with many parameters. Also, you can help us find your dynamic URLs by submitting them to Google Sitemaps.

For more details in this regard, please visit Google.com

 

Also it becomes difficult for the visitors of your website to read and type url containing too many partameters like

 

http://forums.xisto.com/no_longer_exists/ etc. etc.

 

 

So we may employ a technique to chage the dynamically created links to look like a static one.

So a link like this:

 

 

http://forums.xisto.com/no_longer_exists/ = search&category=jewelry

will look like

 

https://my.eurodns.com/login

How is it accomplished? This is what mod_rewrite does for you.

 

 

Create a file named : .htaccess ( no file name please) and upload it in the root directory of your webserver.

 

The file will contain the moe_rewrite instructions for your webserver.

Here is the General Format of a typical .htaccess file.

 

RewriteEngine On

RewriteRule Pattern Substitution OptionalFlags

 

That file will contain the RewriteRules for your website. And whenever your webserver gets a request, it will check for the RewriteRules as written in your .htaccess file and then deliver the webpages as per instructions of the .htaccess file.

 

The structure of a RewriteRule

RewriteRule Pattern Substitution [OptionalFlags]

 

Let us explain what all these mean.

 

RewriteRule

 

This command tells the server that now we are going to state a mod_rewrite rule.

 

Pattern

 

A regular expression which will be applied to the âcurrentâ visible URL (i.e. https://my.eurodns.com/login)

 

Substitution

 

It tells the server how to substitute for the values it recovered from the current url. That is

 

https://my.eurodns.com/login

will be transformed to

 

http://forums.xisto.com/no_longer_exists/ = search&category=jewelry

OptionalFlags

 

This is the only part of the RewriteRule which isnât mandatory.

 

Here is a model .htaccess file

 

RewriteEngine OnRewriteRule ^search/([^/\.]+).htm$ index.php?action=search&category=$1 [L]

Does it look bit ugly? Not at all, let us see how the server interprets it:

 

^search/

Directs the server to see whether the requested url starts with search/. If it doesnât, this rule will be ignored.That is, this rule will track the url like

 

http://www.yourdomain.com/search/jewelry.htm,

http://www.yourdomain.com/search/pets.htm

http://www.yourdomain.com/search/blaa.htm

 

 

([^/.]+)

Here, the enclosing brackets signify that anything that is matched ( and appears after "/") will be remembered by the RewriteRule.

^/. => indicates that it will neither accept a "/" nor a period *

=> Whatever else it will get, it will remember that.

So, in our example, it will remember the word jewelry after reading the url

 

https://my.eurodns.com/login

 

 

$

This symbol indicates the end of string

 

Now the substitution part contains:

 

index.php?action=search&category=$1

 

The actual page which will be loaded by Apache. $1 is magically replaced with the text which was captured previously.

 

A special substitution is -. This substitution tells Apache to not perform any substitution.

 

[L]

Tells Apache to not process any more RewriteRules if this one was successful.

 

 

Now once you have created your .htaccess file, now you require to modify the links of your webpages in accordance with the new RewriteRules.

 

That is link like:

 

http://forums.xisto.com/no_longer_exists/ = search&category=jewelry

Will be changed to :

 

https://my.eurodns.com/login

To test your .htaccess file, you need to

 

Open the httpd.conf file and uncomment the following lines (remove the trailing #s):

 

#LoadModule rewrite_module modules/mod_rewrite.so #AddModule mod_rewrite.c

I have shown here a basic example of mod_rewrite rule.

 

If you feel the need to mod_rewrite your website, then please visit doriat.com.

[warallthetm 0]

Interesting article. Would this parameter rule count if you only have one? Or dose google totally block all php parameters? I read in google that you cant have http://ww38.yoursite.com/?id=1&land=2 it says that you cant have a & after an id variable. Would it be ok if i did http://ww38.yoursite.com/?land=2&id=1 ?

[iGuest 3]

Interesting article. Would this parameter rule count if you only have one? Or dose google totally block all php parameters? I read in google that you cant have http://ww38.yoursite.com/?id=1&land=2 it says that you cant have a & after an id variable. Would it be ok if i did http://ww38.yoursite.com/?land=2&id=1 ?

It is not a matter of blocking, rather, GoogleBOT does n't (or did n't) pick up those urls quickly. Not quite sure of their effect on Google SERPs as I have seen URLs like virtuemart.net/index.php?option=com_weblinks&catid=73&Itemid=4 appearing in SERP. Anyway, it is better to mod_rewrite the URLs instead of using multiple '&' in it.

[lonebyrd 0]

I've got a quick newbie question. Is using something like the <a href="https://www.eurodns.com/?action=search&category=jewelry"> you mentioned, the best way to get google to see your website? I know with some of the other search engines (from what I hear) they use META tags. If so, I will definitly have to look into this mod-rewrite thing.

 

Oh, and one more question. Would you only have to do this to your main index page, or for every page on your website? Like I said, I'm a noob.

[vujsa 0]

I've got a quick newbie question. Is using something like the <a href="https://www.eurodns.com/?action=search&category=jewelry"> you mentioned, the best way to get google to see your website? I know with some of the other search engines (from what I hear) they use META tags. If so, I will definitly have to look into this mod-rewrite thing.

 

Oh, and one more question. Would you only have to do this to your main index page, or for every page on your website? Like I said, I'm a noob.

 

No, the best bay to get Googled PROPERLY is to use simplified URL's:

http://forums.xisto.com/no_longer_exists/

 

It is harder for the bots, crawlers, and spiders to properly identify different pages when you use complex URL's:

http://www.domain.com/index?action=search&category=jewelry

 

This is because index.php is the file name so any other URL with that file name can be confused with the other so the bot may think it has already been done so it can skip that URL.

 

mod_rewrite is a very good way to ensure that your site is throughly indexed by the spiders but has a couple of problems associated with it.

mod_rewrite requires the server to perform one additional step in the page load process which will eventually slow down your server and as a result your website when there is enough traffic.

You have to format the URL's on your pages to the simplified type. Usuually this has to be done one at a time and manually unless you write a script to parse your pages prior to sending to the browser to convert the complex URL's to simplified URL's.

- Then there is always the chance that you'll miss a rule for URL conversion.

- - This could cause many problems in the page if done incorrectly.

I don't mean to be negative about this since I really like the Friendly URL method of SEO.

 

Hope This Helps.

 

vujsa

[pyost 0]

mod_rewrite requires the server to perform one additional step in the page load process which will eventually slow down your server and as a result your website when there is enough traffic.

You have to format the URL's on your pages to the simplified type. Usuually this has to be done one at a time and manually unless you write a script to parse your pages prior to sending to the browser to convert the complex URL's to simplified URL's.

- Then there is always the chance that you'll miss a rule for URL conversion.

- - This could cause many problems in the page if done incorrectly.

As for point number two, most people can easily bypass this problem. I suppose not many of you have created their own dynamic sites, but use Content Management Systems and Blog software. These types of web sites usually support additional components (modifications, hacks, call them whatever you want) that automatically deal with this topic. For example, Joomla! CMS and WordPress have mod_rewrite support by default, without any additions. I don't know about phpNuke, e107 etc.

[iGuest 3]

Then there is always the chance that you'll miss a rule for URL conversion.

1.Usually mod_rewrite is employed for dynamic websites, where the links to different pages are generated from a backend database. So, if you can design the basic webpage and underlying datastructure to store the content in different categories, it will no longer be necessary to change each and every url manually. Only thing you should keep in mind is that you have mod_rewrite your website, so, the script generated links should look like
http://forums.xisto.com/no_longer_exists/

2. It is easier to start designing a website keeping in mind the mod_rewrite factor from the vey beginning. But the reverse is really difficult as at times you may find it difficult to rewrite an existing url.

3. It takes hardly few hours to learn the basics of mod_rewrite, but it is really helpful to give the urls a better shape not only in terms of the search engine crawlers but also in terms of the visitors to your website who will find it much more comfortable to type the entire url instead of typing parameterised urls.

4. ALWAYS KEEP A BACKUP COPY OF YOUR EXISTING .htaccess file, as that is going to play the key role once you have implemented mod_rewrite. In case of any problem, simply restore the previous version.

[Quatrux 4]

I agree that mod rewrite is really cool, but for those who can't use htaccess files on their sites can choose an alternative to this, just use php, for example, you can also "fool" different crawlers and robots (etc.) and use links like this yourdomain.com/file.php/mysection/myid.html the string after .php can be get in the $_SERVER; superglobal 'PATH_INFO' so here is an example, if in a file file.php you would write this code:

echo $_SERVER['PATH_INFO'];

You would get "/mysection/myid.html" So you can do whatever you want with it, of course if you have htaccess, you can remove the usage of .php and use only file, the .php will be found automatically, but nevertheless, if you can use htaccess, better go with mod rewrite.

[jeffc 0]

it is certainly a good strategy for SEO especially for Yahoo and MSN, the keywords in domain name is very important. Nowadays, a lot of CMS comes with this feature or there are a lot of plugins for it. For example, talking about the biggest CMS, Joomla, there are extensions such as opensef, sh404 that once you install them, you site will be a seo friendly site. For google, you may want to focus more on link building and social bookmarking sites