This is a post about dynamic web sites and rewrite urls.
Bottom line: Clean Up Your URL’s to make user and search engine friendly urls
The problem with most e-commerce sites today is that their product pages are dynamically generated. While Google will crawl any URL that a browser can read, most of the other search engines balk at links with “?” and “&” characters that separate CGI variables
(such as “yourdomain.com/store?sku=123&uid=456”).
As a result, many individual product pages don’t show up outside of Google. One way to circumvent this difficulty is to create static versions of your site’s dynamic pages for search engines to crawl. Unfortunately, duplicating your pages is a huge amount of extra work and a constant maintenance chore, plus the resulting pages are never quite up-to-date – all the headaches dynamic pages were designed to eliminate. A far better strategy is to follow the lead of Amazon and rewrite your dynamic URL’s in a syntax that search engines will gladly crawl.
So URL’s that look like this …
amazon.com/store?shop=cd&sku= B00004WFIZ&ref=p_ir_m&sessionID= 107-6571839-6268523
… become …
amazon.com/exec/obidos/ASIN/ref= B00004WFIZ/ref=pd_ir_m/107-6571839-6268523
Amazon’s application server knows the fields in the URL are actually CGI parameters in a certain order, and processes them accordingly. There are Apache rewrite rules that tell Apache how to translate slash-separated URL’s into a format used by their application server. On the back end the server is passed something like this:
yoursite.com/cgi-bin/CssP.exe?CsspApp= yoursiteClient1&CssServer=localhost%3A32401&CsspFn =@/details/ArtistDetail.php:@:getForm&ObjectLocation =ART&ArtistID=3918
But users and search engines see the search engine friendly urls, Apache-served URL’s, which look something like this:
yoursite.com/artists/profiles/3918.php
Tip: rewrite tools
When you run your web site on a windows server you can use little software rewrite script from Smalig to rewrite your dynamic website to search engine friendly urls. You can find it here
A good tutorial for apache rewrite rules to rewrite your dynamic website to search engine friendly urls you can find here at webmasterworld
Not only are the rewritten URL’s crawlable by all search engines and called search engine friendly urls. They’re also more human-friendly, making them easier to pass around the Net.
Dynamic web sites and pay for inclusion programs
With Pay for inclusion programs like Overture / Yahoo Search marketing you can pay to have specific URL’s indexed in Yahoo, Alta Vista and FAST. That works for dynamic URL’s as well! The only thing required to have your dynamic web site indexed through this programs is that you can link directly to the web pages you want indexed. This is very simple to test. If you can copy the URL of the page you want indexed and past it back in a new browser window and get the right page loaded you should be all set.
For further information on how to index your dynamic web site, hire an experienced search engine optimization expert with expertise in these areas. Most often it takes a closer examination of your web site to determine the best strategy.