Answer: Perform any one of these maneuvers wrong and you might wind up serving cheap beer and hot dogs at your next Superbowl party... instead of that premium brew and filet mignon you know your friends really want.

Oh, and the other commonality is their solution - the versatile 301 redirect... which is, amazingly, our most common answer to the myriad tech questions we receive. Here's a sampling...

• How do we switch to a more keyword worthy domain name without losing our search engine rankings.

  Answer: 301 redirect

• What's the recommended way to combine site traffic coming into www.ourdomain.com and ourdomain.com to avoid splitting our PageRank between what Google sees as, technically, two different URLs?

  Answer: 301 redirect

• How do we manage our ancillary domains we've set up to catch traffic from misspellings and variant keyword searches for our company, products, or services?

  Answer: 301 redirect

In fact, if we had to guess the answer before knowing the question, we'd guess 301 redirect - kinda like Carnac the Magnificent or those Jeopardy shows where the answer precedes the asking of the question:

Answer: 301 Redirect

Question: How do I avoid losing valuable search engine traffic so I don't have to scale back my Superbowl Party Menu to cheap beer and hot dogs?

What's a 301 Redirect?

The number 301 identifies a specific 3-digit server header code that is generated by a web server when a browser or search engine spider tries to download a web page. The 301 code says the page has been permanently moved to another location. Browsers and spiders then know to "redirect" themselves to any new address that you specify in your .htaccess file.

What's a .htaccess File?

A .htaccess (pronounced "dot-H-T-access") file is a simple text file that resides in the root directory of your web site where, typically, your home page also resides. The .htaccess file serves a variety of different functions and can live in other directories as well but, for the purpose of doing a 301 redirect, you should place a .htaccess file in your root directory. If one doesn't already exist, you can create it using any plain-text editor (like NotePad, or TextPad), then name the file .htaccess (that's right, no extension - just a period followed by htaccess).

If your site's root domain already contains a .htaccess file, you can simply scroll to the end of whatever code is already there and append your 301 redirect code at the bottom of the file, starting on a new line. While this may sound complicated, it's actually very, very simple to do. If you can upload files to and from your web server, then, chances are you'll have no trouble managing (i.e. altering or creating and uploading) your .htaccess file(s).

There are Three Primary Reasons to Use a 301 Redirect:

1. To change primary domain names - an event that commonly occurs whenever a company changes names or switches to a more keyword-friendly domain.
2. To ensure that SE spiders only index either the www or non-www version of your site. Since each version is technically a different URL, to have both versions available for indexing dilutes PageRank, as it is being split between multiple sites.
3. To catch traffic from ancillary domain sources without presenting duplicate content to the engines and, thereby, running the risk of being penalized.

   For instance, it's good online strategy to protect your trademark or other proprietary product or service names by purchasing related, but different, top-level domain names. Examples would include .net or country-specific extensions (such as .co.uk), or even common misspellings. And, whenever such domains lend themselves to type-in traffic, you'd be well advised to activate them - but use the 301 redirect to seamlessly send that traffic (along with the SE spider) from these ancillary domains to your primary domain.

Changing Your Domain Name

If ever you should need to change your domain name, proceed as follows:

1. First, don't do it unless absolutely necessary. Chances are very good you will lose ranking, inbound links, and traffic - at least temporarily. The steps outlined here can help you minimize, but not entirely eliminate, these negative repercussions.
2. If you are, however, convinced that you must change your domain name, be prepared to leave your old site up for at least three months and possibly longer depending on the length of time it takes for your site logs to show all traffic to the old site has stopped. Of course, you'll also want to be certain that all major engines who had indexed your old site have spidered and re-indexed your new site.

   Your new site logs will show when you are no longer receiving referral traffic from the old site. Your old site logs will show when you are no longer receiving referral traffic from the search engines.

3. When making the switch, be sure to leave the content up on the old site. You'll want to copy the old site to its new location, as opposed to just moving the site to a new location. The difference is that when you copy a site, you leave the original in its old location. If you take the old site's content down and just replace it with a 301 redirect, the spiders may see the old pages as empty content and simply delete those pages from their index.

   While it's true that you run a small risk of being snagged by the duplicate content filter, it's a minor risk compared to the risk of having your site dropped entirely out of the listings. Besides, since both Google and Yahoo recommend the 301 as the preferred method for permanently redirecting to a new site, you've given yourself a defendable reason for the duplicate content to co-exist at least until your logs show that traffic to your old site has ceased.

   Also bear in mind that the duplicate content penalty is usually applied to only one of any two identical pages, so people should still be able to find your site - perhaps just not under your preferred domain at first. Still, that's much better than being dropped altogether if the 301 redirect doesn't work due to a search engine glitch beyond your control.

   Critical point: If the content of your old site appears gone to the indexing spider, then you're out of the listings - no traffic equals no money and it's cheap beer and hot dogs for you and your friends come Superbowl-party-time (ouch!).

4. Contact everyone who links to you and ask them (beg them, pay them?), to change their link to your new site. Monitor this process to make sure they follow through. Sure, this can be a headache if you have a lot of links but, if people don't change them, all those links won't do you much good - and they'll all be broken links when you finally take the old site down (a fact you might want to use as leverage while gently persuading the referral site owners to make the necessary link changes - after all, they probably don't want "broken links" on their site).

   Unfortunately, there isn't an effective shortcut to getting other sites to update their links to you. That's just one of the reasons why we say don't change your domain name unless it's absolutely necessary. The more incoming links you have, the more challenging it is to switch to a new domain name.

5. Once you're sure you want to make the change, use a 301 redirect. On an Apache server (running UNIX) a 301 redirect is very easy to do. Just append the following statement on a separate line at the end of your .htaccess file:
   redirect 301 / http://www.newdomain.com/

   By placing the above statement in the .htaccess file within your website's root directory, you'll be redirecting all traffic from the old site to whatever site you specify in http://www.newdomain.com/.

   Important to note: This simple one-line command only works if you are maintaining the same exact structure of the old site on the new site. Every page on the old domain will be redirected to its equivalent page on the new domain. However, if you have some pages on the old domain that are not going to be on the new domain, this single line command would redirect users and spiders to some non-existent pages... the ones you removed. Therefore, bear in mind that your site move will be easiest on you if you just copy the new site structure exactly the same as the old site.

   Of course, once you've established your site under the new domain name and have phased out the old domain name, you can then start rearranging the site structure. In other words, don't try to change your domain name and restructure your site at the same time unless you're either a professional or a masochist.

   If you only want to redirect portions of the old site to the new site, such as a single directory, you can place the following text within a single line in the .htaccess file:

   redirect 301 /directoryname/ http://www.newdomain.com/directoryname/

   This is useful if you want to test out the redirect on part of your site before redirecting the whole thing.

   You can also redirect individual pages:
   redirect 301 /directoryname/pagename.html http://www.newdomain.com/directoryname/pagename.html

   Remember: the command above needs to all be on a single line in order to work.

6. If you're listed in directories such as DMOZ.org (aka, the Open Directory Project [ODP]), then changing your domain name presents an additional challenge. It's not so much that changing domain names within DMOZ is a problem, it has more to do with the fact that many other directories, including the Google directory, get their directory listings from DMOZ. That means that, depending on how frequently they sync up their listings with DMOZ's, it could be quite a while before those other directories list your new domain. As we've suggested, moving a domain can hurt your rankings and the lack of directory listings for the new domain (resulting in a temporary link popularity deficit) is one of the reasons why this is true.

   Regardless, to change your domain name in DMOZ, go to the category in which you are listed and click on 'Update URL' at the top right. You'll be asked to enter both your old and new URLs. A DMOZ editor will then review your site and change your listing. Nearly all directories have an Update URL feature, so that's where to look for a starting point in updating your domain information within the various directories.

   By the way, don't attempt to list your new domain along with your old domain within a directory. They strongly object to duplicate listings and your new domain definitely won't make it past the directory's editors.

7. Once you've set up your redirect, use a server header checking tool (such as: http://www.seoconsultants.com/tools/headers.asp) to verify the server is correctly processing your redirect. You'll want the site you're redirecting from to return a 301 Moved Permanently header and the site you are redirecting to to return a 200 OK server header.

   Here's a sample 301 redirect to give you an idea of what a server header checking tool will reveal when using a 301 to consolidate www and non-www URL's, giving preference to the www version (notice the relevant portion of the response highlighted in yellow).

   
   
   
   
   

   #1 Server Response: http://sampledomain.com/

   HTTP Status Code: HTTP/1.1 301 Moved Permanently Date: Wed, 15 Dec 2004 23:09:18 GMT Server: Apache/1.3.29 Location: http://www.sampledomain.com/ Connection: close Content-Type: text/html; charset=iso-8859-1 Redirect Target: http://www.sampledomain.com/

   
   

   
   
   

   #2 Server Response: http://www.sampledomain.com/

   HTTP Status Code: HTTP/1.1 200 OK Date: Wed, 15 Dec 2004 23:09:23 GMT Server: Apache/1.3.29 X-Powered-By: PHP/4.3.8 Set-Cookie: PHPSESSID=be8ce4fee0c29c968c8382be1fdfa3bd; path=/ Expires: Thurs, 16 Dec 2004 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Connection: close Content-Type: text/html

   From the example above, we see that, when the spider initially requested the page, the server read the .htaccess file which told it to respond with a 301 Moved Permanently response code. After which the spider redirected itself to the new domain at http://www.sampledomain.com/. There the spider received a 200 OK response code, telling it that the page was successfully located. This is the combination of responses you'll be looking for when you setup your own 301 redirection.

   Most web hosts pre-configure their servers to point both the www and non-www version of your URLs to the same page by default. This is good because someone forgetting to type the www will still be directed to your site rather than getting a 404 not found response code (one of those other 3-digit server codes).

   However, pre-configuring the server to default all pages to the www version can also be a liability. That's because this allows people to link to your site with or without using www. The end result being that search engines may index some of your pages with the www, and some of your pages without. This effectively splits your site into two different sites in the eyes of the search engines and dilutes your incoming link popularity (think PageRank).

   Ideally, you'll want all of your listings and links to use one or the other (we mildly prefer the www - but consistency is really the only important factor).

   Worth noting is that both Google and Yahoo use a redirect to consolidate their traffic to the www version of their URL. For example, entering http://google.com/ will redirect you to http://www.google.com/.

   To ensure that search engines only index the www versions of your site's pages, place the following code into a .htaccess file in your root directory

   
   <IfModule mod_rewrite.c>
   
   RewriteEngine On
   

   RewriteBase /
   
   RewriteCond %{HTTP_HOST} !^www\.your-site\.com [NC]
   
   RewriteRule ^(.*)$ http://www.your-site.com/$1 [L,R=301]
   
   </IfModule>
   

   Remember to replace www.your-site.com with the www version of your domain name.

   To make sure search engines only index the non-www versions of your pages, use the following:

   <IfModule mod_rewrite.c>
   
   RewriteEngine On
   
   RewriteBase /
   
   RewriteCond %{HTTP_HOST} !^your-site\.com [NC]
   
   RewriteRule ^(.*)$ http://your-site.com/$1 [L,R=301]
   

   </IfModule>
   

   Of course, replace your-site.com with the non-www version of your domain name.

   In case you're wondering why the code used to merge your www and non-www pages is a bit more complicated than the code used to change your domain name, it's because the www and non-www versions of your site both point to the same place on the server (they are considered aliases of each other). This means that anything you put in your .htaccess file applies to both types of pages. If you were to use the simpler redirect that we used to change domain names, the redirect would get stuck in an endless loop - redirecting back and forth between the two types of pages. Eventually your users would just see an error message telling them they had exceeded the number of redirects permitted.

Owning Multiple Domain Names

Another use for a 301 redirect involves protecting your trademark or other proprietary keyword-friendly or misspelled domains by purchasing and activating such applicable domain names and then redirecting them to your primary domain.

This is most commonly done with the idea of keeping them out of the hands of competitors or to avoid losing traffic when customers misspell your domain name. For instance, both Google and Yahoo also use a redirect to correct for what is probably their most common misspelling. Try going to http://gooogle.com or http://www.yahooo.com and notice that each misspelling redirects to their correctly spelled main site.

Once again, a 301 redirect is the best way to send all traffic generated by these ancillary domains immediately to your primary domain. Just place the following code in a .htaccess file within the root directory of each of these extra domains:

redirect 301 / http://www.your-site.com/

It isn't necessary to populate such sites with content since their purpose is simply to protect your proprietary names and trademarks or to catch misspelled traffic - and they aren't intended to be indexed by the engines.

Why Not Just Use a Meta Refresh?

The meta-refresh is accomplished by using a specific meta tag placed within the <head> section of a web page as such:
<meta http-equiv="refresh" content="0; url=http://www.your-site.com/new-page.html">

The part of the meta tag that specifies content as being equal to 0 means that this page will immediately redirect to http://www.your-site.com/new-page.html. Unfortunately, this technique has been so heavily abused that it's now considered unforgivable spam by probably every search engine known to humankind.

Literally one of the oldest tricks-in-the-book (described as Trick #4 in our UnFair Advantage Book), the meta-refresh was commonly used to stuff tons of keywords strategically into a truly-ugly-page for the sole purpose of boosting rankings. Then, when a user clicked the link provided by the SE results, they were instantly redirected to the un-optimized, consumer version of the page. The real (keyword stuffed) page was seen and indexed by the spiders, but when consumers visited the page, it was immediately replaced by the un-optimized content page the site owner wanted real people to see.

To say the engines hated this trick would be an understatement. Soooo, uh, ...we recommend you avoid (like the plague) using the meta-refresh tag on any site that you actually care about being indexed and be forewarned that any site you redirect to could actually get banned!

What About Using a 302 Redirect?

The 302 redirect is also know as the "temporary" redirect, as opposed to the 301 permanent redirect. Use this redirect in such cases where you want to tell a search engine spider this page has been temporarily renamed or moved to a new location, but we're planning to put it back before long. This means that the spider will go to the new location to find the page but the engine won't update the path to the page within its index.

As you might imagine, cases where you need to temporarily redirect a page are pretty rare and many sites use the 302 temporary redirect when they actually should be using a 301 permanent redirect. Usually this doesn't cause problems but there was recently one high-profile case where the web site business.com lost most of the PageRank on its 16,000 plus pages. Many experts attributed this snafu to the improper use of a 302 redirect.

Worth noting is that Google didn't actually penalize business.com. Instead, the Google spider was likely confused by the incorrect use of a 302 temporary redirect meant to permanently redirect many of their pages. Google was indexing the new pages just fine but listing them at their old location since Google assumed the move was temporary. And, that's exactly the result a 302 redirect is supposed to produce.

A similar problem can occur when directory links use a 302 redirect to your site. For example, say your site, www.small-site.com, is listed with a fictional directory, www.big-directory.com, that sends visitors to you via a 302 redirect. What often happens is that Google will list your site's content under the www.big-directory.com URL. The duplicate content penalty ensues, and your site's ranking is now sucking sewer water. The lesson here is: avoid 302 redirects unless the move actually is temporary - and take care that directories you're listed in link directly to you rather than processing your link through any kind of redirect. It isn't uncommon to see a site's good ranking tank after being listed in a directory that uses a redirect for the listing link.

Can't I Just Use a Custom 404 Error Page and Avoid the Whole Thing?

This solution only works for human visitors who come to your site. There they'll typically see a page that says something like...

This page has been moved. Click here to go to the new location. Please update your bookmarks.

However, this doesn't work at all for search engine spiders because they don't understand people language - only computer code. There isn't anything in the 404 header code that helps spiders understand that a page has been moved. To an SE spider, 404 means that page no longer exists and it should be purged from their index. Once this happens, the process of getting your page re-listed begins anew and, before you know it, you're faced with a cheap beer and hot dogs Superbowl debacle!

What About the Yahoo 301 Redirect Bug?

We touched on this issue a few months back in se-bytes. The problem was that Yahoo was treating 301 permanent redirects similar to 302 temporary redirects - i.e., they listed the contents of the new page but under the address of the old page.

The good news is that Yahoo seems to have finally fixed this bug. Yahoo recently did a major overhaul in their approach to handling 301 redirects, which is gradually taking effect as the Yahoo spider re-crawls and re-indexes pages across the Internet. Hopefully, within a few weeks we should see the last vestiges of the Yahoo redirect bug disappear forever.

The following image is taken from a PowerPoint slide used in a recent talk by Tim Mayer of Yahoo. It outlines Yahoo's current approach to handling 301 redirects:

If You're On a Windows Server

The examples above all are specific to Unix- or Linux-based web servers. It's also possible to use redirects on a Windows-based web server. This is easiest if you use scripting in a .asp page (which is one good argument for using .asp for all the documents on your site, even if you don't put scripting in them).

Here's an example of a .asp 301 redirect (save to yourfilename.asp):

<%@ Language=VBScript %>

<%

Response.Status = "301 Moved Permanently"

Response.addheader "Location", "http://www.newdomain.com/newurl/"

Response.End

%>

Here's an example of a .asp 302 redirect (save to yourfilename.asp):

<%@ Language=VBScript %>

<%

Response.Status = "302 Moved Temporarily"

Response.addheader "Location", "http://www.newdomain.com/newurl/"

Response.End

%>

Here are some more good .asp redirecting tips. And Microsoft can give you more info on how to redirect pages using their IIS Server.

Troubleshooting Your 301 Redirects

Sometimes it can seem like you're doing everything right but still getting weird behavior from your 301 redirects. Here are the two most common problems we've come across:

First, .htaccess files in your subdirectories will override any directives you've specified in the .htaccess file in your root directory. So if some of your pages don't seem to be redirecting properly, make sure that they aren't in a subdirectory with a .htaccess file that is interfering with your redirection. If they are, you may need to edit the .htaccess file in that subdirectory.

Second, sometimes browsers will cache pages, which may make it appear that your redirects have not taken effect. Internet Explorer for Windows is especially bad about this. You'll often need to clear your cache and do a hard refresh (ctrl-F5) in order to get your redirect to work, and many times even that won't make it work. Fortunately, the problem exists on your browser only. If you were to move to a different computer, the redirect would work fine. Unfortunately, it can also drive you nuts when you're testing out your redirects. If possible, avoid the whole thing by testing your redirects using a more modern browser, such as Firefox (or Safari for Macs).

Looking Forward to a Great Party

The whole point is to help you intelligently manage your domains while steering clear of some itsy-bitsy, potentially catastrophic mistakes that can completely decimate your online cash flow. Consider for a moment the plight of business.com that reputedly lost ranking on 16,000 pages. We can't say for sure, but we suspect that could lead to the loss of lots of premium brew and tasty treats making for a pretty sad Superbowl party.

Fortunately, it's fairly easy to get it right assuming you follow procedures. That should help keep the cash rolling, the prime cuts sizzling, the premium suds flowing, and your friends impressed! Now, if only there were a 3-digit header code to help your favorite team actually win the Superbowl!

Here's to football,
Stephen Mahaney - President
Planet Ocean Communications

---

The above article is reprinted with permission from Planet Ocean's SearchEngineNews.com, copyright 2005, and distributed with permission by WebTrends maker of WebPosition, the award-winning software that helps track and improve your search engine rankings. You may download a FREE trial copy of WebPosition from: http://www.webposition.com/trial/

                 ← What is this?

---