Plus, since Fast and Google, and possibly other engines, consider link popularity on a per page basis, you may find it beneficial to create one page and try to optimize it across the board to cover the major engines.

So, bear with me for a minute, and I'll teach you how to work with generic pages effectively, and then how to take those generic pages and make them engine specific, but only when needed.

Important Note:

As any of my workshop students will tell you, I believe very strongly in focus, focus, focus. So, when working with your generic pages, you'll want to create one page focused on one keyword phrase only. Don't worry about other keyword phrases on the same page. FOCUS!

To create a generic page, we'll need to use Gold's Page Critic feature.

Step-by-step walk through of Page Critic when working with generic pages:

1. After doing your keyword research, create a page that is focused on one high performance keyword phrase only. It won't hurt to insert other keywords in the text as you can, but always keep your main goal in focus.
2. Open WebPosition and choose "New" on the top menu bar, then choose Page Critic.
3. Under Domain Name, type in the name of your domain, such as www.your-domain.com.
4. If the page you want to analyze is on your hard drive, point WebPosition to that page by clicking on Select. Find the page by looking through the listings of folders and files on your hard drive. If the page you want to analyze is online, type in the URL, such as http://www.your-domain.com/name-your-page.html.
5. Under Analyze Page for Keyword or Phrase, type in your keyword phrase. Click Next.
6. Choose MSN and click Next. Why MSN? Because MSN is influenced by the Inktomi engine, so it still considers META tags to be important when determining relevancy. Therefore, it's a very good starting point for working with generic pages.
7. Under Settings, click Next.
8. In the Page Editor section, click Start, and the program will ask if you want to save the Mission. Click Yes.
9. The Page Critic results will now appear in your browser window.
10. Scroll down until you see the first two blue bars, and focus your attention on the information that falls between those two blue lines.
11. Work on each of the suggestions, then re-run Page Critic and watch the suggestions disappear.

You've now got a generic page, and you're ready to see how it ranks across the major engines.

Once you've taken care of as many Page Critic suggestions as possible, submit your page to the engines by using paid inclusion, by letting the engines find links to the page from other pages, or by using WebPosition's Submitter feature.

Once the page has had time to be indexed, check your rankings using WebPosition's Reporter. Watch your rankings for a month or two, because depending on which submission method you've chosen, it can take a while for your rankings to settle.

Then, look for holes in your strategy. Is your page doing well across the board? In many cases, by using MSN and creating a "generic" page, you'll find that the same page stands an excellent chance at ranking well across many of the major engines.

By going the route of the generic page at first, you've saved yourself an enormous amount of time, rather than creating engine-specific pages right off the bat. Plus, you won't have to spend as much time creating robots.txt files to keep engines away from pages that are not designed for them. Therefore, while Robots.txt files can be useful in avoiding the duplicate-content police, they also take time to build.

If your page is doing well in some of the engines but not others, that's the time to begin creating engine-specific pages. If you are needing the best results in the shortest time-frame, then creating engine-specific pages from the start will give you the greatest odds of achieving top rankings your first time out. However, if you can afford to be patient in achieving your rankings, generic pages are the way to go.

You can alleviate the greatest drawback of the generic page strategy by paying for inclusion on the engines that offer such services. That will get your key pages indexed within a week. Those pages that don't do well can then be optimized on an engine-specific basis.

How to create engine-specific pages . . .

Take your generic page and run it through each of the major engines in Page Critic where you aren't getting top rankings. Make changes based on Page Critic's recommendations for each engine, and save each page in a slightly different manner. Be sure to make a note of which page was optimized for each of the engines.

For example, if the name of your generic page is name-your-page.html, and if you need to create engine-specific pages for Fast and Teoma, you may want to name the other pages:

name-your-pages.html (plural version) for Fast
name-your-page.htm ("htm" versus "html" for Teoma)

Try to stay away from making it so obvious that you have engine-specific pages. For example, to avoid potential discrimination by an engine, you may not want to name your pages:

name-your-pages-AV.html (for AltaVista) or,
name-your-pages-FST.html (for Fast)

What about duplicate content?

Let's say your generic page ranks well with Google and the Inktomi-influenced engines, but it's not ranking well with Teoma or Fast/Lycos. If you create engine-specific pages for Teoma and Fast, you could end up having three almost identical pages, which the engines won't like.

Remember that the golden rule when working with content is that the content must be of value to both the search engines and the users. Having duplicate content is not of value to the search engines. They certainly don't want several versions of the same content cluttering up their index.

To keep from getting in trouble with duplicate content, you'll need to create a robots.txt file and allow certain engines to have access to certain pages while keeping them out of pages not meant for them. In other words, you'll direct the engines to whichever pages you want each engine to visit by using robots.txt files.

Robots.txt files:

Create a text file with Window's NotePad, NoteTab Pro, or any other editor that can save ASCII .txt files. Use the following syntax:

User-agent: (PutSpiderNameHere)
Disallow:/(PutFileNameHere)

The "user-agent" portion lets you specify which engines you want to keep out, and the "disallow" portion lets you specify directories or file names.

For example, to tell AltaVista's spider, Scooter, not to index the example files we discussed previously, create a robots.txt file as follows:

User-agent: Scooter
Disallow:/name-your-pages.html
Disallow:/name-your-page.htm

By creating a robots.txt file using this information, we're keeping AltaVista out of our pages created specifically for Fast and Teoma. You'll want to do the same for each of the other engines. Then, you'll want to create entries for Fast and Teoma that will keep them out of the original generic page.

That way, none of the engines will see duplicate content, and they'll only see the pages created specifically for them.

Save the page as robots.txt, and then upload the file to the root directory of your Web site. The "root directory" is where your index.html (or index.htm) page is located.

This is a very simple example of a robots.txt file, but they can get quite complex. One little mistake can cause an engine to find a page that you don't want found. Plus, you have to know the names of each engine's "user agent," or spider. That's why I recommend using a software program that creates the file and does the work for you.

An excellent software program for creating robots.txt files is RoboGen. The Limited Edition is a free version of the software, though they also offer a Standard Version with more features.

In Conclusion...

We all live in a very busy world, and we don't need to make more work for ourselves. Therefore, if your time is limited, start with a generic page, using Page Critic and the MSN engine. Then, if you don't get the rankings you want with that page, begin creating engine-specific pages by running the same page back through Critic for the other engines. If you later find that pages optimized for another engine tend to rank better across the board, then use that engine rather than MSN as your starting point.

You'll be surprised at how well your generic pages will do since many engines look for similar criteria when ranking a page. With the extra time you'll save, you can create more high performance pages for your site, targeting a greater variety of keywords, thus increasing your traffic that much more!

Good luck!!

Robin Nobles is the Director of Training of the Academy of Web Specialists, which teaches online training in search engine marketing. She is also a trainer with Search Engine Workshops, which presents on-location workshops in search engine marketing at various locations across the globe.

Editor's Note: Robin's robots.txt approach is used successfully by many to avoid spamming a search engine with near duplicate content. However, others will argue that a safer method is to avoid any near duplicate content on your Web site altogether. While in theory a properly set up robots.txt could prevent a search engine from indexing two pages that had similar content, it is not foolproof. For example, you could make a mistake in setting up the file, inadvertently introducing similar content to an engine. The Robogen product or one like it can certainly help you in this regard, but no product can prevent every error you might conceivably make.

In addition, some Webmasters believe that the engines do not always honor the robots.txt and could bypass it either because of a glitch in their system, or in a conscious hunt for duplicate pages. To be fair, the majority of Web marketers believe that search engines do not care if you have files residing on your site that are similar, or designed for different engines, so long as those extra pages do not clutter and degrade their index. With billions of pages on the Web, engines have enough content to index without ignoring the robots.txt protocol and indexing pages not intended for them. Lastly, to ignore the robots.txt protocol puts an engine at risk of being sued based on trespassing or copyright laws, a situation most engines wish to avoid.

Ultimately, your choice of strategy is up to you. If you've not yet optimized the existing pages on your site, then obviously that is the logical first step to achieving visibility on the search engines.

                 ← What is this?

---