After you sign up for the Sitemap program, you submit your sitemap to Google and they will use it to index your website content. It's a good idea to participate in the Sitemap program, but it's not an absolute requirement for getting indexed if you follow some simple rules of website design.

Many search engines, like Google, normally use a software program called a "spider" to add website content to its database. The spider is designed to follow links on a website in order to index as many of the site's pages as possible. It's in your best interest to help spiders crawl your website by creating a spider-friendly navigation structure. One way to assist search engines is to create a sitemap. A good sitemap is a simple page that contains links to all of the pages on your site containing important content. You should link the sitemap to your homepage, and (if possible) every page on your website.

If you have a large website, then listing every page URL on a single sitemap page may not be practical. In that case, consider the design heirarchy of your site and make sure that the important category pages are listed on the sitemap. Since spiders do not normally crawl more than 3 levels deep into a website, make an effort to design the linking structure so that pages containing important content are not out of the spider's reach.

To create a sitemap, first generate a list of URLs from your website. WebPosition's Submitter feature has a built-in spider that will crawl a website and provide a list of URLs that can be used as a basis for a sitemap. For Google's Sitemap program, they accept sitemaps in several formats, and a text file list of URLs is one supported format. It's also a good idea to use a link checking program such as Link Defender to ensure that all links are functioning properly. You might also consider building a custom 404 error page which will allow you to capture traffic from dead pages on the site that were listed in search engines in the past.

After you have created a good sitemap and internal linking structure, submit your homepage and sitemap URLs to the search engines. An easy way to submit is to use WebPosition's Submitter feature. However, note that for the Google Sitemap program, you will need to log into your Google account to submit your sitemap.

Finally, as described in this article, measuring website statistics like spider crawl depth and frequency can provide useful information about the effectiveness of your internal linking structure and sitemap. Web analytics software like WebTrends can capture this kind of information for you.

                 ← What is this?