You do a search for a keyword that applies to your site, and a page pops up at the top or very near the top for no logical reason. In many cases, the keyword searched cannot even be found on the page! Or at best, the keyword is only there once, where your page has it many times, yet still ranks lower. Stop banging your head against the wall and learn the various reasons why this happens and how you can combat it to gain the advantage.

There are many reasons why the above scenario can occur. We'll list and discuss each:

1. Out of date pages: A Webmaster can change the content of a page at any time and re-post it to his site. Unfortunately, unless the page is resubmitted and/or respidered, the search engine may not know to update its database for quite some time. Even when the Webmaster resubmits, some engines take weeks before they get around to re-visiting the page.

What can be done in this scenario? Go to the engine's submit URL page and suggest they re-index that page. If this is the problem, the page should drop in rank as soon as the engine visits the page. On the slow indexing engines, you may have to wait awhile, but submitting should accelerate the process.

A strategy that some highly competitive SEO's employ is to submit their competition's URL through pay inclusion. Then, they're guaranteed that the engine will visit the page and respider it within 48 hours, or whatever time frame that pay inclusion program operates under.

2. Cloaking, IP Spoofing or the "Switcheroo" technique: Some unsavory Internet marketers create a server side script that will watch for a list of known search engine spiders. When they see one, they "serve up" a page that is highly optimized to rank well but is generally not pleasing to the human audience.

The page can rank better than most pages because they are not concerned about how it looks to the average user. When anyone besides a search engine visits the page, the site serves up a very "pretty" page, which may not even contain the keyword on it at all!

Obviously, we strongly object to this technique, since it can serve up pages to the searcher which don't even apply to the search term, undermining the search engine's value. Optimizing your page to apply to a search based on its real content is one thing, but "spoofing" is simply bad business and is against the rules.

The good news is some of the search engines are now using a technique that we won't disclose that catches these guys and bans their entire site. However, we don't think they've come close to catching them all yet. If you run into a page where you think this may be happening, you can report it to the engine, and if you're lucky, they will investigate it.

3. New ranking algorithm: Search engines change their ranking algorithm from time to time. Some techniques that worked well before may be penalized now. When this happens, sometimes the engines take awhile before they bother to re-index their entire database under the new "rules." Until that happens, some older pages may continue to rank high, even though your submission emulating them in nearly every way refuses to score well.

In this case, the solution again is to submit the page, which should apply the "new" rules to it and score it under the current relevancy system.

4. Popularity factor: Some pages may have the keyword once or twice but certainly aren't using all the tips and strategies to deserve such a high ranking. This can be very frustrating to those of us reaching for the "brass ring" and seeing it go to a seemingly undeserving page.

An explanation can sometimes be attributed to the popularity factor. If the site has lots of popular, related links pointing to it, and if those links are describing the site using the important keyword phrase, then some engines will score the page higher. Their rationale is that the site must be popular, since so many related and popular sites linked to the Web site and described it using the important keyword phrase. Therefore, the site must be about that keyword phrase.

To determine if link popularity is a possible reason why a page scores high, you can learn the popularity of a page by visiting Link Popularity Check, or by checking the link popularity box on the Settings tab of Page Critic.

The popularity factor is yet another reason to spend part of your marketing time soliciting or exchanging links with other sites. With at least two of the engines, Fast and Google, link popularity is based on the individual pages. With the other engines, how link popularity is determined, whether by the page or by the site, is unknown.

5. Search engine bugs: Yes, even the big commercial search engines have bugs. Since they are continually trying to fine tune their system to provide better results or to get rid of abusers of the system, bugs can easily make their way into the database. Sometimes it may be fixed the next day, or in other cases it may score pages incorrectly or poorly for quite some time.

About all you can do in this situation is to complain to the engine that "xyz" pages rank high on "xyz" search, when they really don't apply to the search. Most search engines will listen to your polite complaint and look into why the search results were so poor. If people don't find what they expect, they'll go off to another engine to do their searches. Search engines don't want that, because they make their money on advertising to those visitors and need them to return to their engine, rather than to their competitor.

Considering writing your letter from the perspective of the researcher doing a search and finding irrelevant results in their engine. Keep in mind the search engines strive to serve the average user, but they aren't in business to guarantee YOU any particular ranking.

6. The page is simply optimized: Often the reason a page ranks high is it simply fits the criteria that a search engine is looking for.

Their algorithms are fairly sophisticated, so sometimes it takes a second look and some background information to understand why a page is positioned where it is. Luckily, this is what the Page Critic does for you.