Search engines rank web pages according to the software’s understanding of the web page’s relevancy.
To determine relevancy, each search engine follows its group of rules. The most important rules are
- The location of keywords on your web page; and
- How often those keywords appear on the page (the frequency)
For example, if the keyword appears in the title of the page, then it would be considered to be far more relevant than the keyword appearing in the text at the bottom of the page.
Search engines find keywords to be more useful if they appear sooner on the page (like in the headline) rather than later. The idea is that you’ll be putting the most important words – the ones that have the relevant information – on the page first.
Search engines also consider the frequency with which keywords appear. The rate is usually determined by how often the keywords used out of all the words on a page. If the keyword is used four times out of 100 words, the frequency would be 4%.
Of course, you can now develop the entire relevant page with one keyword at 100% rate – just put a single word on the page and make it the title of the page as well. Unfortunately, the search engines don’t make things that simple.
While all search engines do follow the same ground rules of relevancy, location, and frequency, each search engine has its particular way of determining rankings.
To make things more interesting, the search engines change the rules from time to time so that the rankings change even if the web pages have remained the same.
One method of determining relevancy used by some search engines (like HotBot and Infoseek), but not others (like Lycos), is the Meta tags. Meta tags hide HTML codes that provide the search engine spiders with potentially relevant information like the page description and the page keywords.
Meta tags often labeled as the secret to getting high rankings, but Meta tags alone will not get you a top 10 ranking. On the other hand, they certainly don’t hurt.
In the early days of the web, webmasters would repeat a keyword hundreds of times in the Meta tags and then add it hundreds of times to the text on the web page by making it the same color as the background. However, now, major search engines have algorithms that may exclude a page from ranking if it has resorted to “keyword spamming”; in fact, some search engines will downgrade ranking in such cases and penalize the page.
Link analysis and ‘click through’ measurement are certainly other factors that are “off the page” and yet crucial in the ranking mechanism adopted by some leading search engines. It is quickly emerging as the most important determinant of classification, but before we study this, we must first look at the most popular search engines and then look at the various steps you can take to improve your success at each of the stages – spidering, indexing and ranking.