Spider Guts - The Core Elements for Page Relevance

Share |

There are a number of things that a spider expects to see when it looks at a web page, many of which are optional but still important in the big picture. The following is a list that describes the core elements considered by a typical search engine when calculating page relevance.

1. Title Tag - The title tag should contain a title relevant to the page, not just "Home Page" or "Contact Us". The title should be used for up to five keywords.
2. Headings - Search engines view <h> tags as terms of emphasis, meaning additional weight is given to terms that appear inside them. Keywords should appear in <h> tags.
3. Bold - Also viewed as terms of emphasis, but with less weight than headings.
4. Alt Text - Brief descriptive sentences should be used in image alt attributes. At least one keyword should appear in each alt attribute.
5. Keyword Meta Tag - Some engines use the keyword meta tags directly, some use them as part of a validation process ensuring that the keywords closely match the page content. The latter is the more typical scenario for modern engines. Keywords should be chosen carefully and be specific to the page they appear on.
6. Description Meta Tag - Most search engines use this tag in a similar fashion as the keyword tags. Each page should have a unique description. The description should contain a few keywords and briefly summarize the content that appears on the page with a high degree of accuracy.
7. Keyword Placement - Terms that are higher up on a page are more heavily weighted.
8. Keyword Proximity - Terms that are close together are probably related, and thus the site will show up in searches for those terms.
9. Comment Tags - Some search engines use comment tags for content, particularly in graphics rich/text poor sites.
10. Page Structure Validation - Proper coding is likely to be of better overall quality, and thus rewarded.
11. Traffic/Visitors - Search engines keep track of how many people follow their links. The more a link is followed for a given search, the more relevant the link is assumed to be.
12. Link Popularity - Also known as PageRank, this is a measure of how many web pages on the Internet link to your site and the relevance of those pages to the page they are linking to. The popularity of the linking site is also evaluated.
13. Anchor Text for Inbound Links - This is a measure of the relevance of the anchor text from the referring site.
14. Page Last Modified - Newer content is regarded as "fresh" and is treated as more relevant.
15. Page Size - Engines tend to weigh content at the start of a document more than content further down. If a page is too long, typically more than 50k in markup only, then it should be broken up into multiple pages.
16. Keywords in URL - URLs are considered important by engines. Use of hyphens rather than underscores in filenames and using keywords in filenames and directories improves a pages potential relevance.

These elements are all poured into an algorithm by the search engine that produces a very specific result: a relevance score for a page based on a given set of keywords. Evaluating page relevance is a constant reciprocal process that involves crawling around all pages indexed by a particular engine and evaluating the relevance of their content and the relevance of references to that content. The items listed above are things search engines expect to find in a page as well as factors that are not necessarily expected, but are considered if available (such as inbound links).