Optimizing your web properties' organic link structure for improved accessibility and search engine rankings
Case Study and Demo Report
Please note that all client data + links were anonymized to protect client's identity.
Client operates a major travel site focused on international hotel bookings.
The web site is administered via a major content management system and encompasses 100,786 dynamically generated web pages.
To accommodate client's traffic peaks, server load and bandwidth requirements, the spidering process was spread across 7 calendar days.
In the course of the crawl, appr. 25,000 undocumented pages were discovered of which client's web administration staff had not been aware of before.
This enabled client to trim and streamline their web property and cut down administrative overhead.
After finalization, the data files referred to in the report were made available to client in CSV format via a dedicated, password protected download link.
Following client's download, this sensitive data was removed from our online system for client's protection.
As a result of our analysis, client was issued a detailed report, for which please see anonymized sample below.
Client's real life domain name was replaced by the generic term "clientdomain.com" for the purpose of this demo report.
PAGERANK ANALYSIS + RECOMMENDATIONS (anonymized)
For domain: CLIENTDOMAIN.COM
Date: 2007-01-10
Introduction
PageRank is determined by iteration: Initially, every page will be assigned a provisional PageRank value of 1.
Based on this merely theoretical assumption, the iteration process proper will then be triggered.
It is important to bear in mind that the PageRank factor determined in this manner is not to be understood in any absolute sense!
Rather, PageRank merely constitutes a mathematical algorithm illustrating weighted relevancy between individual web pages exclusively in their relation to one another.
Some Terminology
For the purpose of this analysis, a "Navigation Menu Page" is a page (target URL) linked to internally within the confines of your Navigation Menu.
The Navigation Menu being included on several standard web pages, these "Navigation Menu Pages" will attain to a different PageRank value than other content pages.
By the same token, all pages not expressly linked to within the Navigation Menu will be referred to as "Standard Pages".
PageRank Calculation for CLIENTDOMAIN.COM (status quo)
PR Result
Home Page: 3394
Pages: 100,786
Sum Total: 74,805
Average PR: 0.74
These values will be discussed at greater length further below.
First, however, we will analysis different linking strategies to illustrate what these numeric values actually stand for.
Furthermore, for illustration's sake we are assuming a web site comprising a total of appr. 100,000 pages.
CENTRALIZED LINKING STRUCTURE SCENARIOS
The primary purpose of this introductory structure is to demonstrate some fundamental PageRank properties.
Centralized Linking Structure: Scenario A
Home Page links to all other pages
Here, we will be working from a highly simplified artificial (i. e. merely notional) structure within which it is only the Home Page that will link to all other pages on a domain.
PR Result
Home Page: 0.15
Other pages: 0.15
Pages: 100,000
Sum Total: 15,000
Average PR: 0.15
Bearing in mind that each page is assigned a provisional PR value of 1, the theoretical Maximum PR for a web site featuring 100,000 pages is: 100,000
Within the Scenario A discussed here, however, we can only achieve a PR total of 15,000.
Thus, such a structure will cause considerable loss of PR!
Centralized Linking Structure: Scenario B
All pages link to Home Page
This second (again: merely theoretical) structure constitutes the inverse of the linking structure A discussed above under a).
Here, however, PR is being clustered.
PR Result
Home Page: 12750
Other pages: 0.15
Pages: 100,000
Sum Total: 27750
Average PR: 0.28
Obviously, this structure will considerably reduce PR loss, with the Home Page gaining a very high PR value.
Centralized Linking Structure: Scenario C
Home Page links to all other pages, and all pages link to Home Page
This is essentially a combination of structures A and B.
PR Result
Home Page: 46027
Other pages: 0.54
Pages: 100,000
Sum Total: 100,150
Average PR: 1.00
This scenario illustrates the critical importance of reciprocal linkage: The average PR is at max value and the Home Page receives an even greater PR value than it can achieve under structure B.
Note: The Sum Total here is 100,150. Obviously, this is above the theoretical limit of 100,000. The explanation being that the calculation of PageRank is an iterative process, hence rounding errors constitute an unavoidable systemic feature.
Centralized Linking Structure: Scenario D
Home Page links to Navigation Menu Pages
Navigation Menu Pages link to Standard Pages
Navigation Menu Pages link back to Home Page
Standard Pages link back to Home Page.
PR Result
Home Page: 33048
Navigation Menu Pages: 2809
Standard Pages: 0.39
Pages: 100,001
Sum Total: 100,013
Average PR: 1.00
Within this scenario, both the Home Page and the Navigation Menu Pages will attain a high PR value.
Moreover, the average PR is again at maximum level.
Centralized Linking Structure: Scenario E
Home Page links to Navigation Menu Pages
Navigation Menu Pages link to Navigation Sub Menu Pages
Navigation Sub Menu Pages link to Standard Pages
Navigation Menu Pages link back to Home Page
Navigation Sub Menu Pages link back to Home Page
Standard Pages link back to Home Page.
PR Result
Home Page: 27735
Navigation Menu Pages: 2358
Navigation Sub Menu Pages: 182
Standard Pages: 0.30
Pages: 100,011
Sum Total: 100,012
Average PR: 1.00
By introducing Sub Menus, the Home Page's PageRank value is further reduced and distributed across the newly introduced Navigation Sub Menu Pages.
Centralized Linking Structure: Scenario F
Home Page links to Navigation Menu Pages,
Navigation Menu Pages link to Navigation Sub Menu Pages,
Navigation Sub Menu Pages link to Navigation Sub Sub Menu Pages,
Navigation Sub Sub Menu Pages link to Standard Pages,
Navigation Menu Pages link back to Home Page,
Navigation Sub Menu Pages link back to Home Page,
Navigation Sub Sub Menu Pages link back to Home Page,
Standard Pages link back to Home Page
PR Result
Home Page: 24683
Navigation Menu Pages: 2098
Navigation Sub Menu Pages: 162
Navigation Sub Sub Menu Pages: 12
Standard Pages: 0.26
Pages: 100,111
Sum Total: 100,111
Average PR: 1.00
This illustrates that every additional Menu Level will reduce the PR value of the previous level, which is instead distributed across the new level(s).
CIRCULAR LINKING STRUCTURE SCENARIO
Each Page to have one outgoing and one incoming link.
Here, there is no specific distinguishing between "Navigation Menu Pages" and "Standard Pages".
PR Result
Home Page: 1
Other Pages: 1
Pages: 100,000
Sum Total: 100,000
Average PR: 1.00
Note: This structure is only referenced for the sake of completeness.
In actual real life practice, it is hardly to be found, however.
CROSSED LINKING STRUCTURE SCENARIO
One page to have 4 outgoing and n incoming links
The pages to link to will be determined in a randomized manner
The value for "n" will be variable.
Here, there is no specific distinguishing between "Navigation Menu Pages" and "Standard Pages".
Note that individual pages' PR value varies between 0.15 through 10.00.
PR Result
Pages: 100,000
Sum Total: 99,969
Average PR: 1.00
One major asset of this linking structure is its inherent stability.
If the PR stream should be interrupted at some point in the overall chain, the effects will only be minimal.
The redundant networking ensures that PR will, instead, be distributed across the structure via alternate links.
Summary Analysis for CLIENTDOMAIN.COM
The current link architecture of the web site CLIENTDOMAIN.COM is commensurable with the Centralized Structure, including multiple menu levels.
This is defined by the products being featured: hotels, categorized by continents,
countries and cities.
At 3,394, the Home Page's PR is evaluated as: TOO LOW.
Average PR is 0.74, i. e. below the optimum of 1.00.
This is invariably caused by the overall PR flow being interrupted, e. g. if no backlinks
are implemented. The effect being that the Home Page's PR will suffer.
Recommendations for Improving the Web Site's PageRank Values
Recommendation #1
Reduce the number of pages featuring outgoing (off-site) links to a minimum.
Rationale:
Every outgoing link pointing to external domains will bleed PageRank away from the web site.
E. g. the Customer Service Link http://somelivecustomerservice.net/ is featured on every page, thus draining PR away to the third party domain somelivecustomerservice.net.
It is recommended to interpose a dedicated page here which links to the Customer Service Link proper.
This would drain PR from this single page only instead of affecting all others.
By the same token, the links below "Languages" are bleeding PR to the subdomains residing on CLIENTDOMAIN.COM.
This is so because third level or subdomains are viewed as alien external domains in terms of the PageRank algorithm.
Here, too, an interposed single internal page "Languages" could remediate the issue with a minimum of effort.
Recommendation #2
Include relevant content and keywords in all your Navigation Menu Pages.
Rationale:
These pages currently have a good PR but only very little content, effectively wasting their inherent PR leverage.
Recommendation #3
Generally, we recommend the Crossed Structure as it constitutes a slightly superior networking structure between site pages in terms of a search engine's Deep Crawl requirements.
(This is of special importance for deep crawlers such as Google, MSN/Live and Yahoo!)
E.g. within the Circular Structure, if the crawling process should be interrupted at any point in the link chain, end results may be skewed considerably.
(Crawling process interruptions can - and regularly will - occur due to a variety of factors ranging from timeouts due to server related issues, net traffic congestion, search engine load balancing parameters, bandwidth economy measures, etc.)
Likewise, the Centralized Structure offers multiple crawling chains (from Navigation Menu Pages to Standard Page and vice versa), any of which may experience interruption.
By contrast, the Crossed Structure is more robust and in better alignment with the search engine spiders' "organic" crawling process.
However, in view of the fact that your web site's product properties demand for a Centralized Structure, we further recommend implementing an additional Crossed Structure in this case.
This could, for example, be effected by implementing 4 randomly selected links to specific hotels on every page - this will effect a comprehensive cross linking of all the site's pages.
When conducting a search for your category pages on Google:
- CUSTOMIZED GOOGLE LINK HERE -
you will get zero results.
From this follows that your site's main nodes are not featured in the Google index.
This goes to illustrate how easily the linking chain can be broken. The effect being that the pages linked to the nodes in question are orphaned in terms of PageRank.
This situation can be remediated by implementing the additional Crossed Structure as recommended.
Recommendation #4
Determing which pages are featured in the Google index:
CUSTOMIZED GOOGLE LINK HERE
you will get "Results 1 - 26 of about 485,000", which, on the face of it, seems fine.
However, only 26 URLs are actually being listed.
Instead of displaying more or all, Google tells us:
"In order to show you the most relevant results, we have omitted some entries very similar to the 26 already displayed.
If you like, you can repeat the search with the omitted results included."
This implies that the remaining 484,974 pages stand an extremely low chance of ever being found in the search results.
The similarity referenced by Google in its message quoted above is furthermore compounded by the fact that these pages contain only a minimum of textual content.
Thus, it is recommended fitting all pages with adequate unique descriptive text for the search engines to evaluate them as featuring relevant, valuable and, hence, display-worthy content.
Note: The apparent discrepancy between the number of pages referenced by Google ("about 485,000") and the pages actually determined by our spiders (100,786) can easily be explained by a) Google counting multiple dynamic links as separate unique pages (after all, only appr. 40% of all pages featured in the Google index were actually spidered before being indexed - instead, their existence is determined by linkage only), and b) the inherent inaccuracy of the approximated value given by Google ("about"). This is a common and entirely normal phenomenon for web sites powered by content management systems.
Recommendation #5
The URLs should preferably contain relevant keywords.
Rationale:
This may improve ranking for the targeted keywords and will also impact searchers' click-through behavior positively.
Recommendation #6
After restructuring your web site and the URLs, it is recommended to generate a Google Sitemap and submit it to Google.
Rationale:
This will let Google know which pages to actually spider, typically leading to greater exposure of your site within the search engine's overall index.
Note that the Google Sitemap can and should be regularly updated and re-submitted whenever your site content is modified, new pages are added, etc.
|
|
|
|
Contributor's Note
Contrary to common perception, Google's PageRank(TM) algorithm does not apply to external inlinks - it relates to a web site's on-site links as well. This case study illustrates the importance of optimal on-site link structuring to ensure best exposure, spidering and, by inference, search engine rankings.
|
|
|
|
|
