How Web 1.0 fails: the mismatch between hyperlinks and clickstreams

The core of the Web is a hyperlink navigation system collaboratively set up by webmasters to help users find desired information. While it is well known that search engines are important for navigation, the extent to which search has led to a mismatch between hyperlinks and the pathways that users actually take has not been quantified. By applying network science to publicly available hyperlink and clickstream data for approximately 1,000 of the top Web sites, we show that the mismatch between hyperlinks and clickstreams is indeed substantial. We demonstrate that this mismatch has arisen because webmasters attempt to build a global virtual world without geographical or cultural boundaries, but users in fact prefer to navigate within more fragmented, language-based groups of Web sites. We call this type of behavior “preferential navigation” and find that it is driven by “local” search engines.