Lesson 1

The early days of the World Wide Web were characterized by rapid growth and the challenge of information discovery. Before sophisticated algorithms, the internet lacked a reliable navigation map. Early search engines and directories relied primarily on simple methods that quickly exposed their fundamental limitations.

In the mid-1990s, the primary tools for finding information were human-curated web directories and search engines based on simple keyword matching. Services like Yahoo! Directory were organized taxonomically, requiring human editors to categorize websites. While robust for general browsing, this approach was non-scalable and inherently biased by editorial judgment.

Search engines like AltaVista moved beyond directories by indexing the full text of web pages. Their ranking algorithms were primarily based on the frequency and location of keywords within a document. A page containing the search term ten times was generally ranked higher than a page containing it five times.

The simplicity of early ranking metrics led to two major, critical flaws:

• Relevance vs. Importance.  Keyword matching conflates relevance (how much a page talks about a topic) with importance (how trustworthy or authoritative the page is). A page filled with repetitive keywords (a technique known as keyword stuffing) could easily outrank a high-quality, authoritative source.


• Vulnerability to Manipulation.  Because ranking relied purely on on-page content, black-hat SEO techniques could easily game the system, resulting in low-quality or spam websites appearing at the top of search results. This compromised user experience and reduced trust in the search results.

These limitations created a strong demand for a ranking mechanism that could measure a website's intrinsic quality or authority, independent of its content. This need paved the way for the development of PageRank.

PageRank, an algorithm developed by Larry Page and Sergey Brin at Stanford University, fundamentally shifted search engine philosophy. Instead of relying solely on a page's content, PageRank measures the quality and quantity of links pointing to a page, treating the entire World Wide Web as a massive graph.

The algorithm is based on two key principles, often explained through the analogy of a random web surfer.

• Links as Votes.  A hyperlink from one page to another one is interpreted as a vote of confidence or endorsement. The more votes a page receives, the more important it is.


• Importance of the Voter.  Crucially, not all votes are equal. A vote from an important or authoritative page (one with a high PageRank itself) counts for more than a vote from an unimportant page.