Search engines and a cool idea from 1995

This Tech Talk column originally appeared in The Exponent (University of Alabama in Huntsville), Vol. 33, No. 20 (February 14, 2002). Digitized issue: UAH LOUIS archive. Reproduced here courtesy of that archive.

Back in 1995, an acquaintance who worked at EDS pointed me to an interesting website where visitors were presented with a simple form. You could type in a few words, and away it would go, searching the entire web for pages containing those words. These days, search engines are commonplace, but AltaVista represented a cool new idea back then and was an interesting way for DEC to showcase its new Alpha processor.

Although cool, AltaVista wasn’t perfect. I usually found myself sifting through at least three to five pages of results to find the page I wanted. Worse, AltaVista sold keyword placement where advertisers could pay to have their links appear first in the results when users searched for certain words. Those were simpler times, and the idea of being able to query the whole web was still dazzling. Like the typical MS Windows user, I didn’t know to expect better.

A few years later, I was on the way to Carnegie Mellon University to speak at yapc99—yapc stands for “yet another Perl conference”—and picked up a copy of Scientific American to pass the time on the plane ride. That particular issue had an article about a new search engine called Google that attempted to solve the relevance problem (i.e., making pages relevant to the query appear early in the results list) with a system known as PageRank™. According to Google’s website, “Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves ‘important’ weigh more heavily and help to make other pages ‘important.’” I take it for granted now, but it was uncanny how frequently Google’s results were right on. I was immediately impressed with the cool technology and the cool company surrounding it (and the cool millions that employees will likely make when they IPO).

Google continues to maintain that reputation. They recently announced the “First Annual Google Programming Contest” (http://www.google.com/programming-contest/) that encourages hackers to submit implementations of interesting new applications of Google’s technology. The winner may get to run code against Google’s mega-cache and, by the way, will be the proud recipient of a check for ten grand. Cynics have accused the company of attempting to profit from cheap labor, but this contest could bear delicious fruit! With all the people out there thinking, “You know, it’d be really cool if…,” I’m anxious to see the winning entry. (If you happen to have the idea for a winning entry in mind, let’s talk.)

In this column, there shall be no unmitigated gushing about the wonders of any technology, no matter how exciting. We must admit that even Google has its limitations. In a moment of resignation, a computer scientist named Jutta Degener wrote, “Once the WebCrawler has visited your essay about the naked incomprehension greeting a women’s group’s demand for censorship of pornographic newsgroups, the vast majority of its visitors will be searching for ‘porn,’ ‘naked,’ and ‘women.’”