Survey of Current Internet Search Facilities

Chip Rosenthal
Unicom Systems Development
<chip@unicom.com>
9 Apr 2002

Types of Search Utilities

• Search Engines
• Web Directories
• Virtual Libraries
• Meta-Search Tools

Types of Search Utilities: Search Engines

• Database of web pages.
• Searchable by keyword.
• Crawler based.
• Examples: Google <www.google.com>
• More in a moment ...

Types of Search Utilities: Web Directories

• Collection of web resources.
• Organized by topic category.
• Manually organized.
• Example: Yahoo! <www.yahoo.com>

Types of Search Utilities: Virtual Libraries

• Collection of web resources.
• Evaluated by subject professionals.
• Example: Librarians Index to the Internet <www.lii.org>
• Low demand for automation; powered by human intelligence.

Types of Search Utilities: Meta-Search Tools

• No local data.
• Queries other search tools.
• Example: Metacrawler <www.metacrawler.com>
• Inferior by design, can only rank by occurrence.
• Liked because they return fewer results, less intimidating.

Important Technologies

• Crawler. <http://www.robotstxt.org/wc/robots.html>
• Query interface, e.g. natural language, thesaurus.
• Ranking algorithm.
• Other features, e.g. document clustering, relevance feedback.

Search Engine Tour: Alta Vista

• www.altavista.com
• Founded in 1995 as a showcase for Digital Equipment Corp. technology.
• An initial favorite, due to (then) large database.
• Acquired from Compaq in 1999 by CMGI, and driven into the ground.
• Database size: 550M pages
• Database update problems. <http://news.com.com/news/0-1005-200-7630946.html>
• Rich query expressions, including wildcards

Search Engine Tour: Google

• www.google.com
• Founded in 1998.
• Privately held. <http://www.nytimes.com/2002/04/08/technology/ebusiness/08GOOG.html>
• Current favorite of most non-casual searchers. (1: MSN, 2: AOL, 3: Google)
• Largest database: 2000M pages
• Pagerank ranking algorithm incorporates a popularity metric.
• Recent meme: Google bombing: search for "talentless hack". <http://www.uber.nu/2001/04/06/>
• Directory is the Open Directory Project <http://dmoz.org/> with Pagerank applied.
• Supports relevance feedback ("more pages like this").
• Biggest drawback: no truncation or wildcards.

Search Engine Tour: Ask Jeeves

• www.ask.com
• Not really a search engine.
• Directory organized into a Q&A knowledge base.
• Natural language query expressions.

Search Engine Tour: AllTheWeb

• www.alltheweb.com
• Founded in 1997 in Norway.
• Database size: 625M pages, second to Google.

Search Engine Tour: Northern Light

• www.northernlight.com
• Founded in 1995.
• Groups documents dynamically into search folders.
• Database size: 309M pages.

Search Engine Tour: Inktomi

• www.inktomi.com
• Founded in 1996.
• Provides search technology to companies and web sites.
• Database size: 500M pages

Search Engine Tour: Overture

• www.overture.com
• Founded in 1997 as goto.com.
• Paid placement in search results. 53,000 advertisers.

Search Engine Tour: Teoma

• www.teoma.com
• Introduced April 2002. Operated by Ask Jeeves.
• New algorithm establishes "communities" about a query topic.
• Offers query refinement features.
• Biggest drawback: tiny database (200M).

BACK: Paperware Archive | Unicom Systems

$Id: SearchSurvey2002.html,v 1.2 2002/04/10 03:31:43 chip Exp $