Web Searching, Sleuthing and Sifting: Lesson Six: Using Meta-Search Engines

Back to the
Cyberlibrarian's Rest Stop

Web Searching, Sleuthing and Sifting

Lesson Six:
Using Meta-Search Engines

What is a Meta-Search engine?

A meta-search, multi-threaded or multi-search engine searches other search engines' databases in parallel (at the same time) and collates the results (which usually consist of a predetermined maximum "number" such as 10, 20 or 30). The search engine does not maintain its own database. Major differences in the meta-search engines include:

Selection of databases searches
Submission of queries to the search engines
Manipulation and presentation of results
Response time

Selection of Databases Searched

Most meta-search engines will search several of the "major" search engines at once, usually: Altavista, Excite, Lycos, Webcrawler, Yahoo and Hotbot. Sometimes because of an agreement with one search engine, the meta-search service will not search another, specified, search engine. A number of directories such as Looksmart, Galaxy, and/or smaller search engines and specialized databases such as Deja News (newsgroups), stock quotes, or news wires may also be included.

Sometimes this is wonderful -- but at other times a lot false hits from irrelevant sources can create a bad search set. This is why the ability to customize is so important -- being able to choose which search engines your query is sent to. But this also requires some experience on the part of the user to be able to determine which search engines or databases to include in the search.

How Queries are Submitted to the Search Engines

Queries are submitted to each search engine at once (in parallel) or one or several at a time. I tend to prefer the "simultaneous" search myself -- I mainly use meta-search engines because I want to see what the search engines find all at once (some meta-search services ask if you would like to "Search X more search engines" instead of sending all queries at once).

Remember how each search engine uses different search syntax and has different advanced features such as Boolean, field and proximity searching? Well here's one of the main problems of meta-search services -- unless specifically stated otherwise, the advanced techniques will be filtered out. This is because the interface of the meta-search engine must be programmed to interpret your search syntax for each and every search engine it queries. As we all know, search techniques frequently change. For the most part, however, simple Boolean (and, or, not) are honored as well as implied Boolean (+ meaning "and" or "require" and - "exclude").

Manipulation and Presentation of Results

Depending on the kind of research I am doing, I may wish to see which search engine retrieved which hit, or results grouped by web site, or results ranked according to relevance and almost always to have the duplicates removed. Unfortunately, all of these features are not available through all meta-search utilities, so you have to choose which is most important to you (and this often varies with the search....). Most annoying, is when relevancy rankings are stripped.

If Meta-Searchers are so great and search my favorite search engines simultaneously, then why wouldn't I always use one?

Well.....again, it depends on what you are looking for...consider

Meta-searchers, for the most part, strip the advanced search syntax and logic from your queries when they submit them to the search engines, so search statements with complex structures are not good candidates for these tools.
you will only see a few results from each of the search engines, which is also limiting.
the presentation of the results may be confusing.
sometimes the time it takes to conduct a search using a meta-search engine may be too long
sometimes relevancy rankings are removed from the listings

When would I use one?

Use a meta-search engine when:

when you can't find anything when you've queried one or two "major" search engines
when your topic is "obscure"
when you want to see the top hits (most relevant selections) from several databases at once
when you want to search a variety of sources on the same topic simultaneously
when you want to compare the indexing of several search engines (perhaps to pick one to search by itself)
when you know the web "literature" on the topic but want to see if you missed something
when you want to search several utilities at once and have all the results ranked by relevance (some engines will do this)
when you want to search using one interface and method of presentation

My favorites are:

Ask Jeeves, Dogpile, Inference Find, Metacrawler, Savvy Search

Meta-Search Engines

Ask Jeeves

(http://www.askjeeves.com)

Cyber411

(http://cyber411.com/)

Dogpile

(http://www.dogpile.com/)

Sends queries to a customizible list of up to 25 search engines including: Yahoo!, Lycos' A2Z, Excite Guide, Go2.com, HotBot, PlanetSearch, Reference.com, Thunderstone, What U Seek, Magellan, Lycos, WebCrawler, InfoSeek, Excite & AltaVista. Also, Newswires, FTP and Usenet.

Highway 61

(http://www.highway61.com/)

Inference Find

(http://www.inference.com/infind/)

Ixquick.com

(http://ixquick.com/)

The BigHub

(http://www.thebighub.com/)

1500

Yahoo, GoTo.com, Infoseek, WebCrawler, AltaVista, Excite and Lycos

Mamma

(http://www.mamma.com/)

MetaFind

(http://www.metafind.com/)

MetaCrawler

(http://www.metacrawler.com/)

Sends search queries to several search engines including: Lycos, Infoseek, WebCrawler, Excite, Alta Vista and Yahoo, then normalizes and ranks results. Of special interest: using Power Search, allows user to limit by continents and by U.S. educational, commercial or government sites.

ProFusion

(http://www.profusion.com/)

SavvySearch

(http://www.savvysearch.com)

Queries several specialized search engines (such as shareware.com, Internet Movie Database) as well as web indexes (such as Alta Vista, Lycos) and web directories (such as Yahoo and EINET Galaxy). User can specify/limit by source (Web Indexes, People, Entertainment, Software) and by "type" of information (images, technical reports, academic).

Learn more....

Guide to Meta Search Engines

(http://www.indiana.edu/~librcsd/search/meta.html)

Multi-Search Tools (from the CyberStack index)

(http://home.revealed.net/albee/#Multi)

Assignments

The purpose of the assignment this week is to compare results across meta-search tools. Finding "the" answer to the question is not as important as comparing the search set results. Use a selection of the above tools. Post your observations to the class bulletin board and please keep those subject lines descriptive!

1. Search using my name (Simone Leroy).

2. Search using your name.

3. Search for a general topic (such as "library science").

4. Search for a very specific topic (such as "dublin core" "metadata" and "collections").

5. Find a meta-search tool not included in the above list.
6. What are good search topics for meta-searchers? (generally..) Post one or two suggestions and tell us why they would be good candidates for a meta-search vs. single search engine.

Back to the
Cyberlibrarian's Rest Stop