Internet Research - Search Tools

Search Tools

The most popular search tools for finding information on the internet include Web search engines, metasearch engines, Web directories, and specialty search services. A Web search engine uses software known as a Web crawler to follow the hyperlinks connecting the pages on the World Wide Web. The information on these Web pages is indexed and stored by the search engine. To access this information, a user enters keywords in a search form and the search engine queries its indices to find Web pages that contain the keywords and displays them in search engine results page (SERP). The SERP list typically includes hyperlinks and brief descriptions of the content found. Search results are ranked using complex algorithms, which take into consideration the location and frequency of keywords on a Web page, along with the quality and number of external hyperlinks pointing at the Web page.

A Metasearch engine enables users to enter a search query once and have it run against multiple search engines simultaneously, creating a list of aggregated search results. Since no single search engine covers the entire web, a metasearch engine can produce a more comprehensive search of the web. Most metasearch engines automatically eliminate duplicate search results. However, metasearch engines have a significant limitation because the most popular search engines, such as Google, are not included because of legal restrictions.

A Web directory organizes subjects in a hierarchical fashion that lets users investigate the breadth of a specific topic and drill down to find relevant links and content. Web directories can be assembled automatically by algorithms or handcrafted. Human-edited Web directories have the distinct advantage of higher quality and reliability, while those produced by algorithms can offer more comprehensive coverage. The scope of Web directories are generally broad, such as DMOZ, Yahoo! and The WWW Virtual Library, covering a wide range of subjects, while others focus on specific topics.

Specialty search tools enable users to find information that conventional search engines and metasearch engines cannot access because the content is stored in databases. In fact, the vast majority of information on the web is stored in databases that require users to go to a specific site and access it through a search form. Often, the content is generated dynamically. As a consequence, Web crawlers are unable to index this information. In a sense, this content is "hidden" from search engines, leading to the term invisible or deep Web. Specialty search tools have evolved to provide users with the means to quickly and easily find deep Web content. These specialty tools rely on advanced bot and intelligent agent technologies to search the deep Web and automatically generate specialty Web directories, such as the Virtual Private Library.