Students Go Beyond Google, Into the Deep Web

When you use Google, Microsoft, Yahoo! and other major search engines you get millions of results that all have one thing in common: the websites have all allowed themselves to be listed there. A web developer can choose to put a bit of code in their web pages that instructs Google, et. al. to not search the site. The major search engines respect this request and ignore these pages. However, there are a few search engines that focus on ignoring these instructions, choosing to index the site anyway. They also focus on sites that the major search engines aren’t able to reach because the information resides in databases. These search engines capture what is known as the Deep Web (or Invisible Web, Hidden Web, etc.) and there can be a wealth of information on these websites that can be beneficial to you, as a student. Three resources are listed below, but it’s not hard to find that there are hundreds of deep web search engines available, many subject-specific. For a more in depth look at the Deep Web, read The Invisible Web: Uncovering Information Sources Search Engines Can’t See by Gary Price and Chris Sherman.

CompletePlanet – One of my favorites, CompletePlanet, searches more than 70,000 databases and other specialty search engines. You narrow your search by subject and then tell it what you are looking for. For instance, choose the subject Education and sub-category Aids & Grants. Now enter the search term “Scholarship” and you get a list of 555 results that could give you an advantage over other applicants looking for scholarships.

Intute – The results you find here were hand selected by subject matter experts and were chosen for the purpose of education and research. The website is divided into four major categories: Science & Technology, Arts & Humanities, Social Sciences and Health & Life Sciences.

The Internet Archive– A database with thousands of photos, movies, music, audio, and scanned documents, this site is a great resource for media to add to your presentations and research projects. The Internet Archive also hosts the Wayback Machine, which allows you to view old revisions of more than 55 billion web pages.