Definitions for "DEEP WEB"
(see invisible Web)
Also known as 'invisible web'. Refers to a vast repository of underlying content, such as documents in online databases, that general-purpose web crawlers cannot reach. Both qualitative and quantitative in difference, deep web content is estimated at 500 times that of the surface web, yet has remained mostly untapped due to the limitations of traditional search engines. (
Related Terms: scraper, spider, dynamic content, web page form A way to more deeply spider a web site, beyond simply following links. For example, a deep web spider often fills in web page forms with a range of terms and can submit the form repeatedly, and then capturing the various results. Many simple web spiders will miss content that is only accessible by searching with forms.
Consists of materials that are available by HTTP and are publicly available, but are not included in standard public indexes such as Google. This includes materials that are difficult or impossible to crawl, such as databases.