a generalized term for an indexer that searches and catalogs sites using the links in hypertext documents
an application that crawls the Web and collects resources (Web pages, images or other data)
an automated search engine, when someone submits a resource to the search engine, it sends a small, but powerful program back to the site of the submission which scans the site for more resources
a program designed to search and, in many cases, catalogue the Web
a program that automatically downloads pages from the Web
a program that browses the worldwide web in a typically arranged, ordered and mechanized manner
a program that crawls through pages on a web site
a program that follows links on web pages on its own
a program that retrieves and stores pages from the Web, commonly for a Web search engine, and its effectiveness has a significant impact on the quality of search results
a program which automatically traverses the web by downloading
a program which browses the World Wide Web in a methodical, automated manner
a software robot that goes through the web and takes snap shots of site, then index's them into their directory
a tool that will automatically find web-sites on the Internet, via hyperlinks to them
A Web crawler searches through all the Web servers it knows to find information about a particular topic requested by the user.
Web Crawler is one of the most popular search engines on the World Wide Web.
a search engine that searches the Web by document title and content, archiving the information for searching purposes.
A web crawler is an automated program that accesses a web site and traverses through the site by following the links present on the pages. Known as a bot, robot, spider or Web Crawler.
A Web search engine or "index", it is searchable by entering the words you want to look for, and brings up the URLs that have those words associated with them. The site URL is: http://www.webcrawler.com
A small program that follows links from one Web page to another and records information about Web sites in a search engine database.
A computer program which starts at a single Web document, and follows the hotlinks from there to slowly retrieve and map the entire contents of the Web. The rate of document change on the Web is making this method of cataloging the Web more and more impractical. The terms Web robot and Web spider are equivalent.
See Main Definition: spider
A software program that systematically traverses the Web unattended, either for the purpose of generating a searchable index of Web content or to gather statistics.
An automated program that can find and retrieve web pages for search engine indexes. Web crawlers are also called spiders.
A web indexing program that builds an index by following hyperlinks continuously from web page to web page.
A program which visits remote sites and automatically downloads their contents for indexing.Web crawlers are part of the effort to organize the millions of documents on the World Wide Web.
WebCrawler® is a metasearch engine that blends the top search results from Google, Yahoo!, Windows Live Search (formerly MSN Search), Ask.com, About.com, MIVA, Look Smart and other popular search engines. WebCrawler also provides users the option to search for images, audio, video, news, yellow pages and white pages. WebCrawler is a registered trademark of InfoSpace, Inc.