|
|
|
Web Crawler |
Web crawler (also known as web spider) is a program which browses the World Wide Web in a methodical, automated manner. A web crawler is one type of bot. Web crawlers not only keep a copy of all the visited pages for later processing - for example by a Search Engine but also Index these pages to make the Search narrower. Comments: | | | | |