What is a web crawler, really?

by Search Off the Record