Anyone here do Web mining/harvesting/scraping?
lazy_nerd: I would recommend hosted crawling/scrapping solutions where you don't need to dedicate your own hardware resources. Also monitoring and maintaining the scrappers/harvesters can be a pain.
Shameless Plug: I work @ beevolve (www.beevolve.com) and we provide hosted crawling and scrapping solutions. PM me if you are interested.