What's your philosophy on page scraping?
rmanocha: I think that if you're generally respectful of the target websites - scraping them is ok. For example, I scrape various government websites for my website. I use a random delay between requests and am generally very careful about not requesting the same page multiple times (this is hard 'cause a lot of the pagination happening on these pages is via JS calls).I am ok if someone decides to scrape my websites in a similar fashion - although if I do see that starting to happen, I'd rather just go ahead and build an API.