Legality of Web Scraping User-Submitted Content?
sh1mmer: My "not a lawyer" answer to this is:It depends on the terms of service of the site.#1 The TOS of the site may not let you use a robot on their site at all#2 The TOS will define who owns the user generated content (UGC), either the user or the site#3 Depending who owns the UGC you may or may not be able to scrape it, if it's the site it's against their TOS if it's the user you would need permission from the user.#4 As other people have said fair use might come into play. If the site owns the material using a single user contribution might be fair use within the context of the whole site. If the users own the content you are likely to be using all their content, thereby not able to use fair use.Again all of these are my observations. Hopefully it will give you something to think about. If you are starting a business based on this, you do need to consult a lawyer. Also starting a business based on page scraping is a pretty risky thing to do. If the scraped site turn you off you could be pretty screwed.