Access to the Corpus - What Would You Do?
caffeine: So .. Google have these data.  Their hacking chops are not unremarkable.  What have they done with it?  Well, basically, you can ask for a word and they'll find other pages that feature that word ...And that's it.They have access to "the world's combined knowledge" and a zillion PhDs, and that's all they can do?!  It's shocking.  But it's not, really, because the data are basically useless without annotations.So let's go to Disneyland and pretend we have a genius NLP engine or an annotated web.  Then,1) 360 on a company/product.  In particular: who are all the stakeholders, and how do they feel? I'd sell this to e.g. analysts.  Same thing on people's online identities.2) Memetracing.  I'd sell this to advertisers. (So, we follow historical product releases and see exactly what memes spread about them and how, and through who.  Related to (3) on ismarc's post)3) Rumors (this would require your feed to be real-time).  I'd also probably sell this to stock traders (the idea here is to monitor e.g. forums frequented by GE employees to guess scoop)OK, so those are basic.  More interesting:4) Organization-tracing.  If you can label social graph edges with influence levels / information intakes, you can start playing with predicting organizational decision-making (behavioral economics / game theory ..There's a TED talk about this).5) Games: procedural content generation that looks really real, i.e. worlds full of people whose identities are plausible, whose interactions with others are plausible, etc.Those all require some analytical / NLP firepower on a scale which I don't think is really doable at the moment.   The problem is that bags of words are meaningless without a social context - the data are pretty worthless unless your computer can figure out who it's important for and why.