<story><title>Web scraping: Reliably and efficiently pull data from pages that don&apos;t expect it</title><url>http://pyvideo.org/video/609/web-scraping-reliably-and-efficiently-pull-data</url><text></text></story><parent_chain></parent_chain><comment><author>epoxyhockey</author><text>My last web scraping activity consisted of using PhantomJS to drive and injecting my own javascript that submitted the pertinent data to my own web service.  &lt;a href=&quot;http://phantomjs.org/&quot; rel=&quot;nofollow&quot;&gt;http://phantomjs.org/&lt;/a&gt;</text></comment>
<story><title>Web scraping: Reliably and efficiently pull data from pages that don&apos;t expect it</title><url>http://pyvideo.org/video/609/web-scraping-reliably-and-efficiently-pull-data</url><text></text></story><parent_chain></parent_chain><comment><author>AdrianRossouw</author><text>i found node.js is a phenomenal scraping tool. There&apos;s also a pretty simple / easy framework for these tasks called node.io (&lt;a href=&quot;http://node.io/&quot; rel=&quot;nofollow&quot;&gt;http://node.io/&lt;/a&gt;).&lt;p&gt;Being able to pop up a jsdom, and extract data from the page using jquery is a lot of fun.</text></comment>