Any open source alternative to element selection on a webpage within a page?
Scriptor: I know exactly what you're talking about and have done a similar thing in one of my own projects. Basically, you need an iframe to contain the HTML, and fetch the actual page through the server. If you load the page directly though the iframe, Javascript will not be able to manipulate its contents.I've posted a Pastie of the Javascript code here: http://pastie.org/369649. It does use jQuery though. The w variable points to the window object of the external page that's loaded, so you can pass it around to other functions. Note that 'pageframe' is the ID of the iframe element.As for the server side code, I used Django, with Python's urllib2 to load the page and BeautifulSoup to turn relative urls into absolute urls. If you end up using BeautifulSoup, note that even though it's supposed to handle invalid HTML, there are some pages that still won't work because Python's HTMLParser library isn't robust enough.