Q:

If you want to implement a XSS scanner, is it absolutely necessary to use a JS interpreter? Why?

In a recent discussion about a security vulnerability scanner that returns false positives for XSS detection, I noticed that the scanner just inject a string like "this_is_my_string_" (without the double quotes) and if it sees the string in the HTML response it says that there exist an XSS.
Talking with the author of the scanner, I asked him how it is possible to say that there exist an XSS just inserting that kind of string without < > ' " or any special char (just _). He says that as the scanner does not still include a JS interpreter and the detection will always be unreliable and manual confirmation will be needed.
Is it absolutely necessary to use a JS interpreter to detect automatically XSS vulnerabilities?
Will it be very unreliable to insert the called XSS locator 2 of the OWASP: 
'';!--"<XSS>=&{()}

and look for special chars not being encoded in the response? Why?

A:

Reliably detecting Cross-Site Scripting is a relatively complex task, just inserting a string with no control characters and looking for it in the response, is a very bad idea as you'll be swamped by false positives.
What most scanners to is take a series of standard vectors (e.g. ">< script >alert(1)< /script ><") and then look at the response from that string unencoded.
Even then this is not perfect as it depends on the context of the response as to whether it's exploitable.
A better way of handling this is to actually look at the DOM of the resulting page and look at the JavaScript executed on page load to see if your vector has been included in an executable location.  This obviously requires an HTML rendering engine and a JavaScript rendering engine.
Also there's the problem of following links if the scanner includes a spider to find content.  Without a JS interpreter the spider will likely miss links that can only be followed by evaluating JavaScript, of which there are a lot these days.
So the answer would be that a JS interpreter isn't an absolute requirement but it will help a lot.

