I think this might be extremely useful addition and will help Arquillian to become the ultimate testing weapon

I remember we were discussing this a bit with Aslak during Devoxx. Just after I came back home I made a "memory dump" to have sort of a reference for the ideas, so here it is (rough cut):

Since it's quite straightforward to embedded any Javascript on tested website through WebDriver (vide my screencasting stuff or John's jQuery selectors for WebDriver which we implemented during Devoxx) it opens some interesting possibilities:

Executing Javascript tests in addition to UI tests which we already have. In general we could put container abstraction one level above and treat a browser as "target of deployments / deployable container". I think it's extremely appealing idea. So I was thinking about:

Code coverage

Javascript code coverage - we can try to collect code coverage for javascript tests which we are going to run using Ike. There are already some tools which we can look at for our stuff (JS Coverage 1JS Coverage 2Jasmine JUnit Runner).

Cross-layers coverage - since we already can exercise system under test from the user standpoint (WebDriver / Selenium / Drone) and will be able to run javascript tests too I was thinking about having cross-layer code coverage. In brief we could measure code coverage on the server side when testing from outside (either through UI or pure JS calls like using jQuery to access REST resources for instance).

Code coverage solution can actually evolve to something more. (WARNING, this is most likely a CRAZY idea). How about something like "UI coverage"? We can use WebDriver to instrument the website and observe on all "action components" while executing tests. Using such technique will give us a possibility to verify which UI components are covered by tests, but what's more interesting it might be also visualized to identify which regions are heavily used in the UI for features which our application is serving the end-user (something like a heat map? - I remember I saw something like that in the report from Google explaining which areas of the search result page user is mainly reading). This might be actually used as some sort of metric and possibly help for identifying usability anti-patterns. For instance it might be an indicator of how much mouse movement is involved to accomplish given functionality.

Having WebDriver "instrumentation/interception" we can also produce page flows for given functionality which is subject of the UI test => we can detect which web sites are not reachable, so either not used at all or not covered by functional tests.

I really like the idea of integrating another test framework instread of developing the wheel once again. As for the integration with Drone, I expect that support for JSTD could be added relatively easily and user can benefit from having browser started and captured for free.

The only tough point I see so far is how the browser auto{start,capture} features will be compatible with IDE support for JSTD, I have to investigate that.

During weekend I have took closer look to JS-Test-Driver and I can confirm now that we can initialize HTML code from external file using test initialization code.

I tried running Sizzle [1] tests with my configuration and some of them are ok, but rest of them fails on uncomplete implementation of QUnitTestAdaptor.js which is responsible for wrapping JSTD API with QUnit API adapter. However fine-tuning that adapter should help to run existing QUnit test suites with JSTD.

I like the idea of integrating javascript on the platform, first time you spoke about it though, I was thinking more about server-side javascript. Which could be a whole new branched feature for arquillian.

I agree with all the "whys" and "whats" and therefore I think that a javascript extension would be great addon.

As I understood, are you doing to integrate JSTD in arquillian to make it use drone extension?

2. When step one is done , a destination directory containing all the instrumented js files will be created.Particularly there is a main generated file by jscoverage called jscoverage.html , inorder to run the html file a simple server like Apache or simpleHttpServer is needed and it has to be loaded first as it is the main window in which we enter the user defined files like index.html to measure the code coverage , measuring the code coverage etc.

3.There is a mode called Inverted Mode in which user defined file like index.html(which need to be tested) can be directly loaded with out first loading the jscoverage.html file.

The approach:

Using this inverted mode we can achieve the integration with jstd by using QunitTestRunnerPlugin.

It can be done as follows.

1. Lets assume we have one index.html file in hand (which contains many js files in it linked) and we need to run it for measuring the code coverage.

2.Even though js Test Driver has no direct support for loading .html files and it can be done as discussed previously how html files can be loaded using jstd.

3.For measuring the code coverage we need to launch the jscoverage.html window and for achieving this we can attach a window.open(../main/jstestcoverage.html) event to a button (named like code coverage ) in the user defined index.html itself.The coverage reports can be captured by just clicking the code coverage button and it can be achieved for jstest driver using htmlDoc feature .

The sample reports generated for jquery suite coverage are attached

The main point is that we are able to run large test suites in js Test Driver like jquery using QunitTestRunner plugin and .html files can also be loaded for jstd , so jscodecoverage can be done by just running an user defined index.html file under js test Driver and capturing the code coverage statistics.