there are several aspects to the present invention . using the present invention , the professional neuro - psychologist will be able to manage a large number of clients on a professional basis and to utilize a system and a method to influence creation of batteries of tests and evaluation parameters . this will provide the means to evaluate a client using up - to - date instruments . monitoring clients or a specific condition in a client , through a battery of tests , will be done using a procedure , which will enable direct and ongoing contact between client and clinician , as necessary . a substantial portion of the tests can be administered , evaluated , and monitored automatically . ongoing results fed back on an on - line basis will be evaluated to determine the need for further testing and the nature of such testing . this automated process further provides the neuro - psychology professional with new parameters and evaluative metrics for determining and assessing a neuro - psychological condition in a client . furthermore , maintaining and improving a professional neuro - psychologist &# 39 ; s ability to manage a large client base with an appropriate level of professional service , is an essential factor of this invention . each client requiring testing , will have the benefits of paying for actual services received , having access to a multiplicity of tests , have access to all appropriate professionals and a reduced burden of testing . however , it must be stated , that it is a specific desire of the neuro - psychology professional , to subject clients to as little testing as possible whilst obtaining a satisfactory diagnosis or reaching an accurate conclusion . the interests of the client are best served by being subjected to a minimum burden of stress , trauma , anxiety or simple bother , during the testing procedure . reference is now made to fig1 which illustrates a schematic view characterizing the steps of a software - driven protocol ( 100 ) for managing a virtual clinical neuro - psychological testing program . the present invention relates to a software - driven protocol for managing a virtual clinical neuro - psychological testing program , the protocol including for each client of a plurality of clients the steps : a ) evaluating ( 102 ) a prior history ( 101 ) of the client ; b ) according to the evaluated prior history , forming ( 105 ) an appropriate battery of tests ( 106 ) for testing the client wherein said battery incorporates pseudo - randomization ( 107 ) of at least one representational or organizational parameter ; and i . delivering ( 109 ), to the client ( 111 ), the formed battery of tests , and ii . accepting ( 112 ), from the client , a substantially completed response to the delivered formed battery of tests ; e ) into the prior history of the client , integrating ( 114 ) each of a multiplicity of clients will be provided with specific software , which will enable a professional clinician to maintain an interchange of information using corresponding software . this will allow a client to feed historical information to the clinician , receive testing procedures and instructions , carry out tests and convey results to the clinician for evaluation and integration into a client &# 39 ; s history in order to determine further testing requirements or to draw conclusions regarding each client &# 39 ; s situation . simply stated , evaluating a prior history of the client is a clinician &# 39 ; s placing an emphasis on automatic determination of the tests needed based on the history of a client or , alternatively , a clinician shifting the emphasis to a higher level of involvement in the progress of the client . more specifically , forming an appropriate battery of tests , is the consequence of a clinician &# 39 ; s evaluation of the condition of a client . such a battery of tests will comprise a variety of tests including arranging some of the above - mentioned representational or organizational parameters in a random manner . the reason for applying a random rearrangement , revolves around the problem that clients tend to learn tests and , consequently , these no longer measure what was intended or required . to avoid this problem it is important that tests are not repeated at respective known close intervals . simply stated , re - use of the same testing instrument is generally restricted according to the protocol that validated that instrument . nonetheless , randomization may be accomplished by changing the order of elements in a test or by changing around actual examples in any question . introducing these different levels of changes ensures maintaining the efficacy of using repeated testing instruments at appropriate intervals . initially , the tests that are formed into a battery , will be appropriate to the client &# 39 ; s evaluated situation . using a data communications medium ( 110 ) interactively , implies sending test materials to a client and receiving the substantially complete results from the client . the cycle of sending and receiving may occur as a single or as a series of interchanges , when administering a single battery . it is preferred that the remainder of the formed battery is modified after each interchange cycle ; as will be described in detail below in the section describing dynamic hierarchical testing . morespecifically , delivering to the client , the formed battery of tests , takes the form of conveying testing instruments by means , for example , of the internet , by email or other data communication media similarly , accepting from the client , will also take place using one of similar data communication media . analyzing the accepted response from the client , more specifically , is the means by which a clinician will decide on comparing accepted results to normal or anticipated standards and then to decide whether to call in a physician or to deliver a further battery of tests . further , by integrating these analyzed results into the prior history of the patient , the clinician becomes able to more easily determine the route to be followed , in particular , with regard to further testing , if needed . subsequently scheduled tests , more specific to a client &# 39 ; s condition or need , will be applied , following on a defining of the current forming of a battery of tests to the next forming event and so to evaluate results . to conclude , the cycle of events defined by the present invention , enables the fulfillment of the broad goals of the present invention . a professional can deal with a large client population and the client will be minimally subjected to invasive interaction with the professional . more sophisticated varieties of the present invention , described below , will more efficiently make use of the professional &# 39 ; s time and more efficiently reduce the client &# 39 ; s burden , without reducing the efficiency of interaction between professional and client . when performed by an expert , a standard neuro - psychological evaluation involves using a loosely structured battery of substantially standardized tests , that are progressively individualized to specific diagnostic issues of each particular patient . hence , the expert forms and then uses a custom configured battery of tests that provide a thorough investigation of a patient , depending on the circumstances of the problem and the state of the patient . sometimes , a rapid screening examination may be adequate if the problem is minor or intermittent . on the other hand , screening may be all that is possible if the condition or participation of the patient does not allow more detailed testing . if certain domains of function are suspected to be abnormal , based on historical information or on positive findings during screening tests , then more detailed testing of those domains is warranted . it is apparent that the neuro - psychological professional will be able to manage a larger number of clients using tools supplied by a broad range of experts . simultaneously , billing and time management will be superior . on the other hand , clients will receive a more cost - effective and professional service , and have the benefit of a significantly reduced testing burden . according to an embodiment of the present invention , the protocol in which analyzing the accepted response includes : applying ( 119 ) rule - based criteria for quantifying ( 120 ) at least one difference between a pair of metrics selected from the list : b ) a metric of expected performance ( 122 ) on at least one test of the formed battery of tests ; c ) actual performance ( 123 ) on at least one test of the formed battery of tests ; and d ) normative values ( 124 ) based on large body of data for at least one test of the formed battery of tests . more specifically , results will be evaluated and applied to a selection of additional tests , if required , by comparing two of the metrics , evaluated prior history , a measure of expected performance , actual performance and normal performance from a body of previously developed data . this enables a system embodying the present method to determine whether the next level of interaction of tests is warranted . a comparison of the evaluated prior history of a client with a metric of expected performance will indicate whether the selected battery of tests is likely to be appropriate . comparing the evaluated prior history with a metric of actual performance will enable the clinician to determine the subsequent course of action , namely to apply further , more specific testing instruments , to call in the services of a physician or to decide that no further testing is indicated at the present time . comparing the evaluated history with normative test values , based on a large body of data , enables the clinician to anticipate testing results so that he can preempt the need for the client to carry the testing instrument to its conclusion or to replace the test with another , more appropriate or to decide that no further testing is indicated at the present time . further , to compare a metric of expected performance with actual performance , confirms , for the clinician , the correct direction of , and the appropriateness of the selected testing instruments . also , comparing the expected performance with normative values for a metric of the same test enables the clinician to determine the nature of the condition of the client , relative to the norm , in the light of the evaluated history of the client . finally , comparing actual performance against normative values , gives the clinician an insight into the nature of the condition of the client and to decide whether further specific testing is called for or whether consultation with other expert professionals is indicated . reference is now made to fig2 which illustrates a schematic view characterizing the steps of a clinical protocol ( 200 ) for normal use activities , the present invention also relates to a clinical protocol for normal - use activities , the protocol including the steps of : a ) for substantially each client in an ensemble of clients ( 211 , 212 , 213 , etc .) monitoring ( 201 ) at least one metric of normal - use activities ; b ) for substantially each metric of the at least one metric of normal use activities , until a predetermined threshold of validation is achieved for an ensemble of clients , first managing ( 202 ) a virtual clinical neuro - psychological testing program ( 205 , 207 , 209 , etc .) for substantially each client in the ensemble , and correlating ( 203 ) analytical metrics derived from the neuro - psychological testing program with the monitored metrics of normal - use activities , thereby validating at least one of the monitored metrics of normal - use activities as a neuro - psychological metric ; and c ) for substantially each validated metric of the at least one metric of normal use activities , second managing ( 204 ) a virtual clinical neuro - psychological testing program for at least one client ( 214 ) wherein a validated normal - use activity metric is used as a classical testing instrument . more specifically , second managing applies to using the technique of using validated normal activities testing . in particular , what has been learned about normal activities , contributes to the way clients &# 39 ; testing programs are managed . current practice requires the clinician to administer a neuro - psychological test , substantially , in the form in which the test was validated . generally , this involves many repetitions over extensive testing sessions . there are currently no validated automated models for making decisions regarding test termination or switching to a different test during the course of the testing session . to partially alleviate this burden on the client , the concept of normal activities testing is introduced . more specifically , software will provide a means for monitoring a large number of clients during respective normal activities ( 206 , 208 , 210 , etc .) managing each client until a sufficient level of validation is achieved by correlating results , and then allowing evaluation of clients &# 39 ; normal activity use with the validated test . of necessity , the optimal method of monitoring such normal activities testing , is supplemented with direct on - line testing . this would seem to require a reduced level of apparent participation by a client and , therefore , substantially less trauma and stress for a client . simultaneously , the clinician is able to deal with a plurality of clients improving the time efficiency and hence the cost effectiveness of such testing . [ 0141 ] fig3 illustrates a schematic view characterizing the steps corresponding to first managing and second managing ( 300 ) in the clinical protocol for normal use activities . in addition , fig9 illustrates a schematic view characterizing the steps of first managing and second managing ( 900 ) in the protocol for validating a non - validated test in a formed battery of validated tests by correlating responses from the validated and non - validated tests . according to a further embodiment of the present invention , the clinical protocol for normal - use activities in which occur first managing and second managing , include the steps : a ) evaluating ( 305 ) or ( 902 ), a prior history ( 302 ) or ( 901 ) of the client ; b ) according to the evaluated prior history , forming ( 306 ) or ( 903 ) an appropriate battery of tests ( 307 ) or ( 904 ) for testing the client wherein said battery incorporates pseudo - randomization ( 308 ) or ( 905 ) of at least one representational or organizational parameter ; and c ) via a data - communications medium ( 310 ) or ( 907 ), interactively i . delivering ( 309 ) or ( 906 ), to the client ( 311 ) or ( 908 ), the formed battery of tests ; and ii . accepting ( 312 ) or ( 909 ), from the client , a substantially completed response to the delivered formed battery of tests ; d ) analyzing ( 313 ) or ( 910 ) the accepted response ; and e ) into the prior history ( 302 ) or ( 912 ) of the client , integrating ( 314 ) or ( 911 ) according to a subsequent embodiment of the present invention , the clinical protocol for a metric of normal use activity of the monitored at least one metric of normal use activities , is calculated by measuring an activity selected from the list of copying , printing , pasting , editing , inserting , formatting , exchanging tasks , web surfing , backspacing , deleting and shifting , substantially , as described earlier i don &# 39 ; t see any of this referenced earlier in the examples of making use of personal computers , palm - held computers , cellular telephones , etc . it is reasonable to presume , once systems are operated according to the present invention , other normal activity monitoring devices will become standard items requested by clinicians for the routine administration of their clients . these could include devices such as a telemetric neurology monitor , a telemetric heartbeat monitor , etc . reference is now made to fig4 this illustrates a schematic view characterizing the steps corresponding to first managing and second managing ( 400 ) in the clinical protocol for validating a new testing instrument , according to an additional embodiment of the present invention , the clinical protocol for validating a new testing instrument , that describes first managing and second managing include the steps : a ) evaluating ( 402 ) a prior history ( 401 ) of the client ; b ) according to the evaluated prior history , forming ( 403 ) an appropriate battery of tests ( 404 ) for testing the client wherein said battery incorporates pseudo - randomization ( 405 ) of at least one representational or organizational parameter ; and i . delivering ( 406 ) to the client ( 408 ), the formed battery of tests , and ii . accepting ( 409 ), from the client , a substantially completed response to the delivered formed battery of tests ; e ) into the prior history ( 412 ) of the client , integrating ( 411 ) there is apparent repetition of aspects and certain embodiments of the present invention but this is necessitated due to differences in emphasis . in the particular case , regarding validating a new testing instrument that has been dealt with , the emphasis has changed from managing a testing program of validated testing instruments to using normal activities as part of the measure and validation of this form of testing . particularly , managing a testing program containing validated testing instruments , and using normal activities as part of the validation of normal use activities , will replace some of the known validated testing instruments . validation of neuro - psychological tests is presently a cumbersome and extremely costly procedure . clinicians are required to administer tests in the form in which these were validated . therefore , development of normal activity testing procedures as well as the introduction of validation techniques for these procedures , will enable a clinician to fulfill this criterion at minimal cost and with a minimum of direct effort . more specifically , the effects of the utilization of such a testing procedure on the client are manifold . inevitably , present testing procedures cause stress and trauma and , in many instances , render little or no useful results , whereas normal activities testing should be virtually free of such problems . with the present invention , the client will have a benefit of expertise from a multitude of professionals in many connected fields . another significant factor will certainly relate to substantially reduced costs . [ 0168 ] fig5 illustrates a schematic view characterizing the steps of a software driven protocol ( 500 ) for managing a virtual clinical neuro - psychological dynamic hierarchical testing program . the present invention further relates to a software - driven protocol for managing a virtual clinical neuro - psychological dynamic hierarchical testing program , the protocol including for each client of a plurality of clients the steps : a ) evaluating ( 504 ) a prior history ( 501 ) of the client ; i . interactively forming ( 505 ) an appropriate battery of tests ( 506 ) for testing the client wherein said battery incorporates pseudo - randomization ( 507 ) of at least one representational or organizational parameter ; ii . via a data - communications medium ( 511 ), interactively ( 512 ) delivering ( 510 ), to the client ( 513 ), the formed battery of tests , iii . accepting ( 514 ), from the client , a substantially completed response to the delivered formed battery of tests ; v . returning ( 508 ) to step b ) i until complete ; and c ) into the prior history ( 518 ) of the client , integrating ( 517 ) simply stated , evaluating a prior history of the client , includes determining the basis on which a battery of tests will be constructed to test specific aspects regarding the client . further , forming this appropriate battery of tests incorporating pseudo - randomization , is specifically for the purpose of ensuring that repeating these tests at a later time , will prevent the client from learning to perform the tests and thus , defeat the objective of the tests . in addition , delivering these formed batteries of tests and later accepting the substantially completed tests , using a data communication medium , facilitates accomplishing the purpose of interactively analyzing the test results and of delivering additional or more specific tests . this cycle of evaluating , forming tests , delivering and accepting these tests , and analyzing the results , is repeated until the clinician is satisfied with the results . integrating these into the prior history of the client , to determine the course to be followed with respect to the client , will assist in deciding whether further testing or whether consultation with other experts is needed . once again apparent repetition of aspects and certain embodiments of the present invention is necessitated due to differences in emphasis . in the particular case that has been dealt with , relating to a software - driven protocol for managing a virtual clinical neuro - psychological dynamic hierarchical testing program , the emphasis has changed from managing a testing program and from validation of normal activities testing , to managing a virtual clinical neuro - psychological dynamic hierarchical testing program . more specifically , managing a virtual clinical neuro - psychological dynamic hierarchical testing program , implies allowing testing programs to proceed only to a point at which the result will permit a better or more specific test or battery of tests to be applied . this will occur without the client having to , necessarily , complete the originally formed battery of tests . there are currently no validated automated models for making decisions regarding test termination or test switching during the course of a testing session . embodied in the present invention is a means for using a model whereby performance of neuro - psychological tests is evaluated on - line . an automated decision process is used to determine one of several options , including continuation of the test , premature termination of the test , change in level of difficulty of the same test , or switching to another test . for example , for each level of difficulty for each test , an upper and lower boundary form the range of sensitivity for performance at that level . performance may be determined abnormal for that level based on predetermined values of certain performance parameters in relation to said boundaries . decisions to terminate the test on the basis of poor validity may be based on indices indicative of poor cooperation with the test instructions or of fictitious results from malingering ( using known techniques to detect inconsistencies in the results ). tests may be terminated based on general poor performance signifying severe general cognitive dysfunction , or poor performance in specific domains signifying focal deficits worthy of more focused testing . within a given test , a hierarchical design is used to move the test through different levels of difficulty . this is accomplished , for different types of tests , by utilizing the principles of utilizing a series of tests having changing levels of difficulty . these are : a ) screening for subtle deficits may be accomplished by starting at the level of difficulty expected for normal individuals . if performance is in the range of normal , then the test is terminated . if there is abnormal performance , the test is repeated at a lower level of difficulty , until satisfactory performance is accomplished . for example , the common eye chart exam for visual acuity may be administered as a screening test by starting with the smallest letters . if the subject reads the lowest line satisfactorily , the exam is terminated . if not , the subject is presented with a line of larger letters until he is able to read the letters satisfactorily . b ) when seeking the highest level of performance , such as in vocational testing , the level of difficulty may start with that of average performance and may gradually increase until performance becomes inadequate or abnormal . c ) when quantification of the performance is required , such as in patient longitudinal follow - up during a drug rehabilitation or therapeutic program , the level of difficulty may start at the subject &# 39 ; s previous level of performance and either increase or decrease depending on the performance . decisions to switch to other tests within the prescribed battery or to tests outside of the original battery may be made based on test performance measured on - line : a ) if enough information has been garnered from a given test to determine whether performance is normal or , alternatively , to determine the degree of abnormality , a decision may be made to move to the next test in the given pre - determined battery . b ) if a specific domain of abnormality has been identified , a decision may be made to switch to one or more test that further expand upon the area ( s ) of abnormality . [ 0192 ] fig6 illustrates a schematic view characterizing the steps of the protocol ( 600 ) for validating a non - validated test in a formed battery of validated tests by correlating responses from the validated and non - validated tests . according to another embodiment of the present invention , the protocol ( 600 ) further includes the steps of : a ) establishing ( 601 ), upgrading ( 602 ) and maintaining ( 603 ) as appropriate , a validity metric ( 611 ) for a substantially non - validated test ( 604 ), for each client ( 607 ) of an ensemble of clients ; b ) introducing ( 605 ) a substantially non - validated test into the formed battery of tests ( 606 ); c ) analyzing ( 608 ) responses to the substantially non - validated tests ; and d ) correlating ( 610 ) responses with the responses from the other tests ( 609 ) in the formed battery of tests . existing data , describing the significance of normal use activities , is , in the main , weak . the simplest advantageous use of normal use activity monitoring is to create advisory notices to the attending professional whenever normal use changes occur . the professional , then , can decide to bring forward further testing . alternatively , in the event that there are no changes in monitored normal use , an attending professional , in many circumstances , is likely to approve an occasional postponement of a testing event . again , this is based on his professional assessment of the client and supplemented by an observation of stability or improvement in a normal use metric . the introduction of monitoring normal use activities for advancing or postponing testing events will , inevitably , result in accumulation of a large ensemble of data . from this data , monitored normal use can be correlated with validated testing instruments . it is not unlikely to presume that metrics of monitored normal use activities will become valid substitutes for numerous clinical diagnostic testing instruments . this also supports the objective of the present invention , allowing the professional to manage a large body of clients and reduce the testing burden of clients . more specifically , results of applied non - validated normal activities tests are compared with results of validated tests . this technique allows for the expansion and validation of normal activities tests . resulting from this , it becomes possible to use normal activities for providing valid but non - burdensome tests , especially , but not exclusively , for clients where more conventional testing is problematic . normal activities testing methods become , then , a classical testing instrument . [ 0203 ] fig7 illustrates a schematic view ( 700 ) characterizing the steps indicated in fig6 of establishing ( 601 ), upgrading ( 602 ) and maintaining ( 603 ) in the protocol for validating a non - validated test in a formed battery of validated tests , by correlating responses from the validated and non - validated tests . according to a variation of embodiments of the present invention , the clinical protocol for normal - use activities , establishing , upgrading and maintaining , include : a ) for substantially each client ( 701 ) in an ensemble of clients , monitoring ( 703 ) at least one metric of normal - use ( 702 ) activities ; b ) for substantially each metric of the at least one metric of normal use activities , until a predetermined threshold of validation is achieved for an ensemble of clients , first managing ( 704 ) a virtual clinical neuro - psychological testing program for substantially each client in the ensemble ; and correlating ( 706 ) analytical metrics derived from the neuro - psychological testing program with the monitored metrics of normal - use ( 705 ) activities , thereby validating ( 707 ) at least one of the monitored metrics of normal - use activities as a neuro - psychological metric ; and c ) for substantially each validated metric of the at least one metric of normal use activities , second managing ( 708 ) a virtual clinical neuro - psychological testing program for at least one client ( 709 ) wherein a validated normal - use activity metric ( 710 ) is used as a classical testing instrument . the consequences of using this software are significant for each of the participants . benefits to neuro - psychologist professionals will include being able to easily bill for services , the ability to better control a much larger client population , and availability of significantly more up - to - date testing materials . furthermore , it will provide a facile conduit to consult with appropriate specialists , easily enable the professional to manage referrals and to compare his evaluation with an expert system . clients , however , will benefit by making payment only for services actually utilized or needed , results of testing will be easily evaluated , a much broader range of testing materials will be available , and access to a wider range of professional services will be feasible . significantly , the burden in terms of stress and trauma will be much reduced . for the researcher , there are also significant advantages . these are : access to a large , well managed client population ; an environment to introduce weakly validated testing instruments and improve their validation ; well - structured pool of background data to correlate results from testing instruments — allowing validation of these instruments at a lower cost ; and provide an easy way to market these now validated tests and receive remuneration , for these now validated tests . turning to fig8 which illustrates a schematic view characterizing the steps of a software driven protocol ( 800 ) for managing a virtual clinical neuro - psychological testing program based on normal use activities the present invention furthermore relates to a software - driven protocol for managing a virtual clinical neuro - psychological testing program based on normal use activities , the protocol including for each client ( 801 ) of a plurality of clients the steps : a ) evaluating ( 802 ) of a prior history ( 804 ) of the client ; b ) according to the evaluated prior history , forming ( 805 ) at least one normal use activity factor of tests ( 806 ) for testing the client wherein said at least one normal use activity factor incorporates pseudo - randomization ( 807 ) of at least one representational or organizational parameter ; c ) via a data - communications medium ( 809 ), interactively ( 810 ) i . delivering ( 808 ) the formed at least one normal use activity factor tests to the client ( 811 ), ii . the client substantially completing the normal use activity factor tests , and iii . accepting ( 812 ), from the client , a substantially completed response to the delivered normal use activity factor tests ; e ) into the prior history ( 801 ) of the client , integrating ( 814 ) this further apparent repetition of aspects and certain embodiments of the present invention has been necessitated due to additional differences in emphasis . in the particular case that has been dealt with , the emphasis has changed from managing a testing program , from validation of normal activities testing and from managing a virtual clinical neuro - psychological dynamic hierarchical testing program , to managing a virtual clinical neuro - psychological testing program based on normal use activities . more specifically , there is a presumption that there are validated testing instruments . nevertheless , substantially every neuro - psychologist of an expert level agrees that there is no substantial substitute for observing the client and interacting with the client until such time as the attending professional is convinced that he has an understanding of inter - related functional aspects of the client &# 39 ; s state . according to any of the embodiments of the present invention , the protocol in which forming an appropriate battery of tests includes choosing in fig1 ( 106 ), in fig3 ( 307 ), in fig4 ( 404 ), in fig5 ( 506 ), and in fig8 ( 806 ), at least two tests for measuring at least one mental or motor function , and the tests are selected from the list of testing instruments for measuring : mental health , intelligence , iq , memory , immediate recall , memory encoding , memory retrieval , working memory , semantic memory , procedural learning , sequence learning , conditioned response , pavlovian learning , associative learning , implicit learning , explicit learning , block learning , motor learning , pattern matching , judgment , attention , concentration , visual - spatial perception , velocity perception , distance perception , visual searching , calculational ability , mathematical ability , abstract thinking , symbolic thinking , adaptation , sensory - motor adaptation , language , reading , naming , comprehension , classification , direction - following , vigilance , motor , sensory - motor , coordination , psychomotor performance , dexterity , motor skills , tremor , physiological tremor , simple reaction time , choice reaction time , sustained attention , selected attention , divided attention , driving safety , ballistic movement , bradykinesia , hypokinesia , akinesia , hypometria , movement speed , movement smoothness , movement accuracy , repetitive movement , accurately timed movements , bimanual coordination , hand - eye coordination , personality , scholastic performance , depression , psychosis , neurosis , anxiety , stress , post - traumatic stress , dementia , static visual acuity , dynamic visual acuity , handwriting analysis , speech analysis , voice tremor , or metrics of interacting with a testing instrument . according to any of the embodiments of the present invention , the protocol in which forming occurs , includes for at least one test in the battery of tests defining a subset of equivalent validation of testing objects and therein pseudo randomization includes randomizing amongst the substantially validated testing objects . simply stated , an additional means of validation of testing may be facilitated in accordance with embodiments to the present invention . by insertion of a number of condition non - specific tests randomly within established and validated tests and batteries of tests , the clinician will be able to accumulate data to validate these inserted tests or sub - tests . expanding on the scope and multiplicity of neuro - psychological testing will enable the professional clinician to expand the scope and depth of investigative testing and will improve the ability to manage a substantially larger client base with improved levels of professional service . according to any of the embodiments of the present invention , the protocol in which evaluating occurs , includes a disclosure of information by the client . according to any of the embodiments of the present invention the protocol in which evaluating of a prior history of the client occurs , includes : interactively merging ( 104 ), ( 503 ) or ( 804 ), practitioner recommendations ( 103 ), ( 502 ) or ( 803 ), for this client into the prior history . according to any of the embodiments of the present invention , the protocol further includes the step of scheduling a next evaluating of a prior history of the client . notwithstanding the existence or the original or current mode of arranging monitoring of the client , scheduling a next evaluating of a prior history of the client means simply scheduling the next testing selection , or alternatively , scheduling describes bringing scheduling out of automatic mode to the eyes of the expert professional . even if normal use activity has no validation as a metric , nevertheless , monitored normal use should either cause the next scheduling to be advanced or should cause the professional to re - certify his evaluation . according to any of the embodiments of the present invention , the protocol in which analyzing the accepted response occurs , includes : calculating ( 117 ), an analytical metric of client performance using the accepted response in the formed battery of tests of at least one client performance parameter from the accepted response of a test in the formed battery of tests . this analyzing and calculating , simply stated , means appreciating the significance of the results of a single variable testing instrument according to any of the embodiments of the present invention , the protocol in which analyzing the accepted response occurs , includes : calculating ( 118 ), an analytical metric of client performance using the accepted response in the formed battery of tests of at least one convergence parameter from the accepted response of at least two tests in the formed battery of tests . in this calculating an analytical metric of client performance , neuro - psychological testing makes use of multi - parametric instead of single variable factors . this analyzes and calculates the results of more than one variable converging to a significant definition of the client &# 39 ; s condition . according to any of the embodiments of the present invention , the protocol of a test of the delivered and accepted formed battery of tests includes a metric of the clients interaction with a client &# 39 ; s peripheral device , and the device is selected from the list : color graphic display ; b / w graphic display ; audio speaker ; audio stereophonic headphones ; mouse ; joystick ; roller - ball ; keyboard ; galvanic skin response monitor ; web - cam camera ; microphone ; personal communication device ; touch pad , touch screen ; etc . simply stated , the client &# 39 ; s testing will include interaction , from test to test , with a peripheral device , and includes examining such effects as speed of operation , consistency of color discernment , mouse or joystick tremor , speech variations , etc . according to another embodiment of the present invention , the clinical protocol for substantially each client in an ensemble of clients , monitoring at least one metric of normal - use activities includes downloading a plug - in ( 205 , 207 , 209 , etc .) and ( 805 ) to the client &# 39 ; s machine , installing the plug - in , and uploading data collected by the plug - in , etc . according to an embodiment of the present invention , in the clinical protocol for substantially each client in an ensemble of clients , monitoring at least one metric of device - dependent normal - use activities , includes copying , pasting , editing , inserting , formatting , exchanging tasks , web surfing , keyboard backspacing , deleting and shifting , etc . more specifically , the application of device dependent testing capable of software enablement as well as various data communication systems is important in the application of normal activities testing techniques , for example : a ) a personal computer , internet enabled , where measurements are possible , relating to keyboard use , mouse use , etc . b ) palm - held computer which , on occasion , interacts with various types of data communication devices , can be applied to normal activity testing using the key - pad , using the search and select functions , using stylus and other data entering modes , etc . c ) personal communication devices such as cellular telephones are also applicable . making use of functions including key - pad use , device function uses like dialing or searching , voice print access interpreting the level of various actions like slurring or decipherability , etc . are significant measures . according to a subsequent embodiment of the present invention , in the clinical protocol for substantially each client in an ensemble of clients , monitoring at least one metric of normal - use activities , includes computer related interactivity , wireless activity , voice prints frequency analysis , mouse tremor , geographic positioning system tremor , key strokes , and special case use , etc . simply put , monitoring normal use activities such as those listed , enables the neuro - psychological professional to make decisions regarding the state of a client , whether such testing has been validated or not . significance can even be placed on monitoring unvalidated tests simply by making comparisons over a period of time . properly validated tests will give considerably more information to a clinician . changes in a monitored factor would indicate to the professional , that a client requires further testing immediately or at some later time . monitoring of these factors can be on a continuous basis or simply whenever the client makes use of any of the data communication devices available . a direct consequence to use of this software and plug - ins is : a ) maintaining and improving professional neuro psychologists &# 39 ; ability to manage large client bases with an appropriate level of professional service , b ) billing control for testing becomes a straightforward matter for both client and clinician , e ) providing clients with equal or lower test participation and simpler active cooperation with test instruments , f ) the possibility of constant online contact and interaction between client and clinician , and [ 0255 ] fig1 illustrates a schematic view characterizing the steps of a computer program product ( 1000 ) including a computer usable medium having computer readable code ( 1011 ) embodied therein for a software driven protocol for managing a virtual clinical neuro psychological testing program . the present invention additionally relates to a computer program product ( 1000 ) including a computer usable medium having computer readable program code embodied therein for a software - driven protocol ( 1001 ) for managing a virtual clinical neuro - psychological testing program , on the clinician side ( 1009 ), the computer readable program code in said article of manufacture including : for each client of a plurality of clients : a ) first computer readable program code for causing a computer to evaluate a prior history of the client ; b ) tied to the first computer readable code , second computer readable program code for causing the computer , according to the evaluated prior history , to form an appropriate battery of tests for testing the client wherein said battery incorporates pseudo - randomization of at least one representational or organizational parameter ; c ) tied to second computer readable code , third computer readable program code for causing the computer , via a data - communications medium , to interactively deliver to the client the formed battery of tests , and to accept from the client a substantially completed response to the delivered formed battery of tests ; d ) tied to third computer readable program code , fourth computer readable program code for causing the computer to analyze the accepted response ; and e ) tied to the fourth computer readable program code , fifth computer readable program code for causing the computer , into the prior history of the client to , integrate the accepted substantially completed response or at least one analytical metric thereof the present invention collaterally relates to a computer program product including a computer usable medium having computer readable program code embodied therein for , a client side ( 1010 ) compliance with a software - driven protocol ( 1002 ) for managing a virtual clinical neuro - psychological testing program , the computer readable program code in said article of manufacture including : a ) first computer readable program code for causing the computer to , via a data - communications medium , interactively , receive by the client , the formed battery of tests ; c ) the client to return the substantially completed response to the formed battery of tests . the present invention likewise relates to a computer program product including a computer usable medium having computer readable program code embodied therein for , clinician side ( 1009 ) management of a clinical protocol ( 1003 ) for normal - use activities , the computer readable program code in said article of manufacture including : a ) first computer readable program code for causing the computer , for substantially each client in an ensemble of clients , to monitor at least one metric of normal - use activities ; b ) tied to the first computer readable program code , second computer readable program code for causing the computer , for substantially each metric of the at least one metric of normal use activities , until a predetermined threshold of validation is achieved for an ensemble of clients , to first manage a virtual clinical neuro - psychological testing program for substantially each client in the ensemble ; and to correlate analytical metrics derived from the neuro - psychological testing program with the monitored metrics of normal - use activities , thereby to validate at least one of the monitored metrics of normal - use activities as a neuro - psychological metric ; and c ) tied to the second computer readable program code , third computer readable program code for causing the computer , for substantially each validated metric of the at least one metric of normal use activities , to second manage a virtual clinical neuro - psychological testing program for at least one client wherein a validated normal - use activity metric is used as a classical testing instrument . the present invention further relates to a computer program product including a computer usable medium having computer readable program code embodied therein , on the client side ( 1010 ), for a clinical protocol ( 1004 ) for normal - use activities , the computer readable program code in said article of manufacture including : a ) first computer readable program code for causing the computer , for substantially each client in an ensemble of clients , to monitor at least one metric of normal - use activities by the client ; b ) tied to the first computer readable program code , second computer readable program code for causing the computer , for substantially each validated metric of the at least one metric of normal use activities , for at least one client to perform a virtual clinical neuro - psychological testing program wherein a validated normal - use activity metric is used as a classical testing instrument . the present invention also relates to a computer program product including a computer usable medium having computer readable program code embodied therein for a software - driven protocol ( 1005 ) for the managing a virtual clinical neuro - psychological dynamic hierarchical testing program , on the clinician side ( 1009 ), the computer readable program code in said article of manufacture including for each client of a plurality of clients : a ) first computer readable program code for causing a computer to evaluate a prior history of the client ; b ) tied to the first computer readable program code , second computer readable program code for causing the computer , according to the evaluated prior history , to : i . interactively form an appropriate battery of tests for testing the client wherein said battery incorporates pseudo - randomization of at least one representational or organizational parameter ; ii . via a data - communications medium , interactively deliver to the client the formed battery of tests , iii . accept from the client a substantially completed response to the delivered formed battery of tests ; c ) tied to the second computer readable program code , third computer readable program code for causing the computer , into the prior history of the client , to integrate : the present invention furthermore relates to a computer program product including a computer usable medium having computer readable program code embodied therein for , a client side ( 1010 ) compliance with a software - driven protocol ( 1006 ) for managing a virtual clinical neuro - psychological dynamic hierarchical testing program , the computer readable program code in said article of manufacture , tied to the first computer readable program code , second computer readable program code for causing the computer , according to the evaluated prior history : a ) via a data - communications medium , the client , interactively to receive the test program , b ) the client to substantially complete a response to the delivered test program , and c ) via a data - communications medium , the client to return the substantially complete test program . the present invention likewise , relates to a computer program product including a computer usable medium having computer readable program code embodied therein for a software - driven protocol ( 1007 ) for managing a virtual clinical neuro - psychological testing program based on normal use activities , on the clinician side ( 1009 ), the protocol , for each client of a plurality of clients , the computer readable program code in said article of manufacture including ; a ) first computer readable program code for causing a computer to evaluate a prior history of the client ; b ) tied to the first computer readable code , second computer readable program code for causing the computer to , according to the evaluated prior history , form at least one normal use activity factor of tests for testing the client wherein said at least one normal use activity factor incorporates pseudo - randomization of at least one representational or organizational parameter ; c ) tied to the second computer readable code , third computer readable program code for causing the computer , via a data - communications medium , to interactively i . deliver the formed at least one normal use activity factor tests to the client , ii . the client substantially complete the normal use activity factor tests , and iii . accept , from the client , a substantially completed response to the delivered normal use activity factor tests ; the present invention similarly relates to a computer program product including a computer usable medium having computer readable program code embodied therein for a software - driven protocol ( 1008 ) for managing a virtual clinical neuro - psychological testing program based on normal use activities , on the client side ( 1010 ), the protocol , for each client of a plurality of clients , the computer readable program code in said article of manufacture including : a ) first computer readable program code for causing the computer , via a data - communications medium , to interactively i . receive the formed at least one normal use activity factor tests by the client , ii . the client substantially complete the normal use activity factor tests , and iii . send from the client , the substantially completed response to the delivered normal use activity factor tests . [ 0302 ] fig1 illustrates a schematic view characterizing a program storage device ( 1100 ) readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for a software driven protocol for managing a software driven protocol for managing a virtual clinical neuro psychological testing program . the present invention , in addition , relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for a software - driven protocol ( 1101 ) for managing a virtual clinical neuro - psychological testing program , the protocol including for each client of a plurality of clients , on the clinician side , steps including : b ) according to the evaluated prior history , forming an appropriate battery of tests for testing the client wherein said battery incorporates pseudo - randomization of at least one representational or organizational parameter ; ii . accepting , from the client , a substantially completed response to the delivered formed battery of tests ; the present invention additionally relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for a software - driven protocol ( 1102 ) for managing a virtual clinical neuro - psychological testing program , the protocol including for each client of a plurality of clients , on the client side , steps including : a ) via a data - communications medium , interactively the client receiving the formed battery of tests , c ) via a data - communications medium , the client sending , a substantially completed response to the delivered formed battery of tests . the present invention in addition relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , in the clinician side , a clinical protocol ( 1103 ) for normal - use activities , steps including : a ) for substantially each client in an ensemble of clients , monitoring at least one metric of normal - use activities ; b ) for substantially each metric of the at least one metric of normal use activities , until a predetermined threshold of validation is achieved for an ensemble of clients , first managing a virtual clinical neuro - psychological testing program for substantially each client in the ensemble ; and correlating analytical metrics derived from the neuro - psychological testing program with the monitored metrics of normal - use activities , thereby validating at least one of the monitored metrics of normal - use activities as a neuro - psychological metric ; and c ) for substantially each validated metric of the at least one metric of normal use activities , second managing a virtual clinical neuro - psychological testing program for at least one client wherein a validated normal - use activity metric is used as a classical testing instrument . the present invention additionally relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , in the client side , a clinical protocol ( 1104 ) for normal - use activities , said method steps including : a ) for substantially each client in an ensemble of clients , performing at least one metric of normal - use activities ; b ) for substantially each metric of the at least one metric of normal use activities , until a predetermined threshold of validation is achieved for an ensemble of clients , carrying out a virtual clinical neuro - psychological testing program by substantially each client in the ensemble ; and c ) for substantially each validated metric of the at least one metric of normal use activities , carrying out a virtual clinical neuro - psychological testing program wherein a validated normal - use activity metric is used as a classical testing instrument . further , the present invention also relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , on the clinician side , a software - driven protocol ( 1105 ) for managing a virtual clinical neuro - psychological dynamic hierarchical testing program , the protocol including for each client of a plurality of clients the steps : i . interactively forming an appropriate battery of tests for testing the client wherein said battery incorporates pseudo - randomization of at least one representational or organizational parameter ; ii . via a data - communications medium , interactively delivering , to the client , the formed battery of tests , iii . accepting , from the client , a substantially completed response to the delivered formed battery of tests ; moreover the present invention also relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , on the client side , a software - driven protocol ( 1106 ) for managing a virtual clinical neuro - psychological dynamic hierarchical testing program , the protocol said method including for each client of a plurality of clients the steps : a ) via a data - communications medium , interactively the client receiving , from the clinician , the formed , dynamic hierarchical testing program ; c ) via a data - communications medium , the client sending , a substantially completed response to the delivered tests . in addition the present invention likewise relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , a software - driven protocol ( 1107 ) for managing a virtual clinical neuro - psychological testing program based on normal use activities , the protocol including , on the clinician side , for each client of a plurality of clients the said method steps including : b ) according to the evaluated prior history , forming at least one normal use activity factor of tests for testing the client wherein said at least one normal use activity factor incorporates pseudo - randomization of at least one representational or organizational parameter ; i . delivering the formed at least one normal use activity factor tests to the client , ii . the client substantially completing the normal use activity factor tests , and iii . accepting , from the client , a substantially completed response to the delivered normal use activity factor tests ; finally , the present invention also relates to a program storage device readable by machine , tangibly embodying a program of instructions executable by the machine to perform method steps for , a software - driven protocol ( 1108 ) for managing a virtual clinical neuro - psychological testing program based on normal use activities , the protocol including , on the client side , for each client of a plurality of clients the said method steps including : a ) via a data - communications medium , the client , interactively receiving the formed at least one normal use activity factor tests ; b ) the client substantially completing the normal use activity factor tests , and c ) via a data - communications medium , the client returning a substantially completed response to the delivered normal use activity factor tests . having described various large - scale embodiments of the present invention , it would seem appropriate to introduce some of the novel working components , which are incorporated into the various embodiments . at present , in the field of neuro - psychology , a trained , fully qualified expert should have the opportunity to spend a substantial amount of time with each client . this time should include a significant amount of observation and testing , on and one - on - one basis . unfortunately , a shortage of fully trained , expert , professionals in this field , has given rise to relatively few clients receiving sufficient testing and adequate treatment . the present invention seeks to address this problem . it should be understood , at the outset , that a neuro psychologist professional has , as a prime objective , a desire to perform as little testing as possible on each client , to spare clients from some of the inevitable stress and trauma resulting from any testing . furthermore , if such testing can be largely limited to normal activity testing , this would present each client with a minimum of stress and trauma . registering a new client : for the facile convenience of the attending clinician , a template is provided when registering a new client ( and , optionally , when a clinician intervenes by evaluating that client &# 39 ; s performance ). the template targets potential functions that may be important in an evaluation of the following situations : 1 . normal life activity including aging — even in people who are not thought to have a disease . for example , measuring memory - loss , personality change , confusion , delirium , slowed motor function , decreased dexterity , poor coordination , tremor , driving safety , baseline intelligence , baseline personality profile , etc . 2 . high level function . for example , vocational evaluation , security clearance , etc . child developmental metrics . for example , language or speech problems , dyslexia , social problems , behavioral problems , attention deficit hyperactivity disorder ( adhd ), poor scholastic performance , hyperactivity , poor attention , motor delay , lack of or poor coordination , mental retardation , intellectual impairment , recommendation for medication , special education , speech therapy , occupational therapy , etc . 3 . dementia ( decreased functioning associated with mental decline ). for example , to provide information useful for diagnosis and follow - up of alzheimer &# 39 ; s disease or other dementia disorders ( such as vascular dementia , dementia caused by depression , aids , hiv , and other known causes ); decision to initiate medical treatment ; decision to change or terminate medical treatment ; determine cognitive or motor side effects of drugs ; recommendation to stop driving ; recommendation for living assistance at home or in an institution ; cognitive rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs , etc . 4 . parkinsonian syndromes . for example , to provide information useful for diagnosis and follow - up of parkinson &# 39 ; s disease , and other conditions that result in slowed movement ( multiple system atrophy , striatonigral degeneration , diffuse lewy body disease , progressive supranuclear palsy , frontal lobe syndromes , and other known causes ); decision to initiate medical treatment ; decision to change or terminate medical treatment ; determination of cognitive or motor side effects of drugs ; recommendation to stop driving ; recommendation for living assistance at home or in institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs . for surgical candidates , to evaluate exclusion factors that would indicate that there is excessive risk to the patient ; to evaluate effects of the surgery ; to provide objective information to guide adjustment of post - surgical medical or deep brain stimulation therapy ( or other therapy , such as adjustment of flow rate from an implanted pump that may deliver medication at pre - programmed rates ); etc . 5 . rehabilitation . for example , to evaluate current status and past trends in cognitive and motor function , as it impinges on adls ; to design focused treatment programs ; to evaluate efficacy of treatment and to decide future treatment ; to use ( versions of ) computerized exercises as treatment modalities , etc . 6 . involuntary movements . for example , to characterize and help diagnose uncontrolled movement such as rest tremor , postural tremor , physiological tremor , chorea , ballismus , athetosis , tics , dystonia , myoclonus , dysmetria , action tremor , and cerebellar tremor ; to evaluate efficacy of treatment and decide regarding future treatment ; etc . 7 . stroke . for example , to evaluate effects of a stroke and to track rehabilitation ; to evaluate efficacy of treatment and decide regarding future treatment ; determine cognitive or motor side effects of drugs ; recommendation to stop driving ; recommendation for living assistance at home or in institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs , etc . 8 . epilepsy . for example , to evaluate cognitive and motor status ; to evaluate efficacy of treatment and decide regarding future treatment ; determine cognitive or motor side effects of drugs ; recommendation for living assistance at home or in institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs . for surgical candidates , to evaluate exclusion factors that would indicate that there is excessive risk to the patient ; to evaluate effects of the surgery ; to provide objective information to guide adjustment of post - surgical medical or deep brain stimulation therapy ( or other therapy , such as adjustment of flow rate from an implanted pump that may deliver medication at pre - programmed rates ); etc . 9 . multiple sclerosis . for example , to evaluate cognitive and motor status ; to evaluate efficacy of treatment and decide regarding future treatment ; determine cognitive or motor side effects of drugs ; recommendation for living assistance at home or in institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs ; etc . 10 . head trauma . for example , to evaluate cognitive and motor status ; to evaluate efficacy of treatment and decide regarding future treatment ; determine cognitive or motor side effects of drugs ; recommendation for living assistance at home or in an institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs ; etc . 11 . neuro - toxicology . for example , evaluate effects of chronic alcohol intake ; evaluate effects of acute alcohol intoxication ; evaluate effects of illicit drugs ; evaluate effects of legal sedative drugs ( valium , etc . ); evaluate effects of detoxification programs ; recommendation for cognitive , physical or occupational therapy ; recommendation to stop driving ; etc . 12 . psychiatric diseases ( depression , mania , psychosis , anxiety , obsessive - compulsive disorder , etc ). for example , to evaluate cognitive and motor status ; to evaluate efficacy of treatment and decide regarding future treatment ; determine cognitive or motor side effects of drugs ; recommendation for living assistance at home or in institution ; cognitive , physical , speech , and occupational rehabilitation therapy ; standardized tool to judge efficacy and safety of new or existing drugs ; to evaluate psychiatric comorbidity ( conditions present in addition to other neurological or psychiatric disease ); etc . 13 . conversion disorders and malingering . for example , used as objective measure to aid in the diagnosis of non - organic neurological syndrome ; performance is inconsistent with complaints ; detect malingering by negative response bias . 14 . driving ability . for example , on - road testing to determine whether the subject is ‘ driving while intoxicated ’; testing before dispatch ; etc . 15 . self - improvement . for example , used by client ( with or without clinician recommendation ) for practice to improve certain mental or motor skill ; the same or similar tests ( with parameter randomization ) to be used to assess progress of improved performance ; etc . turning to fig1 which illustrates a schematic view characterizing a system overview ( 1200 ) of a software driven protocol for managing a virtual clinical neuro psychological testing program . this system overview includes a number of steps , namely : clinician user interface ( 1201 ), patient user interface ( 1202 ), pretesting input stages ( 1203 ), construction of test battery ( 1204 ), data acquisition ( 1205 ), data transfer ( 1206 ), data analysis for each test ( 1207 ), interpretation of battery ( 1208 ), database ( 1210 ), report generation ( 1209 ) and clinician access to results and report ( 1211 ). the clinician user interface ( 1201 ) procedure is further described in fig1 with regard to pretesting and the choice of a test battery ( 1300 ) indicating the clinician ( 1301 ) initiating a clinical data screen ( 1302 ), choice of test battery options ( 1303 ), choosing from a list of standard batteries ( 1305 ), utilizing the test battery wizard ( 1304 ) with regard to both a custom ( 1306 ) and a standard test battery ( 1307 ) resulting in the clinician &# 39 ; s affirmation of choice ( 1308 ) to carry on the testing or to reconsider the choice of test battery options . in fig1 , the stage of data acquisition ( 1205 ) described above , is further expanded ( 1400 ) as including the steps of test battery code downloading ( 1401 ), delivering screening battery ( 1402 ), screening battery data processing ( 1403 ), decision ( 1404 ) as to whether client is able to perform the battery and , if not , to terminate testing session ( 1405 ). if the client is able to perform the battery , the level of performance ( 1406 ) is assessed as at a high or at a low level , respectively then resulting in delivery of a high - level ( 1407 ) or low - level ( 1408 ) battery and this leads to the raw data ( 1409 ) stimuli and responses for each test . [ 0377 ] fig1 illustrates a schematic view characterizing the steps of a test battery wizard ( 1500 ) for use in discerning the nature of a battery of tests required with regard to a client &# 39 ; s needs as defined ( 1200 ), regarding construction of a test battery ( 1204 ). the steps as indicated ( 1500 ) are initiated by preferences for testing collected via the wizard ( decision tree ) ( 1501 ), and followed by determining ( 1502 ) if there is a need for assessment or rehabilitation . if rehabilitation is indicated , then a brain function related rehabilitation battery ( 1504 ) is selected . for further assessment testing , a primary criterion is decided ( 1503 ) for the type of battery , namely , brain function related battery ( 1505 ), system related battery ( 1506 ) or disease related battery ( 1507 ). also indicated is the stage ( 1207 ) of data analysis for each test further expanded and described in fig1 which illustrates a schematic view characterizing the steps of data analysis ( 1600 ) for each test carried out in accordance with the software driven protocol for managing a virtual clinical neuro psychological testing program . these steps include commencing with raw data stimuli and responses ( 1601 ) and a set of performance parameters ( 1602 ) per design of each test and continues with an algorithm step ( 1603 ) for data or signal processing to extract performance parameters ( specific for each test ) leading to the step ( 1605 ) of conversion of raw score to a statistical measure “ z - score ” for each output parameter when compared with normative data ( 1604 ) from the data base , and resulting in set of z - scores ( 1606 ) for the entire test battery . referring to the stage of interpretation of test battery ( 1208 ), this is further enlarged upon in fig1 , which illustrates a schematic view characterizing the steps of the interpretation of a test battery ( 1700 ) carried out in accordance with the software driven protocol for managing a virtual clinical neuro psychological testing program . this interpretation emanates from a set of z - scores ( 1701 ) for an entire battery and information ( 1702 ) on the make of the tests in the battery in terms of the brain function tested . these are applied to an algorithm step ( 1703 ), for comparing statistical results from multiple tests that overlap regarding the brain functions tested . the level of difficulty and extent of assessment differs between tests , and then lead to a summary of performance ( 1704 ) in terms of brain functions tested ( for each function , whether normal or abnormal , and extent of abnormality ). thereafter , based on the client testing history ( 1705 ) from the database , an algorithm step ( 1706 ), for incorporating past performance and to look at trends in the data , provides a summary of brain functions tested ( 1707 ), as change over time is personalized for the client . 6 . client interacts with the testing program via standard or customized computer input devices , including but not limited to keyboard , mouse , joystick , digitizing tablet , touch - pad , “ games ” controller , customized keyboard , microphone , touch - screen . 7 . client normal activity responses including all types of communication devices . 8 . testing software records all data in the form of stimuli and responses . depending on the nature of the test , responses may be in the form of binary data , ordinal data , discrete data ( scalar values such as reaction time and movement time ), or continuous digitized data ( a vector or matrix of data that may represent performance such as movement trajectory , sequence or key presses , or vocal response inputs ), or on - line or off - line derived parameters ( data extracted from the raw data summarizing the key features of the response , such as response accuracy , total movement time , negative response bias index , etc .) 1 . ‘ raw ’ data are analyzed to produce output parameters according to the design of the test and the nature of the data thereof . this step summarizes the data set from each test into a set of relevant numbers . 2 . for binary or ordinal data collected via questionnaires or rating scales , the data analysis consists of weighted combinations of individual responses for each subsection of the test . these are then compared to relevant normative data , resulting in a statistically relevant score ( e . g . z - score ) for each mental or motor function assessed by the test . 3 . in the case of continuous digitized data collected from mouse , joystick , or other input device , data are analyzed by different time and frequency domain mathematical techniques to extract output performance parameters relevant to the test . these generally include , but are not limited to , accuracy , reaction time , movement time , tremor amplitude and frequency , indices of learning . 4 . performance parameters thus computed are compared to relevant normative data using statistical methods to determine whether individual output parameters are statistically different from reference data , based on predetermined criteria for significance ( e . g . z - score ). these are referred to as test results . 5 . test results from some or all tests in the testing battery are compared for patterns of convergence or divergence from expected trends . as each test assesses a fixed set of mental or motor functions , and as test batteries are constructed to provide overlap or redundancy between different tests in the battery , there are predicted patterns of convergence of test results that are known to ( or intended to ) test the same or similar mental or motor functions . this may utilize statistical methodologies , automata , rule - based , neural network , or other methods . 6 . testing battery interpretation is a summary of the mental or motor functions tested in the battery ; for each an indication of normality or abnormality as compared to relevant reference populations , the confidence at which the interpretation was reached based on consistencies or inconsistencies in the test results , and in certain cases the degree of abnormality . 7 . client - specific interpretation is arrived at by comparing test interpretation to the client &# 39 ; s testing history . 8 . recommendations for further testing or for referral to other health care evaluation are provided , based on the interpretation . this may be derived from an expert system utilizing other demographic or other information provided by the clinician , client , or caretaker , in combination with the test battery interpretation . test batteries are constructed to focus on specific diagnostic or therapeutic issues . different options are available according to the preference of the user . 1 . function - related batteries are constructed to comprehensively test one or more brain functions . this may be appropriate for research use or for when there is a focused clinical question requiring thorough testing of only one or a few areas of brain function . 2 . symptom - related batteries are constructed to characterize specific symptoms or complaints , as relevant to diagnosis and clinical follow - up . these are designed to test brain functions that are directly or tangentially related to the symptom , as well as brain functions that may aid in including or excluding certain disease entities from the differential diagnosis . technology : done using an expert system containing known neuro - psychological deficits from experimental and clinical literature . 3 . disease - related batteries are constructed for use when following a client with a known disease entity . here , the batteries are designed to test brain functions that are known to be ( or suspected to be ) affected by the disease or its treatment . these may provide useful data to the clinician to determine the stage of the disease , provide prognostic data , and guide treatment decisions . these are considered essential for evaluating new or experimental therapies for specific disease entities or syndromes . technology : done using an expert system containing known neuro - psychological deficits from experimental and clinical literature . 4 . custom batteries are constructed according to specific questions posed by the clinician , or suggested by a rule - based system of the present invention , which evaluates the client testing history . technology : done using an expert system containing known neuro - psychological deficits from experimental and clinical literature . 5 . cognitive - rehabilitation testing batteries are constructed for situations where the client is a participant in a cognitive or motor rehabilitation program utilizing computerized tests ( described below ). here , the testing batteries are chosen automatically by the ‘ computer ’ to be independent of the interactive computer exercises used for the individualized cognitive rehabilitation program . the test batteries are chosen to assess progress on the cognitive rehabilitation program , based on both the client &# 39 ; s previous testing history and treatment plan and progress . therefore , each battery is designed to focus on specific brain functions , as applicable to the individual client . this type of testing provides useful data to the rehabilitation professionals ( psychiatrist , occupational therapist , physical therapist , speech therapist , cognitive therapist , others ) caring for the client . technology : done using an expert system containing known neuro - psychological deficits from experimental and clinical literature . 6 . self - improvement batteries are constructed similar to the function - related batteries , except to provide for the purpose of the client to practice on certain cognitive or motor skills . here , the tests will consist of those with sufficiently steep learning curves , randomize - able parameters , and multiple levels of difficulty . the clinician may choose appropriate test batteries with the help of an embodiment of a rule base system of the present invention : 1 . choose from a list of standard batteries . a list of batteries can be accessed according to an index , categorized by type of use . subcategories include symptom and disease - related batteries , each containing a description of the test including length of test , brain functions tested , intended use , level of difficulty . 2 . test battery ‘ wizard ’ to guide clinician in choice of battery , with recommendation based on client testing history . this consists of a decision - tree algorithm that poses questions to the user , and proceeds according to choice of an answer from a menu . at each step , the user verifies the choice . a ) alternatively , a free data entry field is available for the user to enter his answer . an algorithm will search from a list of keyword for the best fit ( like mesh headings from grateful med ). b ) eventually , the algorithm suggests the appropriate test battery that best satisfies the criteria requested by the user . c ) if the user &# 39 ; s criteria are not satisfied by the available batteries , or if the user refuses the batteries suggested by the wizard , then an option is given for formation of a custom battery . the wizard may suggest the need for formation of a custom test battery . once an appropriate battery is selected , a short screening battery is presented . based on the results of the screening battery , the following paths might be automatically chosen : 1 . if the client is unable to perform the tests because of insufficient level of arousal , attention capacity , or language skills , then the testing is terminated . 2 . if the results indicate high level of performance , then a detailed high - level testing battery is presented . 3 . if the results indicate low level of performance , then a detailed low - level testing battery is presented . 4 . note : for any given test battery the difference between the high - level and low - level options may be different tests ( as some tests are geared either for high or low ability ), or the same test set for different levels of difficulty . 5 . each testing battery focuses on a small set of brain functions , but screens additional brain functions . the interpretation for the battery includes recommendation for further in - depth testing of those screened brain functions that were found abnormal . 1 . test batteries are stored in a database . the database is referenced by multiple fields , including but not limited to : name of test , category of test , symptoms , diseases , intended uses , brain functions tested . 2 . as custom batteries are built , they are added to the test battery database . technology relating to dynamic development of normative values based on large body of data : 1 . data from each test is added to the database for that test , stored with user - input clinical data ; including age , level of difficulty , clinical condition ( based on client input ; e . g . normal , known disease , suspected disease ), special circumstances ( e . g . parkinsonian on / off state , time of day ), etc . 2 . for scoring ( interpretation ) of tests : appropriate reference norms are compiled from database , and z - score is computed . technology : application of statistical models , such as linear , multiple , or logistic data regression may be appropriate . for certain tests , whenever possible and appropriate , some non - essential test parameters are varied in a pseudo - random fashion : 1 . essential parameters are defined as those that directly relate to the brain functions being assessed by the test . 2 . non - essential parameters are defined as those that may be changed without affecting the brain functions being tested . illustration — in a test of visual acuity the essential parameter is the size of the characters on each line . a non - essential parameter is the actual sequence of characters on each line . it is either assumed or known that changing the non - essential parameter will not affect the assessment of visual acuity . rather , the randomization of this parameter enhances validity of test repetition by removing practice effects or the ability of the client to memorize the sequence from trial to trial . technology : set of non - essential parameters determined per test design , with “ expert ” input . for repeat client testing , previous choices will be removed from set , and then random ( or pseudo - random ) choice of parameter value is applied to the present test .
the present invention software driven protocol for managing a virtual clinical neuro - psychological testing program delivers , to individual or small groups of certified neuro - psychology professionals , an interface protocol for clinically monitoring and managing the neuro - psychological needs of very large numbers of individual patients . the preferred approach to such monitoring and testing , is within the scope of normal and routine activities . these patients are most often in need of regular testing , screening , and monitoring and , only occasionally , of referral or intervention . the protocol provides a majority of patients , with regular testing , screening , and monitoring services on a “ virtual - out - patient ” basis by interactively delivering individually configured batteries of tests to each patient ; via the internet . these tests include normal activity observation and monitoring . the delivery and respectively corresponding acceptance of the testing batteries is a substantially automatically scheduled feature of the protocol .