There is a need for a clear statement of what Evidence-Based Practice (EBP) means, a description of the skills required to practise in an evidence-based manner and a curriculum that outlines the minimum requirements for training health professionals in EBP. This consensus statement is based on current literature and incorporating the experience of delegates attending the 2003 Conference of Evidence-Based Health Care Teachers and Developers ("Signposting the future of EBHC").A variety of definitions of evidence-based practice (EBP) exist. However, definitions are in themselves insufficient to explain the underlying Evidence-Based Practice has evolved in both scope and definition. Evidence-Based Practice (EBP) requires that decisions about health care are based on the best available, current, valid and relevant evidence. These decisions should be made by those receiving care, informed by the tacit and explicit knowledge of those providing care, within the context of available resources.Health care professionals must be able to gain, assess, apply and integrate new knowledge and have the ability to adapt to changing circumstances throughout their professional life. Curricula to deliver these aptitudes need to be grounded in the five-step model of EBP, and informed by ongoing research. Core assessment tools for each of the steps should continue to be developed, validated, and made freely available.All health care professionals need to understand the principles of EBP, recognise EBP in action, implement evidence-based policies, and have a critical attitude to their own practice and to evidence. Without these skills, professionals and organisations will find it difficult to provide 'best practice'. Willing is not enough, we must do" Health care delivered in ignorance of available research evidence, misses important opportunities to benefit patients and may cause significant harm -4. Proviprocesses of EBP and to differentiate between an evidence-based process and evidence-based outcome.A variety of definitions of evidence-based practice (EBP) have been proposed. However, definitions are in themselves insufficient to explain the underlying Towards this goal, we propose three points to clarify and promote the realisation of EBP:1) A clear statement of what EBP means.2) A description of the minimum skill set required to practise in an evidence-based way.3) A curriculum that outlines the minimum standard educational requirements for training health professionals in EBP.This statement was conceived by the delegates of the second international conference of Evidence-Based Health Care Teachers and Developers held in Sicily in September 2003 . In resDuring the last century there has been an exponential growth of research and knowledge ,14. The With this expansion of information, our knowledge should be greater and our practice should be more effective. Unfortunately this is too often not the case . This resome of the uncertainties in this decision process by using the explicit knowledge obtainable from research information. But to do so the research information must be transformed into clinicians' knowledge. Information can be defined as data that has been sorted, analysed, & displayed and communicated through language, graphic displays, or numeric tables. Explicit knowledge is then the meaning people create using this information and its application through action in specific settings [Good practice including effective clinical decision making – step 4 of the EBP process – requires the explicit research evidence and non-research knowledge (tacit knowledge or accumulated wisdom). Clinical decision making is the end point of a process that includes clinical reasoning, problem solving, and awareness of patient and health care context . This prsettings . For exaThe term "Evidence-based medicine" was introduced in the medical literature in 1991 . An origEBP evolved from the application of clinical epidemiology and critical appraisal to explicit decision making within the clinician's daily practice, but this was only one part of the larger process of integration of evidence into practice. Initially there was a paucity of tools and programmes to help health professionals learn evidence-based practice. In response to this need, workshops based on those founded at McMaster by Sackett, Haynes, Guyatt and colleagues were set up around the world. During this period several textbooks on EBP were published accompanied by the development of on-line supportive materials.all knowledge gaps led to early recognition of practical limitations and disenfranchisement amongst some practitioners [Clinical Evidence [The initial focus on critical appraisal led to debate on the practicality of the use of evidence within patient care. In particular, the unrealistic expectation that evidence should be tracked down and critically appraised for itioners . The groitioners . In respitioners , the eviEvidence  and secoEvidence  have beeEvidence , though The five steps of EBP were first described in 1992  and most1. Translation of uncertainty to an answerable question 2. Systematic retrieval of best evidence available 3. Critical appraisal of evidence for validity, clinical relevance, and applicability 4. Application of results in practice 5. Evaluation of performance This five-step model forms the basis for both clinical practice and teaching EBP, for as Rosenberg and Donald observed, "an immediate attraction of evidence-based medicine is that it integrates medical education with clinical practice" .Different practitioners at different levels of responsibility within evidence-based organisations will require different skills for EBP and different types of evidence. It is a minimum requirement that all practitioners understand the principles of EBP, implement evidence-based policies, and have a critical attitude to their own practice and to evidence. Without these skills and attitidues, health care professionals will find it difficult to provide 'best practice'. Teachers, commissioners, and those in positions of leadership will require appraisal skills that come with higher training and continued use .The wider knowledge and use of these skills will help health professionals meet some of Hurd's list of desired educational outcomes  in being• distinguish evidence from propaganda (advertisement)• probability from certainty• data from assertions• rational belief from superstitions• science from folkloreEvidence-based practitioners need additional skills to supplement traditional knowledge. Health care graduates should "be able to gain, assess, apply and integrate new knowledge and have the ability to adapt to changing circumstances throughout their professional life" . Observaconscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients" [Learning has three components: knowledge, skills and attitudes. It is said that "attitudes are caught, not taught" . Attitudatients" . This pratients"  and is aatients"  where prThe curriculum framework for EBP should consider the importance of all steps shown in Table The term 'EBM' has evolved into a larger phenomenon, as increasing numbers of practitioners in various disciplines recognise the importance of evidence to inform all types of health care decisions. Furthermore, greater patient choice and complexity of care mean that many professionals practise as a team. In recognition of the importance of a united commitment to the principles of 'best practice', we propose that the term 'evidence-based practice' (EBP) be used to describe all aspects of this discipline.To ensure that future health care users can be assured of receiving 'best practice' regardless of the type or location of the care received, we make the following recommendations for education:1. The professions and their colleges should incorporate the necessary knowledge, skills and attitudes of EBP into their training and registration requirements.2. Curricula to deliver these competencies should be grounded in the "five-step model" Table .3. Further research into the most effective and efficient methods for teaching each step should be fostered, and linked with ongoing systematic reviews on each step.4. Core assessment tools for each of the steps should be developed, validated, and made freely available internationally.all components.5. Courses that claim to teach EBP should have effective methods for teaching and evaluating Evidence-Based Practice (EBP) requires that decisions about health care are based on the best available, current, valid and relevant evidence. These decisions should be made by those receiving care, informed by the tacit and explicit knowledge of those providing care, within the context of available resources.Finally, EBP requires a health care infrastructure committed to best practice, and able to provide full and rapid access to electronic databases at the point of care delivery. We believe that without the skills and resources for all the relevant components of this framework, the practice of a health care professional, or a health care organisation, cannot be said to provide their users with evidence-based care.1. This consensus statement is from an international working group representing both organisations and individual teachers and developers of evidence-based practice.2. Evidence-Based Practice (EBP) requires that decisions about health care are based on the best available, current, valid and relevant evidence. These decisions should be made by those receiving care, informed by the tacit and explicit knowledge of those providing care, within the context of available resources.3. All health care professionals need to understand the principles of EBP, recognise it in action, implement evidence-based policies, and have a critical attitude to their own practice and to evidence. Without these skills professionals will find it difficult to provide 'best practice'.4. The teaching of EBP should, as far as possible, be integrated into the clinical setting and routine care so that students not only learn the principles and skills, but learn how to incorporate these skills with their own life-long learning and patient care.All of the authors have received honoraria for teaching EBP FP is a consultant of Lilly Deutschland GmbH. The International Conferences of EBHC Teachers and Developers do not accept sponsorship from health technologies  manufacturers.MD, WS & PG wrote the original draft. AC, JM, KH, FP, AB & JO contributed to the concept and all revised drafts of the statement.The pre-publication history for this paper can be accessed here:


Virtual Reality (VR) offers a blend of attractive attributes for rehabilitation. The most exploited is its ability to create a 3D simulation of reality that can be explored by patients under the supervision of a therapist. In fact, VR can be defined as an advanced communication interface based on interactive 3D visualization, able to collect and integrate different inputs and data sets in a single real-like experience.However, "treatment is not just fixing what is broken; it is nurturing what is best" . For rehabilitators, this statement supports the growing interest in the influence of positive psychological state on objective health care outcomes.transformation of flow. In this view, VR can be used to trigger a broad empowerment process within the flow experience induced by a high sense of presence.This paper introduces a bio-cultural theory of presence linking the state of optimal experience defined as "flow" to a virtual reality experience. This suggests the possibility of using VR for a new breed of rehabilitative applications focused on a strategy defined as The link between its experiential and simulative capabilities may transform VR into the ultimate rehabilitative device. Nevertheless, further research is required to explore more in depth the link between cognitive processes, motor activities, presence and flow. What is virtual reality (VR)? For many health care professionals, VR is first of all a technology. Since 1986, when Jaron Lamier used the term for the first time, VR has been usually described as a collection of technological devices: a computer capable of interactive 3D visualization, a head-mounted display and data gloves equipped with one or more position trackers.But is this definition enough to describe the potential of VR for rehabilitation? If we look at its actual applications in rehabilitation, the answer is probably not .VR can be considered the leading edge of a general evolution of present communication interfaces like television, computer and telephone . The maiIn fact, VR is used in rehabilitation as "an advanced form of human-computer interface that allows the user to interact with and become immersed in a computer-generated environment in a naturalistic fashion" .et al. [Following this vision Rizzo et al.  identify• The capacity to systematically deliver and control dynamic, interactive 3D stimuli within an immersive environment that would be difficult to present using other means.ecologically valid assessment and rehabilitation scenarios.• The capacity to create more • The delivery of immediate performance feedbacks in a variety of forms and sensory modalities.• The provision of "cueing" stimuli or visualization tactics designed to help guide successful performance to support an error-free learning approach.• The capacity for complete performance capture and the availability of a more naturalistic/intuitive performance record for review and analysis.• The capacity to pause assessment, treatment and training for discussion and/or integration of other methods.• The design of safe testing and training environments that minimize the risks due to errors.• The capacity to improve availability of assessment and rehabilitation by persons with sensorimotor impairments via the use of adapted interface devices and tailored sensory modality presentations built into VE scenario design.• The introduction of "gaming" features into VR rehabilitation scenarios as a means to enhance motivation.• The integration of virtual human representations (avatars) for systematic applications addressing social interaction.• The potential availability of low cost libraries of VEs that could be easily accessed by professionals.• The option for self-guided independent testing and training by clients when deemed appropriate.In summary, VR provides a new human-computer interaction paradigm in which users are no longer simply external observers of images on a computer screen but are active participants within a computer-generated three-dimensional virtual world -8: in thThis outline better clarifies the possible role of VR in rehabilitation: a communication interface based on interactive 3D visualization, able to collect and integrate different inputs and data sets in a single real-like experience ,11.presence [This is possible because the key characteristic of VR, differentiating it from other media or communication systems, is the sense of presence , usuallypresence , or the presence .This feeling is theorized to contribute to the efficacy of VR as rehabilitation tool: the successful use of VR exposure therapy for phobias -18, postThanks to presence, not only knowledge acquisition is possible in VR, but also this acquired knowledge can be transferred in a real environment ,27. ThisNow, the challenge within this area is the creation of new paradigms . As cleaWithin this context we propose to investigate the impact of VR on rehabilitation and subjective experience from a theoretical perspective that stresses the active role of individuals in interacting with their natural and cultural environment ,30. In tWhat is presence? Answering to this question is not a simple task ,31,32. Iperceptual illusion of non-mediation [A first group of researchers describes the sense of presence as a function of our experience of a given medium -41. The ediation , produceThis approach, however, does not address some broader questions: What is presence for? Is it a specific cognitive process? What is its role in our daily experience?To answer these questions a second group of researchers considers presence as a neuropsychological phenomenon, evolved from the interplay of our biological and cultural inheritance, whose goal is the control of agency ,12,42-49Within this paper, we will support the second vision, trying to link it with the outcome of the first one . Here, pTo fully understand the key ideas behind this vision, three points are critical:– Presence has a simple but relevant role in our everyday experience: the control of agency through the unconscious separation of "internal" and "external". The meaning of "internal" and "external" is not related only to the body but also to the social and cultural space (situation) in which the self is in;– The presence-as-process (the separation mechanism) produces, but it is different from the presence-as-feeling (the experienced level of presence);– The presence-as-feeling is experienced indirectly by the self through the characteristics of action and experience. In fact the self perceives directly only the variations in level of presence-as-feeling: breakdowns and optimal experiences.distal attribution – the referencing of our perception to an external space beyond our boundaries – and effectively control its agency.First, presence is described here as a defining feature of self and it is related to the evolution of a key feature of any central nervous system: the embedding of sensory-referred properties into an internal functional space. As noted by Waterworth and Waterworth , the appIn this vision it is important to distinguish between presence-as-process and presence-as-feeling. The presence-as-process is the continuous activity of the brain, organized around the three functionally and phylogenetically different layers discussed in the next paragraph, in separating "internal" and "external" within different kinds of afferent and efferent signals.A critical point here is to explain why we need to introduce a new cognitive process – presence-as-process – to monitor our activity. The answer comes from a recent paper by de Vignemont and Fourneret . These aHowever, both the neuroscience of action and the neuropsychology of schizophrenia are countering his position . For insAs clarified by Russell : "Actionbeing-in-the-world" In fact, a higher level of presence-as-feeling is experienced by the self as a better quality of action and experience [being-in-the-world when its level is modified. More in detail, the self perceives directly only the variations in the level of presence-as-feeling: breakdowns and optimal experiences.For this reason, the presence-as-feeling (level of presence) is not separated from the experience of the subject but it is related to the quality of our actions. It corresponds to what Heidegger  defined perience ,56. HoweThe process of adaptation to the natural environment provided humans with specific biological features, such as the upright position, the opposing thumb and the increase in brain mass that allowed survival and reproduction in any environmental niche. At the same time, by means of the differential investment of attention and psychic resources, the individual selects and organizes the information acquired from his/her context according to an emergent, autonomous criterion: the quality of experience ,58. In oOn one side we have optimal experiences. According to Csikszentmihalyi ,60, indibreakdowns: a breakdown occurres when, during our activity, an object or an environment becomes part of our consciousness. If this happens, we shift our attention from action to the object/environment to cope with it: e.g., when a wall stops our movement.On the other side we have breakdowns. Winograd and Flores  refer toWhy do we experience these breakdowns? Our hypothesis is that breakdowns are a sophisticated evolutionary tool used by the presence-as-process to control the quality of experience: the more the breakdown, the less is the level of presence-as-feeling, the less is the quality of experience, and the less is the possibility of surviving in the environment.The importance of breakdowns for understanding presence is well reflected by Slater's concept of "break in presence" (BIP) : a breaktotal response  to being in a place, and to being in a place with other people. The 'sense of being there' is just one of many signs of presence – and to use it as a definition or a starting point is a category error: somewhat like defining humor in terms of a smile" (p. 7).The answers to these questions are related to another important point of our vision: the meaning of "internal" and "external". In our vision, the boundaries are not only physical and related to our body (being there), but also social and cultural (making sense there). As underlined by Slater , PresencIf, in relatively simple organisms, this separation involves only a correct coupling between perceptions and movements, in humans it also implies the relation of the subject with a social and cultural space ,64. In f– it is also "external" to the subject what is not related to his/her activities, interests and values.– to be more "present" in the situation  defined by a symbolic system, the user has to be aware of its meaning. Only "making sense there", the user really experiences a full sense of presence. In giving sense to a situation an important role is usually played by narratives ,65.To make these concepts clearer an example may help. I'm in a restaurant for a formal dinner with my boss and some colleagues, but I don't know how to use the many different strange forks I have around my dish. In this situation I'm physically there, but the lack of knowledge puts me outside, at least partially, from the social and cultural space of the "formal dinner". The result is a limitation in my agency: I don't use the forks to avoid mistakes. This example shows clearly how both physical boundaries  and social and cultural boundaries have a strong influence on the possibility of action and the quality of experience of the subject.At this point our conclusions are:In the real world, the feeling of presence is not the same in all the situations but can be different in relation to the characteristics of the social and cultural space the subject is in. For instance, if I'm attending a lesson in university, my level of presence can be lower or higher in relation to the interest I have in the topic discussed. If the lesson is totally boring I can be "absent" : in absence attention is mostly directed towards internally-generated scenarios (in imagination) which are not currently present in the world [• he world . The rol• There are some exceptional situations in real life in which the activity of the subject is characterized by a higher level of presence. In these situations the subject experiences a full sense of control and immersion. When this experience is associated to a positive emotional state, it can create an optimal experience, usually defined "flow". An example of flow is the case where a professional athlete is playing exceptionally well (positive emotion) and achieves a state of mind where nothing else matters but the game (high level of presence).At this point we have defined what presence is and its role in the human experience. However, there is yet another open question: how can we achieve high level of presence-as-feeling? To answer this question we have to analyze the neuropsychological nature of presence-as-process.Even if presence is a unitary feeling, on the process side it can be divided in three layers/subprocesses (for a broader and more in-depth description see ). These The proto self: a coherent collection of neural patterns that map, moment by moment, the physical state of the organism;• core self: a transient entity which is continuously generated through encounters with objects• The extended self: a systematic record of the more invariant properties that the organism has discovered about itself.• The proto presence (self vs. non self), core presence , and extended presence .Each layer of presence solves a particular facet of the internal/external world separation and it is characterized by specific properties. In particular we can make conceptual distinctions between proto presence as an embodied presence related to the level of perception-action coupling (self vs. non-self). The more the organism is able to couple correctly perceptions and movements, the more it differentiates itself from the external world, thus increasing its probability of surviving. Proto presence is based on proprioception and other ways of knowing bodily orientation in the world. In a virtual world this is sometimes known as "spatial presence" and requires the tracking of body parts and appropriate updating of displays.More precisely we can define Core presence can be described as the activity of selective attention made by the self on perceptions : the more the organism is able to focus on its sensorial experience by leaving in the background the remaining neural processes, the more it is able to identify the present moment and its current tasks, increasing its probability of surviving. Core presence is based largely on vividness of perceptible displays. This is equivalent to "sensory presence" (e.g. in non-immersive VR) and requires good quality, preferably stereographic, graphics and other displays features.extended presence is to verify the significance to the self of the events experienced in the external world . The more the self is present in significant experiences, the more it will be able to reach its goals, increasing the possibility of surviving. Extended presence requires intellectually and/or emotionally significant content.Finally, the role of In humans the sense of presence-as-feeling is a direct function of these three layers: the more they are able to separate between "internal" and "external", the more is the feeling of presence, the better is the quality of action and experience.A corollary of the proposed vision is critical for our goals: it is possible to design mediated situations that elicit exceptionally high presence -69.In particular, here we argue that virtual reality is the medium able to support the highest level of presence because it can trigger at the same time all the three layers discussed before. To understand this point and in particular the difference between VR and other media, here are some examples :• In reading an engrossing book while sitting in a comfortable, safe place, extended presence will be engaged by media (engagement) but the other layers will not be involved.• In looking at a movie, you can activate a high level of core presence (vividness), a high level of extended presence (engagement) but no proto presence .• In experiencing an interesting immersive VR experience, proto , core presence (vividness) and extended presence (engagement) will be activated by the medium.• In an immersive VR experience, if you are pre-occupied with personal worries and the mediated content is not very engaging, proto  and core presence (vividness) will be invoked by the medium, but not extended presence.perceptual illusion of non-mediation [The possibility of activating all the three layers at the same time reduces the occurrence of breakdowns. As suggested by Marsh and colleagues , one of ediation , produceTo achieve an optimal experience, however, a next step is required: the highest level of presence has to be linked to a positive emotional experience.Csikszentmihalyi ,70 definStarting from this definition, different authors tried to define flow in an operational way. For Ghani and Deshpande  the two In this paper we suggest that VR is the preferred medium for the activation of the flow experience. A number of recent experimental results might be considered to foster this vision. A first support comes from the work of Hoffman and his group in the treatment of chronic pain -25. Few A second set of results comes from the work of Gaggioli ,30. GaggA final result, is the preference of phobic patients for VR vs. traditional treatments as showed by two studies from García-Palacios and colleagues ,76. In tAuthentic rehabilitation implies the active participation of the patient in the cultural context, their exposure to opportunities for action and development, their freedom to select opportunities they perceive as the most challenging and meaningful ones for the subject ,77. Follcultivation; it fosters the growth of complexity not only in the performance of flow activities but in individual behavior as a whole." (p. 28).Optimal experiences promote individual development. As underlined by Massimini and Delle Fave,  "To repltransformation of flow. Where possible, they keep cultivating former flow activities. Otherwise, as often happens, they manage to identify new and unexpected sources of concentration and involvement, sometimes in areas very different from their previous interests.This process can be activated also after a major trauma. As noted by Delle Fave , to copeThe vision behind the concept of transformation of flow is the one from "Positive Psychology" . AccordiIn this area VR may offer a critical advantage: the possibility for the patient to manage successfully in a VE a problematic situation related to his/her disturbance. Using VR in this way, the patient is more likely not only to gain an awareness of his/her need to do something to create change but also to experience a greater sense of personal efficacy . This apIn another case study, Riva  tested tempowerment is considered a multi-faceted construct reflecting the different dimensions of being psychologically enabled, and is conceived of as a positive additive function of the following three dimensions [Generally, these techniques can be used as triggers for a broader empowerment process within the flow experience induced by a high sense of presence. In psychological literature mensions :perceived control: includes beliefs about authority, decision-making skills, availability of resources, autonomy in the scheduling and performance of work, etc;– perceived competence: reflects role-mastery, which besides requiring the skillful accomplishment of one or more assigned tasks, also requires successful coping with non-routine role-related situations;– goal internalization: this dimension captures the energizing property of a worthy cause or exciting vision provided by the organizational leadership.– Virtual reality may be considered the preferred environment for the empowerment process, since it is a special, sheltered setting where patients can start to explore and act without feeling threatened. In this sense the virtual experience can be described as an "empowering environment" that rehabilitation provides to patients: nothing the patient fears can "really" happen to them in VR. With such assurance, they can freely explore, experiment, feel, live, and experience feelings and/or thoughts. VR thus becomes a most useful intermediate step between the therapist's office and the real world. Within this frame, therapists are encouraged to explore whether and how VR induced optimal experiences may facilitate recovery .The use of VR as an empowerment tool was recently tested in the support of HIV-AIDS patients . The sysIn terms of emotional impact, the participants found their experience with the system mostly encouraging, particularly the narratives relating to adjustment and coping.VR is an advanced communicative interface based on interactive 3D visualization. Its simulative capabilities allow for the precise presentation and control of dynamic multi-sensory 3D stimulus environments, as well as providing advanced methods for recording behavioral responses.presence [However VR, differently from other media, can induce a high sense of presence , usuallypresence , or the presence . Thanks This paper introduces a bio-cultural theory of presence linking the state of optimal experience, defined as "flow", to a virtual reality experience.The key ideas behind the proposed vision of presence are:– Presence has a simple but critical role in our everyday experience: the control of agency through the unconscious separation of "internal" and "external". The meaning of "internal" and "external" is not related only to the body but also to the social and cultural space (situation) in which the self is in.– The presence-as-process (the separation mechanism) produces, but is different from, the presence-as-feeling (the experienced level of presence);– The presence-as-feeling is experienced indirectly by the self through the characteristics of action and experience. In fact the self perceives directly only the variations in level of presence-as-feeling: breakdowns and optimal experiences;proto presence (self vs. non self), core presence , and extended presence .– The presence-as-process can be divided in three different layers/subprocesses. They are phylogenetically different, and strictly related to the three levels of self identified by Damasio: A corollary of the proposed vision is the possibility to design mediated situations that elicit exceptionally high presence. In particular, here we argued that virtual reality is the medium able to support the highest level of presence because it can activate at the same time all the three layers. To achieve an optimal experience (flow), however, a next step is required: the highest level of presence has to be linked to a positive emotional experience.transformation of flow. In this view, VR can be used to trigger a broad empowerment process within the flow experience induced by a high sense of presence. Linking this possibility to its simulative capabilities may transform VR in the ultimate rehabilitative device.The link between presence, flow and VR suggests the possibility of using it for a new breed of rehabilitative applications focused on a strategy defined as Applications of VR in rehabilitation include the following disturbances: memory disorders, planning and motor disabilities, executive functions and spatial knowledge disabilities. For a full review, see Morganti . HoweverEven if the significant advances in computer and graphic technology drastically improved the characteristics of a typical VE, VR is still limited by the maturity of the systems available. Today, no off-the-shelf solutions are available. So, the set up of a VR system usually requires much patience for dealing with conflicting hardware or lacking drivers. Nearly every VR system requires a dedicated staff or at least a computer technician to keep the system running smoothly.Moreover, introduction of patients and clinicians to VEs raises particular safety and ethical issues. In fact, despite developments in VR technology, some users still experience health and safety problems associated with the use of immersive headsets. Generally, for a large proportion of VR users these effects are mild and subside quickly.Further, even if the clinical rationale behind the use of VR in rehabilitation is now clear, much of this research growth has been as feasibility studies and pilot trials. Hence, there is still limited convincing evidence coming from controlled studies.Why there are so few controlled trials in VR research? The possible answers are two.First, there is a lack of standardization in VR devices and software. To date, very few of the various VR systems available are interoperable. This makes difficult their use in contexts other than those in which they were developed. Second, there is a lack of standardized protocols that can be shared by the community of researchers.Clearly, building new and additional virtual environments is important so therapists will continue to investigate applying these tools in their day-to-day clinical practice. In fact, in most circumstances, the clinical skills of the rehabilitator remain the key factor in the successful use of VR systems.Future research should explore how to develop VR environments able to provide the degree of challenge required to elicit the optimal experience. Further, research should also deepen the analysis of the link between cognitive processes, motor activities, presence and flow. This will allow a new generation of VEs in which the added value of VR is not only simulation and control.


The evidence base of clinical interventions in paediatric hospitals of developing countries has not been formally assessed. We performed this study to determine the proportion of evidence-based therapeutic interventions in a paediatric referral hospital of a developing countryThe medical records of 167 patients admitted in one-month period were revised. Primary diagnosis and primary therapeutic interventions were determined for each patient. A systematic search was performed to assess the level of evidence for each intervention. Therapeutic interventions were classified using the Ellis score and the Oxford Centre for Evidence Based Medicine Levels of EvidenceAny dehydration due to diarrhoea (59 cases) and pneumonia (42 cases) were the most frequent diagnoses. Based on Ellis score, level I evidence supported the primary therapeutic intervention in 21%, level II in 73% and level III in 6% cases. Using the Oxford classification 16%, 8%, 1% and 75% therapeutic interventions corresponded to grades A, B, C, and D recommendations, respectively. Overall, according to Ellis score, 94% interventions were evidence based. However, out of the total, 75% interventions were based on expert opinion or basic sciences. Most children with mild to moderate dehydration (52 cases) were inappropriately treated with slow intravenous fluids, and most children with non-complicated community acquired pneumonia (42 cases) received intravenous antibioticsMost interventions were inappropriate, despite the availability of effective therapy for several of them. Diarrhoeal dehydration and community acquired pneumonia were the most common diagnoses and were inappropriately managed. Existing effective interventions for dehydration and pneumonia need to be put into practice at referral hospitals of developing countries. For the remaining problems, there is the need to conduct appropriate clinical studies. Caution must be taken when assigning the level of evidence supporting therapeutic interventions, as commonly used classifications may be misleading Previous studies have shown that medical interventions based on scientific evidence range from 10 to 80% . These sIn paediatric practice, the proportion of evidence based interventions reported ranges from 75 to 91% . These sThe assessment of quality of care at referral level for severely ill children is an important component of the efforts for reducing the child mortality rate in poor countries, for reducing the burden on health systems and for investing money in favour of high priority health interventions ,10. ThusReferral care provided to children hospitalized in the paediatric department of the Instituto Especializado de Salud del Niño (IESN) was assessed for evidence base. IESN is a national paediatric hospital with more than 500 inpatient beds, serving mostly patients from deprived socioeconomic areas of Lima and inner cities of the country.The clinical records of 195 children aged 1 month through 16 years old and hospitalized during January 2003 were initially revised.One of the investigators (NYC) assessed the clinical records of children and assigned to each one a primary diagnosis and one or more primary therapeutic interventions, on the basis of the main clinical features and/or definitive diagnostic laboratory aids. Patients in whom a primary diagnosis was not possible to determine or those without a clear diagnosis were excluded.The primary intervention was defined as the treatment or other manoeuvre that represented the most important attempt to cure, alleviate, or care for the patient in respect of his or her primary diagnosis .To determine the level of evidence for each primary intervention, Cochrane reviews were searched. If there was not such a systematic review, a search through PubMed (MEDLINE) was performed by one of the investigators (NYC). All Cochrane reviews were searched through their own search tools. For PubMed, the period of search was 1966 through 2002. The key words used included those related to the primary diagnosis . Limits: "Title", "All child: 0–18 years". Publication type was sequentially searched for "Practice Guideline", "Meta-Analysis", "Randomized Controlled Trial" and "Review". Articles in English or Spanish were included. The National Guideline Clearinghouse was additionally visited for additional references. Published recommendations for judging the quality of guidelines were used for deciding the selection of the guidelines . In addiThe level of evidence assigned to interventions was based on Ellis score  and the Oxford Centre for Evidence Based Medicine Levels of Evidence  ,12. ElliAs a next step to our study, we planned the dissemination of the results among the hospital policy makers and the suggestion of corrective courses of action for those interventions needing improvement.Results of the search strategies for PubMed are included as an appendix See . A guideOverall, one hundred and ninety five clinical records were revised. Twenty eight clinical records were excluded because of undefined primary diagnosis Table . One hunOut of 167 primary interventions, 21% were supported by level I evidence, 73% were level II, and 6% were ranked as level III, according to Ellis classification .Most assessed interventions were considered as level II Table . DiarrhoOvertly unsubstantiated therapy according to Ellis classification is shown in Table Comparison of grade of recommendation of the prescribed intervention with the appropriate one is shown in Table In this study 94% of therapeutic interventions were evidence-based by Ellis score. It may seem encouraging that more than 90% of therapeutic decisions in a referral paediatric hospital of a developing country are evidence based. However, the level II of evidence from Ellis includes interventions based in cohort studies, case-control studies, case series, expert's opinion, and even those based in basic sciences. We attempted therefore to classify the primary interventions according to more specific criteria. Using the Oxford classification, 75% of therapeutic interventions were based in expert opinions or in basic sciences (Grade of Recommendation D).Some limitations of our study include the possible author' bias when assigning the primary diagnosis and primary intervention. The assignment of diagnosis by the clinician may have been influenced by both the choice of treatment and the available evidence. Only one of us classified the primary intervention. In addition, we evaluated only a primary intervention for a single primary diagnosis. Actually, many patients had more than one diagnosis and obviously more than one therapeutic intervention.We used as evidence-base for rating the interventions assessed in our study, guidelines and evidence-based resources published in the developed world. This raises the issue of whether they are fully applicable to our setting. The most prevalent conditions found were diarrhoeal dehydration and community acquired pneumonia. For both of them we used British produced guidelines ,21 becauRegarding community acquired pneumonia, the British guidelines recommend antibiotic treatment for all children with pneumonia, due to the difficulties in identifying the aetiology, and they also specify criteria for hospitalization . These rWe acknowledge that the evidence derived from studies performed in developed countries should be translated with caution to developing settings. However, when the native research is scarce or of low quality, we think that the transfer of knowledge from the developed countries is an acceptable approach, as far as the particular characteristics of patients in developing countries are considered on an individual basis.Dehydration due to diarrhoea and pneumonia were the most frequent diagnoses. Oral rehydration for diarrhoeal dehydration and antibiotics for pneumonia are considered as interventions with sufficient evidence for implementing them widely ,10. In oIn addition to their enormous potential for saving lives, outpatient antibiotic therapy for pneumonia and outpatient oral rehydration can drastically reduce the rate of hospitalizations, the hospital stay, the hospital mortality rate, and the costs incurred. At our hospital, the mean stay time for hospitalized children is 4.7 days, and the mean crude mortality rate is 3.6%. We estimated the cost of managing hospitalized children with pneumonia and diarrhoeal dehydration as US$ 10.6/day and US$ 8.6/day, respectively. These costs are referred only to hospital bed and laboratory tests. A substantial amount of money could be saved treating these conditions on an outpatient basis.We planned the dissemination of our results among the hospital policy makers. The tools that will be suggested for improving the standards include the development and systematic application of locally produced guidelines and/or the adaptation of published guidelines. A useful alternative that has been experienced for several years at our inpatient ward unit is to make available personal computers connected to Internet for attending physicians, residents and interns, and to encourage the use of online evidence-based resources. This last alternative may work better, particularly where there are motivated physicians who are able to lead the efforts for improving the health care standards. However, the ultimate decision to systematically introduce and monitor the suggested changes will rest on hospital managers. Such changes should also depend on taking into account the role of several other determinants of the clinical decision making by individual practitioners, such as continuous training, motivation, time, availability of drugs, equipments and supplies, supervision, and long-term health system strengthening strategies.Caution must be taken when assigning the level of evidence that supports therapeutic interventions, as commonly used classifications may be misleading. Existing effective interventions for dehydration and pneumonia are not being implemented in developing countries, even at referral level, and thus there is the need to change the current medical interventional behaviours. For the remaining diagnoses, the majority of assessed interventions were based on weak or non-existent evidence, highlighting the need to conduct appropriate clinical studies.The author(s) declare that they have no competing interests.NYC and LH conceived and designed the study. All authors analyzed and interpreted the data and contributed to earlier drafts of the manuscript.The pre-publication history for this paper can be accessed here:"Appendix: Literature search results" details of the Medline (PubMed) search strategy and of the results are provided in this additional file. The selected articles are in black, bold characters. In addition, the search results from the National Guideline Clearinghouse website are provided for painful crisis in sickle cell anaemia.Click here for file


A number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations.Six prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches.There was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered . None of the systems was considered usable for all of the target groups we considered . The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised.All of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings. In 1979 the Canadian task Force on the Periodic Health Examination published one of the first efforts to explicitly characterise the level of evidence underlying healthcare recommendations and the strength of recommendations -28.The original approach used by the Canadian Task Force was based on study design alone, with randomised controlled trials (RCTs) being classified as good (level I) evidence, cohort and case control studies being classified as fair (level II) evidence and expert opinion being classified as poor (level III) evidence. The strength of recommendation was based on the level of evidence with direct correspondence between the two; e.g. a strong recommendation (A) corresponded to there being good evidence. A strength of the original Canadian Task Force approach was that it was simple; the main weakness was that it was too simple. Because of its simplicity, it was easy to understand, apply and present. However, because it was so simple there were many implicit judgements, including judgements about the quality of RCTs, conflicting results of RCTs, and convincing results from non-experimental studies.For example:• Should a small, poorly designed RCT be considered level I evidence?• Should RCTs with conflicting results still be considered level I evidence?• Should observational studies always be considered level II evidence, regardless of how convincing they are?The original approach by the Canadian Task Force also did not include explicit judgements about the strength of recommendations, such as how trade-offs between the expected benefits, harms and costs were weighed and taken account of in going from an assessment of how good the evidence is to determining the implications of the results for practice.The GRADE Working Group is an informal collaboration of people with an interest in addressing shortcomings such as these in systems for grading evidence and recommendations. We describe here a critical appraisal of six prominent systems and the results of the critical appraisal.We selected systems for grading the level of evidence and the strength of recommendations that we considered prominent and that included features not captured by other prominent systems. These were selected based on the experience and knowledge of the authors through informal discussion. A description of the most recent version (as of summer 2000) of each of these systems (Appendix 1 to 6), was prepared by one of the authors familiar with the system, and used in this exercise. The following six systems were appraised: the American College of Chest Physicians  21], A, A21], AThese descriptions of the systems were given to the twelve people who independently appraised the six systems, all of the authors minus GEV appraised the six systems, three of the authors  appraised as a group and reported as one (see contributions). The 12 assessors all had experience with at least one system and most had helped to develop one of the six included systems. Twelve criteria described by Feinstein  provided1. To what extent is the approach applicable to different types of questions? -effectiveness, harm, diagnosis and prognosis 2. To what extent can the system be used with different audiences? -patients, professionals and policy makers 3. How clear and simple is the system? 4. How often will information not usually available be necessary? 5. To what extent are subjective decisions needed? 6. Are dimensions included that are not within the construct (level of evidence or strength of recommendation)? 7. Are there important dimensions that should have been included and are not? 8. Is the way in which the included dimensions are aggregated clear and simple? 9. Is the way in which the included dimensions are aggregated appropriate? 10. Are the categories sufficient to discriminate between different levels of evidence and strengths of recommendations? 11. How likely is the system to be successful in discriminating between high and low levels of evidence or strong and weak recommendations? 12. Are assessments reproducible? No training was provided and we did not discuss the 12 criteria prior to applying them to the six systems.Our independent appraisal of the six systems were summarised and discussed. The discussion focused on differences in the interpretation of the criteria, disagreement about the judgements that we made and sources of these disagreements, the strengths and weaknesses of the six systems, and inferences based on the appraisals and subsequent discussion.In order to identify important systems that we might have overlooked following our appraisal of these six systems we also searched the US Agency for Health Care Research and Quality (AHRQ) National Guidelines Clearing House for organisations that have graded two or more guidelines in the Clearing House using an explicit system . These sThere was poor agreement among the 12 assessors who independently assessed the six systems. A summary of the assessments of the sensibility of the six approaches to rating levels of evidence and strength of recommendation is shown in Table The poor agreement among the assessors likely reflects several factors. Some of us had practical experience using one of the systems or used additional background information related to one or more grading systems, and we may have been biased in favour of the system with which we were most familiar. Each criterion was applied to grading both evidence and recommendations. Some systems were better for one of these constructs than the other and we may have handled these discrepancies differently. In addition each criterion may have been assessed relative to different judgements about the evidence, such as an assessment of the overall quality of evidence for an important outcome (across studies) versus the quality of an individual study. Some of the criteria were not clear and were interpreted or applied inconsistently. For example, a system might be clear and not simple or visa versa. We likely differed in how stringently we applied the criteria. Finally, there was true disagreement.There was agreement that the OCEBM system works well for all four types of questions. There was disagreement about the extent to which the other systems work well for questions other than effectiveness. It was noted that some systems are not intended to address other types of questions and it is not clear that it is important that a system should address all four types of questions that we considered , although criteria for assessing individual studies must take this into account ,32.Most of us did not find that any of the systems are likely to be suitable for use by patients. Almost all agreed that the ACCP system was suitable for professionals and most considered that the USPSTF system was suitable for professionals. There was not much agreement about the suitability of any of the other systems for professionals or about the suitability of any of the systems for policy makers, although most assessed the USTFCPS system to be suitable for policy makers.There was no agreement that any of the systems are clear and simple, although USPSTF, ACCP and SIGN systems were generally assessed more favourably in this regard. It was generally agreed that the clearer a system was the less simple it was; e.g. the OCEBM system is clear but not simple for categorising the level of evidence. There was some confusion regarding whether we were assessing how clear and simple the system was to guideline developers (as some interpreted this criterion) or how clear and simple the outcome of applying the system was to guideline users (as others interpreted this criterion). Either way, the simpler a system is the less clear it is likely to be.Most of us judged that for most of the systems necessary information would not be available at least sometimes. The OCEBM system came out somewhat better than the other systems and lack of availability of necessary information was considered to be less of a problem for the USTFCPS system. However, the OCEBM and USTFCPS systems were considered by most to be missing dimensions which may, in part, explain why missing information was considered to be less of a problem. This would be the case to the extent the missing dimensions were the ones for which information would often or sometimes not be available. The dimension for which we considered that information would most often be missing was trade-offs; i.e. knowledge of the preferences or utility values of those affected. Additional problems were identified in relationship to complex interventions and counselling, particularly with the USTFCPS and USPSTF systems. It was pointed out that the USTFCPS system addressed this problem by including availability of information about the intervention as part of its assessment of the quality of evidence.Most of the systems were assessed to require subjective decisions at least to some extent. The OCEBM system again stood out as being assessed more favourably, although it may be related to omission of dimensions that require more subjective decisions. Judgement is clearly needed with any system. The aim should be to make judgements transparent and to try to protect against bias in the judgements that are made by being systematic and explicit.Inclusion of dimensions that are not within the constructs being graded was not considered a problem for most of the systems by most of us. Several people considered that it might be a problem for the USTFCPS and USPSTF systems. On the other hand, all of the systems were evaluated to be missing at least one important dimension by at least one person. The challenge of missing dimensions were considered less of a problem for the ACCP and ANHMRC systems. There was not agreement about any of the systems having a clear and simple approach to aggregating the dimensions, although this was considered to be less of a problem for the ACCP, SIGN and USTFCPS systems.There was also not agreement on the appropriateness of how the dimensions were aggregated. This was considered to be more of a problem for the ANHMRC and USTFCPS systems than the other four systems, all of which were considered to have taken an approach to aggregating the dimensions that was at least partially inappropriate by more than half of us.Most of us considered that most of the systems had sufficient categories, with the exception of the ANHMRC system. There was almost agreement that the USPSTF system has sufficient categories. We agreed that it is possible to have too many categories as well as too few, the OCEBM system being an example of having too many categories.There was not agreement that any of the systems are likely to discriminate successfully, although everyone thought that the ACCP, SIGN and USPSTF systems are somewhat to highly likely to discriminate. Lastly, we largely agreed that we were not sure how reproducible assessments are using any of the systems, although half of us considered that assessments using the ANHMRC system are unlikely to be reproducible and about 1/3 considered that assessments using the OCEBM and ACCP systems are likely to be reproducible.We identified 22 additional organisations that have produced 10 or more practice guidelines using an explicit approach to grade the level of evidence or strength of recommendations. Another 29 have produced between two and nine guidelines using an explicit approach. These systems include a number of minor variations of the six systems that we appraised in detail.There was generally poor agreement between the individual assessors about the scoring of the six approaches using the 12 criteria. However, there was general agreement that none of these six prominent approaches to grading the levels of evidence and strength of recommendations adequately addressed all of the important concepts and dimensions that we thought should be considered. Although we limited our appraisal to six systems all of the additional approaches to grading levels of evidence and strength of recommendations that we identified were, in essence, variations of the six approaches that we had critically appraised. Therefore we are confident that we did not miss any important grading systems available at the time when these assessments were undertaken.Based on discussions following the critical appraisal of these six approaches, we agreed on some conclusions:• Separate assessments should be presented for judgements about the quality of the evidence and judgements about the balance of benefits and harms.• Evidence for harms should be assessed in the same way as evidence for benefits, although different evidence may be considered relevant for harms than for benefits; e.g. local evidence of complication rates may be considered more relevant than evidence of complication rates from trials for endarterectomy.• Judgements about the quality of evidence should be based on a systematic review of the relevant research.• Systematic reviews should not be included in a hierarchy of evidence (i.e. as a level or category of evidence). The availability of a well-done systematic review does not correspond to high quality evidence, since a well-done review might include anything from no studies to poor quality studies with inconsistent results to high quality studies with consistent results.• Baseline risk should be taken into consideration in defining the population to whom a recommendation applies. Baseline risk should also be used transparently in making judgements about the balance of benefits and harms. When a recommendation varies in relationship to baseline risk, the evidence for determining baseline risk should be assessed appropriately and explicitly.• Recommendations should not vary in relationship to baseline risk if there is not adequate evidence to guide reliable determinations of baseline risk.Based on discussions of the strengths and limitations of current approaches to grading levels of evidence and the strength of recommendations, we agreed to develop an approach that addresses the major limitations that we identified. The approach that the GRADE Working Group has developed is based on the discussions following the critical appraisal reported here and a pilot study of the GRADE approach . Based o and a discussion list.The GRADE Working Group has continued to grow as an informal collaboration that meets one or two times per year. The group maintains web pages DA has competing interests with the US Preventive Services Task Force (USPSTF), PAB has a competing interest with the US Task Force on Community Preventive Services (USTFCPS), GHG and HS have competing interests with the American College of Chest Physicians (ACCP), DH, SH and DO'C have competing interests with the Australian National Health and Medical Research Council (ANHMRC), BP has competing interests with the Oxford Centre for Evidence-Based Medicine (OCEBM). Most of the other members of the GRADE Working Group have experience with the use of one or more systems of grading evidence and recommendations.DA, PAB, ME, SF, GHG, DH, SH, AL, DO'C, ADO, BP, HS, TTTE, GEV & JWW Jr as members of the GRADE Working Group have contributed to the preparation of this manuscript and the development of the ideas contained herein, participated in the critical assessment, and read and commented on drafts of this article. GHG and ADO have led the process. GEV has had primary responsibility for coordinating the process.The pre-publication history for this paper can be accessed here:American College of Chest Physicians (ACCP), a brief description of the ACCP approach.Click here for fileAustralian National Health and Medical Research Council (ANHMRC), a brief description of the ANHMRC approach.Click here for fileOxford Centre for Evidence-based Medicine (OCEBM), a brief description of the OCEBM approach.Click here for fileScottish Intercollegiate Guidelines (SIGN), a brief description of the SIGN approach.Click here for fileU.S. Preventive Services Task Force (USPSTF), a brief description of the USPSTF approach.Click here for fileU.S. Task Force on Community Preventive Services (USTFCPS), a brief description of the USTFCPS approach.Click here for file


Our goal was to quantify the evidence that is available to the physicians of a pediatric emergency department (PED) in making treatment decisions. Further, we wished to ascertain what percentage of evidence for treatment provided in the PED comes from pediatric studies.We conducted a retrospective chart review of randomly selected patients seen in the PED between January 1 and December 31, 2002. The principal investigator identified a primary diagnosis and primary intervention for each chart. A thorough literature search was then undertaken with respect to the primary intervention. If a randomized control trial (RCT) or a systematic review was found, the intervention was classified as level I evidence. If no RCT was found, the intervention was assessed by an expert committee who determined its appropriateness based on face validity . These interventions were classified as level II evidence. Interventions that did not fall into either above category were classified as level III evidence.Two hundred and sixty-two patient charts were reviewed. Of these, 35.9% did not receive a primary intervention. Of the 168 interventions assessed, 80.4% were evidence-based (level I), 7.1% had face validity (level II) and 12.5% had no supporting evidence (level III). Of the evidence-based interventions, 83.7% were supported by studies with mostly pediatric patients.Our study demonstrates that a substantial proportion of PED treatment decisions are evidence-based, with most based on studies in pediatric patients. Also, a large number of patients seen in the PED receive no intervention. The term "evidence-based medicine" has become a catchphrase for the twenty-first century. Physicians are being called upon to justify their treatment decisions with valid, up-to-date evidence. There have been attempts within many areas of medicine to quantify the evidence that is available to, and used by, the physicians of that discipline. The first such study, conducted by Ellis et al  in 1995,Within pediatrics, there have been three such studies. Moyer et al  examinedThere is a need to investigate whether and to what extent the treatment of pediatric emergency patients is based on evidence. The wide variety of ailments seen within this area should offer a more comprehensive assessment of the overall state of evidence within pediatrics – encompassing surgical, outpatient and inpatient medicine, across all ages. At the same time, we need to gain an understanding of the quality and quantity of evidence in existence to support pediatric clinical decisions. How often is adult evidence generalized to the treatment of children? This is an important issue given that children differ from adults in terms of their physiology, biology, and developmental processes. There is limited health information available from research in children and, as a result, health care providers repeatedly turn to other sources of evidence, such as adult studies, to assist with managing health care issues in children .In the present study, we sought to identify the extent to which evidence exists for use within the unique environment of the pediatric emergency department. A secondary objective was to determine the amount of evidence, applied to pediatrics, that is derived from adult studies. We hypothesized, a priori, that this was a common practice within pediatrics. Our hope was to identify areas within the body of evidence for pediatric emergency medicine that require future study.This study was approved by the Health Research Ethics Board at our institution.We conducted a retrospective chart review of randomly selected patient charts based on patients seen in a Pediatric Emergency Department at a tertiary care facility during the 2002 year (January 1 to December 31). The sampling frame was created using the Emergency Department Information System (EDIS) database. The following variables: hospital record number, patient hospital number, patient visit number, date, time, and age made up the sampling frame. A random sample of patient visits were then generated using random-number computer software (S-plus 6.0). Any individual patient was included only once in the sample. A sample size calculation of 295 showed that a minimum of 5.6% precision would be achieved for any given estimate of percentage. The primary investigator (PI) retrospectively reviewed the sampled records and, using a standardized abstraction form, identified a primary diagnosis and primary intervention for each based on the definition set out by Ellis et al. That is, the primary diagnosis was defined as "the disease, syndrome or condition entirely or, if there were several diagnoses, [the diagnosis] most responsible for the patient's [visit to the pediatric emergency department]"  his or her primary diagnosis" . This waA thorough literature search was then undertaken with respect to the primary interventions. Current (at the time of searching) MEDLINE, EMBASE and the Cochrane Library databases were searched comprehensively to determine if any evidence existed with respect to the individual primary interventions. Abstracts and references of relevant scientific papers were also searched. Local experts were contacted and asked if they were aware of RCTs in existence that had not yet been published or that were not found through the search strategy .The primary interventions were then divided into the following categories that have been used in previous research -4.The abstract or paper demonstrated that this intervention had been evaluated in an RCT, systematic review or meta-analysis. No attempt was made to rate the quality of the study or its findings as this did not relate directly to our goal of determining the quantity of evidence available to PED physicians.No abstract or paper could be found that investigated this intervention in an RCT, systematic review or meta-analysis. However, a committee determined there to be convincing non-RCT evidence for this intervention. That is, the face validity of such an intervention was considered to be so great that randomized trials were unanimously judged to be unnecessary or, if a placebo were to be involved, potentially harmful . The coNo abstract or paper could be found that involved this intervention in an RCT, systematic review or meta-analysis and the committee did not unanimously agree that there was convincing non-RCT evidence to support it.Level I evidence was further subdivided into those studies that included a majority  of pediatric patients and those that did not. Systematic reviews and meta-analyses were included under the "pediatric studies" grouping if they included one RCT of predominantly pediatric patients. Every attempt was made to search out pediatric studies, even if an adult RCT was found first for any given intervention.A further category was included of those patients who received no primary intervention. This category included patients who were discharged from the pediatric emergency department after receiving only reassurance and/or advice. The percentage of primary interventions which were included in each category was calculated as was the percentage of primary interventions deemed to be from pediatric and adult studies, along with their exact 95% confidence intervals [From the 18,855 patient visits to the PED during the year January 1, 2002 and December 31, 2002, a random sample of 295 patient charts was computer generated, representing approximately 1% of patient visits. Any individual patient was only included once in the sample. Of these 295 charts, 272 (92.2%) were available to be viewed. The other 23 were missing from the chart area. Of the 272, 10 were discarded because of insufficient information to determine a primary diagnosis and intervention (i.e. the emergency record was missing from the chart). Ninety-four patients received no intervention, representing 35.9% of our sample (95% CI 29.9 to 41.9).Table The most common interventions were analgesics for musculoskeletal pain (17/135 = 12.6%) and inhaled beta-agonist for asthma exacerbations (12/135 = 8.9%); both were supported by pediatric RCTs. Out of the 135 patients who received interventions that were based on level I evidence, 113 patients' interventions  were supported by pediatric RCTs.The interventions unanimously determined, by the five person committee of pediatric emergency department (PED) physicians, to be worthy of level II evidence status are listed in Table Including level I and II evidence, 87.5% (CI 81.3 to 91.9) of interventions can be considered evidence-based.Those interventions which were not considered to be evidence-based (level III evidence) are listed in Table The results of this study are important for several reasons. First, we have shown that a substantial proportion of the children seen in the PED do not receive any treatment or intervention. In this sample of 1.4% of all PED visits over one year, 35.9% of children received only reassurance or advice. One of the most common reasons that children were seen in the PED without intervention was upper respiratory tract infection (26%) (data not shown), usually associated with fever. This leaves one wondering whether parental education about benign fever and supportive care for upper respiratory tract infection could help shorten PED wait times. Studies on the effect of education in this area could have very important beneficial effects in terms of costs to the healthcare system.Our study has revealed that a great majority of interventions in the PED are based on pediatric studies (83.7%). Although this is a positive finding, there are several areas of pediatric medicine, such as migraine treatment and fracture management, for which treatment continues to be based on adult studies. Ideally, we would like to see 100% of pediatric interventions based on pediatric studies.A comparison of our results to similar studies is shown in Table If one compares what was categorized as level II evidence by Moyer's study with our study, there are important differences. For example, interventions such as IV rehydration and antibiotics for urinary tract infection, categorized as level II evidence in the Moyer study, were classified as level I evidence in our study. This may be a reflection of the four year difference between studies and the accumulation of research evidence during that time. Alternatively, our study involved not only inpatients, but also patients discharged directly from the PED. Therefore, interventions which have been better studied such as antibiotics for otitis media  and glucThe limitations of this study are similar to those found with the studies previously carried out in other disciplines -5. CertaAnother potential limitation is derived from the inclusion of interventions under the category of level I evidence without interpreting the results of the RCT. This was done in order to compare our results to the original study by Ellis which, likewise, did not comment on the content of level I evidence.Finally, while this was a random sample, it represented only 1% of all PED patients seen over the course of a year, therefore it provides an estimate of the proportion of PED treatments that are evidence-based but may not be representative of the full spectrum of different presenting conditions.There were many strengths of our study when compared to previous studies. The relative inflexibility of our expert panel in committing interventions to the level II evidence group is one positive feature. Although there must be subjectivity in a process such as this, our team was quite rigorous in applying the criteria for this category and limited what must be assigned to this group accordingly. This is reflected in the lower percentage of interventions ranked as level II evidence by our committee (5% compared to 44% in the Moyer study [We have shown that a surprisingly large proportion of patients seen in the pediatric emergency department (35.9%) receive no intervention, while approximately half of patients in this setting receive interventions that are based on evidence from randomized controlled trials (level I evidence). The vast majority of these are derived from randomized control trials containing a majority of pediatric patients. Although our study speaks positively of the breadth of knowledge within the unique environment of the pediatric emergency department, we have also identifed numerous important areas for future study within pediatric research.The author(s) declare that they have no competing interests.TK conceived the study. KW, KC, NW, LH and TK designed the study. KW and KC obtained research funding. KW and KC supervised and conducted the data collection. KW managed the data, including quality control. NW provided statistical advice on study design and analyzed the data. KW drafted the manuscript, and all authors contributed substantially to its revision. All authors read and approved the final manuscript. KW takes responsibility for the paper as a whole.The pre-publication history for this paper can be accessed here:


We want to congratulate Chinnock and colleagues, who summarize very well the main problems that evidence-based medicine faces in developing countries . As membhttp://www.bibliotecacochrane.net). In addition to systematic reviews and protocols, this database contains evidence-based information not indexed in other sources.We have learned from the experience of working in and with Latin American countries that one of the first barriers to overcome is inequity in accessing evidence. The second barrier is the English language. Efforts have been made by our network to overcome both barriers by providing free access to Biblioteca Cochrane Plus (BCP) .