This article explores the theme of presence of the psychotherapist, a concept that has been of particular interest in humanistic and existential approaches. Presence was first associated with the humanistic attitudes of the practitioner and the way he or she embodies these attitudes in the here and now of the encounter. Since the publication in 2002 of Geller and Greenberg’s model of therapeutic presence, several quantitative studies have explored the relationship between the therapist’s perception of presence and other dimensions of the therapeutic process. However, qualitative explorations still seem necessary to account for the complexity of the therapist’s presence and its role in the therapeutic process. Centered on the therapist’s perspective, we use an idiographic methodology and refer to lived clinical experience to highlight the dimension of sensory contact that, through the body, actualize a connection to a virtual space of the therapeutic relationship. We so describe how a therapist can achieve an embodied processing to clinical material from what we describe as “traces of presence” of the other. From this point of view, the patient’s presence incorporates itself into the therapist’s experience and the therapist can perceive aspects of this presence in a tangible, concrete, and useful way. The therapist’s presence thus takes on a meaning that is not reduced to what the patient will perceive and interpret of his or her attitude. It becomes the main material from which the therapist orients his or her clinical interventions. Presence may be considered as a posture of action of the therapist in the temporal horizon of the immediate. This posture requires leaving a theoretical description of the practitioner's action and amounts to describing  each singular event  in itself. It requires reintroducing a role for the ephemeral and the unpredictable, approaching each seemingly innocuous event for what it proposes in the patient’s and the therapist’s experience.We also consider a presence that extends to a global configuration of communication, which could also be a communion between the patient and the therapist. The meaning and value of the therapist’s experiences should not be limited to what the patient will interpret in terms of empathy or therapeutic alliance. Some of these experiences are not ordinary in nature and intensity. They could be understood in terms of a continuum that links the patient and the therapist in an extended context that is not limited to the time and space of the sessions.This perspective implies attentional receptivity and brings back to the dimensions of contact, of incarnation, of the relationship to time, to space and to objects of perception. Thus, the concept of presence questions the role of these dimensions of the relationship regarding therapeutic change.The concept of presence is mainly promoted in the field of humanistic and existential approaches . These aSeveral studies  confirm Many leading practitioners have discussed and considered the role of presence in therapeutic outcomes. The concept is found in the work of However, as a concept, Presence remains difficult to handle. A first aspect of presence may be associated with the therapist’s humanistic attitudes, particularly the attitudes of congruence, empathy, and unconditional positive regard proposed by However, it is not easy to draw a clear line between the concept of presence and the necessary conditions for therapeutic change proposed by This leads us to reflect further on the notion of presence and its clinical perspectives. After having presented the difficulties associated with the study of the concept, we will use an idiographic approach. We will highlight the role of the therapist’s receptivity and the possibility of a clinical embodied perspective carried out from a material made of experiences whose nature appears pluralistic. We will show that it is possible to conceive of another form of presence alongside an in-session presence.More precisely, by taking up From early on, Rogers held that contact is a necessary and minimum condition for relationships and change: “Two persons are in psychological contact, or have the minimum essential of a relationship, when each makes a perceived or subceived difference in the experiential field of the other” .In this sense, presence involves being available to oneself and the other in a complex way that is centered on the bodily: we welcome the other into our being in a manner that is palpable, kinesthetic, sensual, physical, emotional, and mental . For ErsThese are the elements that are possible to connect with Rogers’s conditions of congruence, unconditional positive regard, and empathic understanding , 1961. Hperhaps I have stressed too much the three basic conditions . Perhaps it is something around the edges of those conditions that is really the most important element of therapy—when my self is very clearly, obviously present , p. 28.He also added, “I recognize that when I am intensely focused on a client, just my presence seems to be healing, and I think this is probably true of any good therapist” , p. 28.Concerning the healing power of “just my presence,” Rogers mentions that he makes use of his self and his responses, but that he does not express every aspect of himself. There is thus a subtractive approach to presence, in which irrelevant aspects of the therapist’s personality and experience are set to the side. Presence is what remains when everything else has been removed. This state can be described in terms of its relation to immediacy and embodiment, and one can also wonder about its relation to altered states of consciousness.Presence occurs in reference to a specific temporal horizon. naïf immediacy”—which precedes whatever we can say of it, into the field of psychology, and had given it a primary role.In his later writings on the therapeutic process, Stern  included insights from phenomenology and certain currents of embodied cognition. He provides a theory of “present moments,” subjective moments of very short duration , 2010. ICentering himself also in this temporal horizon, Roustang—as In the infinitesimal of a relation, the therapeutic relationship becomes a space where bodies live, move, and locate themselves in respect to one another. The question of presence mingles with those of the body, the living being, and vitality. Since the body only lives in immediacy, returning to the body also means occupying the immediate.Roustang’s views are supported by approaches concerned with embodied cognition. These posit that there are forms of cognition that do not need to be represented . The uniIn terms of relationships, For these authors, emotions arise from the circuit of interaction between the affective qualities or affordances of the environment and the bodily resonances of a subject. Bodily movements that have been amplified or inhibited become the source of lived emotional experience. The concept of affordance, which was suggested by Gibson in 1966 , dependsIn these terms, the practitioner’s presence in the patient’s environment can be understood as proposing affordances, actions, and affective investments. What seems superficially to be trivial has the potential to convey meaning for the patient and can do so from the outset of the relationship: what s/he perceives and feels about the spatial organization of the practitioner’s office; how s/he experiences the first contact over the telephone or the first meeting with the therapist, and the therapist’s behavioral invariants. Practitioners should pay attention to these things, especially since, if we follow Roustang, what appears initially becomes essential, in the first instants when bodies take their places and position themselves in relation to each other.The first model of therapeutic presence was provided by Geller and Greenberg, on the basis of qualitative research conducted with seven experienced psychotherapists who had written on this theme. They provide the following definition: “Therapeutic presence involves bringing one’s whole self into the encounter with the client, being completely in the moment on a multiplicity of levels, physically, emotionally, cognitively and spiritually”  with embodied and sensory experience. When the patient is less engaged in conceptual inference, lived experience becomes meaningful in reference to the immediate context, which is structured by the therapist’s presence. In this way, people can distance themselves from their complaints and open themselves to new possibilities of meaning and action that are found in the moment.The client’s experience of the therapist’s presence does not explain, by itself, how patients can live experiences outside their usual concept of self. So, beside the client’s perception of the therapist’s presence what is the implication of the therapist’s receptivity in achieving therapeutic change?From a large review of results, – “Sensitivity to the patient’s characteristics.– Flexibility in the choice of interventions.– competence to intervene without inducing a process of resistance.– finesse in knowing how to follow the patient’s coping styles.– ability to build a therapeutic alliance.– affective sensitivity conducive to a secure attachment.– receptivity to encourage not only empathic responses but also responses of genuine warm acceptance.– mastery and appropriate application of techniques adapted to the patient’s needs.”From this perspective, as an openness to one’s own experience, presence could be seen as what guides the therapist in the choice and direction of his/her attitudes and interventions. But if presence is about the therapist’s receptivity and engagement of their self in the here and now, how do they do this? Presence is often understood on a scale of intensity, in terms of presence/absence. Its usefulness is then thought in terms of congruence, in terms of patient’s perception of the empathy shown by the therapist, or in terms of therapeutic alliance. This amounts to considering a presence perceived by the patient. This is obviously essential. However, we need to go deeper into the description, analysis, and use of the therapist’s experiences and perhaps discover other meanings of those experiences.Ordinary presence” would be a prerequisite for Rogerian conditions. It refers to presence that can be experienced by the patient. “Extraordinary presence” refers to a “deep state of connexion to oneself and to a fine source of energy of which one is typically unaware”  there appears no universal element of which all things are made.” , p. 27.The post-session 2 “letter to your presence” shows that the experience of the therapist that remains as a trace after or between sessions can be processed and transformed in a way that is useful for the therapeutic process.The therapist conceptualized this process as follows.“When I am in contact with these patients during a session, sensory and perceptual experience takes the form of a tangible, palpable material that imposes itself on lived experience; this material can be located either inside or outside the body, and can be worked, “handled,” transformed, and used in the clinical situation. It appears as a compendium of information that is not initially comprehensible in an explicit manner, seeming to belong to the dimension of that which initially evades conceptual thought. However, this compendium can unfurl or unfold over time. An instantaneous lived experience can be unfolded over the course of several minutes or more, through an exchange, and a meaning can then emerge from it. These traces of the presence of the other can sometimes be there without my having any idea of what to do with them. I am not required to do anything with them unless they become insistent, but it is also important not to ignore them or try to pretend that they do not exist.In a session, they provide the sensation that a dialogue is taking place on several parallel lines or levels. One line of communication is located on the verbal level, while another transpires with another dimension of the person; this dimension appears to be dissociated from his/her physical body, yet it remains perceptible and can even be located spatially in his/her vicinity. This other dimension of the person can appear as a sort of insistence, as if a request were manifest on an implicit level, one that is separate from verbal expression. There is thus a contrast between what the physical person is asking for and what comes from something else. At times, it is as though, while the person is speaking, some invisible “other” is communicating in another way, as if “it” were calling out. Sometimes this other “person” is “on hold,” “keeping watch,” remaining “quiet,” “crying out,” or “coming into contact.”With patients who elicit this kind of experience, certain residual traces persist after the session.I conceptualized this idea by evoking the notion of residual traces of the other's presence. The spontaneous method that arose on this occasion was structured and rehearsed. It consists in carrying out a dialogue of the as-if type based on the following rules– writing as soon as possible, but perhaps not too soon in relation to the session. Within 24 hours or more, or at the moment when this presence is felt sufficiently– writing in the present tense, without reference to your own history or as little as possible. This favors the evocation and description of perceptions and sensations.– writing in short sentences. Combined with the present tense, they create a particular dynamic that helps in this process– writing in “one go”– writing without referring to the person's first name in order to make less reference to already existing concepts about the person– writing addressing the residual presence while referring to interpretative aspects of the experience in relation to this residual presence. Describing not only moments, perceptions, sensations, emotions, but also what comes to mind, from what has already been elaborated about the person, from the fiction we have constructed of him/her. There are things that may have been said and also all those things that were not thought or said during the process. It is as if what has been said initiates other things that emerge outside the session– writing with compassion in mind– As few contextual elements as possible. The residual presence becomes the only element of our context.– Like in dreams, there is no anteriority to the scene described. Just what comes in reference to that scene; we describe the material contained in that scene, then another scene is there, etc. The scenes follow one another without necessarily an anteriority or chronology.In doing this, I was, on one hand, giving a concrete form to the wealth of information that had been incorporated over the course of an interaction. Carrying out an “as if” sort of conversation with the incorporated residual presence allows the available compendium of information pertaining to the therapeutic relationship, which has not yet been put into words, to be actualized. On the other hand, I was able to observe that this way of proceeding has specific effects. For example, following such a “remote session,” the residual presence changes and becomes less distinct. It can also happen that it leaves me with a foreknowledge of the patient, in which certain elements that were located at the margins of what s/he said become central in the written conversation, and then emerge spontaneously in the therapy session, when the patient brings them up as the main theme. Similarly, other elements that were implicit then become explicit for me in my interaction with the residual presence, and later become explicit for the patient. Finally, this work with my lived experience of the other’s residual presence seems to enhance my long-term sensory and perceptual acuity.”How can we understand this type of experience, which suggests that there is a form of responsive, sensory contact between patient and therapist? Merleau-Ponty’s definition of virtual space is based on the work of Wallon in the 1920s. As As a result, the place where the therapeutic relationship occurs can be interpreted as one in which the therapist deploys a presence in terms not only of what happens in the physical space or in his/her thoughts, but also of what is going on in the virtual space. One hypothesis is that an aspect of presence involves the psychotherapist’s ability to grasp and actualize in an embodied manner the virtual space of the relationship. This space contains gestures and movements that are not performed in any actual space. When bodies are in presence, their movements may be “on the way,” interrupted, or held back; they may not find a means of expression in the physical space or they may be repressed. In all of these cases, movements exist as possible, in a non-physical or virtual space. They are not perceived consciously; instead, they are actualized by the psychotherapist as traces in the lived body, without actually being conceptualized. The therapist’s use of these movements—which exist in the possible and not yet in the physical—by the intermediary of his/her bodily experience, may encourage the patient to (re)turn to new possibilities. This use guides the patient’s experience beyond his/her habitual concepts and ways of thinking. In this way, the patient can add to the sum of all his/her experiences, as Presence thus becomes presence to the virtual space as much as to the physical space or to the already conceptualized aspects of immediate experience. It may also be a sign of the therapist’s relation to a virtual body, a relation that develops over the course of professional practice in his/her relations to patients.Presence thus appears to be intimately linked to the capacity to actualize the information resulting from the relation to the virtual space as well as to the actual space of the relation. It allows the emergence of a clinical material which is elaborated from the presence of the other as it appears in the experience of the therapist. This presupposes the existence of a primary disposition to welcome the other in our psychic reality.I and Thou, which has served as a point of reference for humanistic and existential approaches. For  with the patient’s presence, simultaneously at different dimensions of the relationship, in a way that is experienced as tangible, palpable, and concrete and in a context that is intended to be therapeutic. These different dimensions are in particular:– Temporal. They imply changes in the orientation of time with a focus on the horizon of the smallest moments, the immediate, the birth of movements.– Spatial. They imply an orientation of the therapist to the actual or virtual space of the relationship.– Corporeal. They imply the relation to the actual body and movements as well as to the virtual body and movements, and the relation to the traces of the patient’s presence in the therapist’s bodily experience.– Identity. They imply putting aside aspects of the self that should not interfere with the encounter.This presence is inscribed in a general context of the relationship that is not limited to objective time and space. It produces perceptual, sensory, emotional, and conceptual material that the therapist can use to set up the conditions for therapeutic change.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.Ethical review and approval were not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.PM: conceptualization, methodology, investigation, and writing original draft. AB and AS: conceptualization, supervision, and writing—review and editing. All authors contributed to the article and approved the submitted version.Université Paris 8 Vincennes – St Denis: contribution for translation from French to EnglishThe authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The evidence to policy and practice facilitated process represents a journey that countries and their development partners can embark on to increase the likelihood that health policy makers will use cost evidence for policy making and planning. Simply making immunization cost evidence available will not likely lead to uptake at country level, with several potential causes identified.Facilitating the 6-step evidence to policy and practice process led to increased recognition by national- and subnational-level stakeholders of the importance of generating and using cost evidence in all 3 countries. However, this did not necessarily translate to actual use.Six lessons learned can help future researchers improve the use of immunization cost evidence in country planning and budgeting processes.Researchers need to provide a clear use case for cost evidence. Engaging a small multidisciplinary stakeholder group to develop, implement, and later champion the research can help.Researchers should make cost evidence available within windows of opportunity required by policy makers and practitioners, even if it occasionally requires a sacrifice between rigor and speed. Researchers need help from local stakeholders to properly tailor, package, and deliver the key messages.Researchers need to be prepared to support potential users of cost evidence in translating cost data for use and should build this time into project and funding cycles. In many low- and middle-income countries, planning cycles and policy decisions are not always informed by cost evidence, even where relevant and recent cost evidence is available. The Immunization Costing Action Network (ICAN) project was a research and learning community designed to strengthen country capacity to generate immunization cost evidence and to understand and improve the evidence-to-policy linkages for the evidence. We identified key factors that increase the likelihood that health policy makers will use evidence for policy making or planning, which shaped the development of a 6-step evidence to policy and practice (EPP) facilitated process. ICAN used the EPP process in Indonesia, Tanzania, and Vietnam from 2016–2019. The experience resulted in several insights regarding country priorities related to cost evidence and factors that determine uptake. Cost evidence is more likely to be used if it answers a specific policy question prioritized by the immunization program, while the use case is less clear and urgent for routine planning and program management. Nonhealth ministries and subnational stakeholders can provide important perspectives to inform the research and its usability. The use case for evidence should be revisited periodically as divergences from formal planning cycles are common and new policy windows open. Ensuring evidence is available at the right time is critical, even if this requires a sacrifice between rigor and speed. Engaging a small group of stakeholders, rather than an individual, to champion the research may be more effective, and the research has greater legitimacy if it is produced by multidisciplinary country teams. Evidence and messages should be tailored for and packaged targeting different audiences. Going forward, continued support is necessary to bridge the divide between those who generate cost evidence and those who translate evidence for policy and planning decisions. Governments need to understand what it costs to deliver vaccines to reach coverage goals, address health equity, manage the introduction of new vaccines, and ensure efficient use of resources. Even in low- and middle-income countries where relevant and recent cost evidence is available, planning cycles and policy decisions are not always informed by evidence. Globally, there is limited understanding of how economic or other types of evidence are used in health policy processes and decision making and how influential they are.–,Costing and cost-effectiveness studies have not had as significant an impact as they could have had in vaccine policy and decision-making processes where domestic and institutional politics often dominate.Simply making evidence available will not lead to uptake, with several potential causes identified. First, operational costs may play a larger role than economic evidence in decision making.Simply making evidence available will not lead to uptake, with several potential causes identified.To address the identified gaps, the Immunization Costing Action Network (ICAN)ICAN was a research and learning community to increase the visibility, availability, understanding, and use of immunization delivery cost information. It was designed such that countries led the agenda, with technical facilitation and coaching from ThinkWell and John Snow, Inc. Country-based research institutions, led by local principal investigators, conducted immunization costing studies addressing challenges at the top of domestic immunization agendas, driving the research from conceptualization to dissemination, and supporting policy translation. The research institutions included a collaboration between Universitas Indonesia and the National Institute of Health Research and Development in Indonesia, the Ifakara Health Institute in Tanzania, and the Hanoi University of Public Health in Vietnam. The research institutions led country teams comprised of health economist researchers, immunization managers, and budget officers and planners from ministries of health (MOHs). The 3 country teams were convened several times during the 3-year project in workshops aimed at improving the capacity to design research, sharpen methods, and learn from each other regarding how to interpret and leverage cost evidence for policy making and planning.We conducted semistructured interviews with key informants and a literature review on other initiatives and research to summarize the existing evidence around factors that influence the uptake of costing evidence. This resulted in 8 key factors that increase the likelihood that health policy makers will use evidence for policy making or planning .BOXDemand: Policy maker and practitioner demand for the evidence regarding their broader policy and program imperatives increases the likelihood that it is used.Right time/right place: Evidence needs to be made available for key policy moments or within windows of opportunity required by policy makers and practitioners to act on it. Researchers need to deploy strategic opportunism within those windows or create windows to increase the likelihood of use.,Evidence strength and quality: Evidence is considered more credible if it is developed following an endorsed and rigorous methodology. Analysis needs to be done by a well-respected source.Relationships and networks: Evidence is more likely to be used if a focal point from within the Ministry of Health is engaged in leading or coordinating the research and if research teams are multidisciplinary with the various stakeholders involved from the beginning.Decision space: An understanding of the decision space and types of decisions that can be affected by evidence at various levels of the health system can help ensure the relevance of the evidence for different stakeholder groups and increase its usability.Packaging and targeting: Simple, attractive, easy-to-understand formats  and clear and definitive takeaway points and requested actions are important for busy policy makers and practitioners.,Policy translation: The format of the evidence might require adaptation to accommodate what is needed for policy or program use. Policy makers and practitioners may benefit from additional support to extrapolate the evidence to different contexts, questions, or geographic settings. Because researchers may not have the interest, time, or skillset for policy translation or policy advocacy, engaging policy actors can help.Implementation climate: Stakeholder readiness for use of the evidence can depend on their receptiveness to using economic data. Country fiscal space issues, policy priorities, and political and administrative structures, as well as the nature of institutions, can also support or impede the use of evidence.Supplement 1 presents the EPP facilitated process along with key questions and objectives driving each process step.Based on the identification of these critical success factors, we used a 6-step evidence to policy and practice (EPP) facilitated process in each country to increase the likelihood of generating policy and program-relevant cost evidence and improving its uptake and use . SupplemWe aimed to identify real country demand for cost evidence. We consulted the primary customers for the research  and other immunization stakeholders from government ministries, donors, and development partners to understand their high-level objectives and priorities, how cost evidence might help them achieve those objectives, and any barriers to the use of cost evidence. Through this process, country researchers became embedded in the network of immunization stakeholders and gained a strong understanding of the challenges that policy makers and program managers face. We approached stakeholders with a short menu of potential research topics, and they selected the costing of alternative vaccine delivery strategies as the most relevant. Within this topic, the priorities varied by country .We aimed to identify real country demand for cost evidence.At various points during the project, we consulted with a broad set of stakeholders to identify promising opportunities for use of cost evidence. We documented the timing and approval processes for policy development and planning cycles, upcoming policy decisions , and program management activities .During cross-country workshops, we conducted a mapping exercise of the annual planning and budgeting process in each country. We specified the events where cost evidence could be introduced, the tactics and tools to deploy to increase the likelihood of the evidence being taken up, and any anticipated bottlenecks to the use of evidence.We identified key stakeholders involved in policy formulation, planning, and budgeting to determine to whom (people and institutions) the cost evidence should be targeted for greatest impact and the best ways to reach them with cost evidence. Originally, we envisioned a decision space mapping exercise involving a broad range of stakeholders to capture connections beyond costing, immunization, and even the health sector. However, we ultimately deprioritized this as we focused on generating evidence for routine policy and program activities as opposed to policy decisions that require significant advocacy and decision-making authority beyond the immunization program or health sector.We focused on generating evidence for routine policy and program activities as opposed to policy decisions that require significant advocacy and decision-making authority.When preliminary findings became available, we came together for a cross-country analysis and evidence interpretation workshop. Country teams discussed the findings, began to formulate conclusions, posed questions to provide direction for additional analysis, identified key messages, and further delineated use cases for the cost evidence. We conducted a similar exercise with a broader group of stakeholders during dissemination activities in each country.We shared best practices in presenting data to different audiences, and country teams gained confidence in developing slides and charts, writing policy briefs, and crafting an elevator pitch. We developed a dissemination plan for the results, including high-level immunization meetings at which a broad set of stakeholders would be present, followed by other smaller targeted meetings. The principal investigators also gained additional experience in presenting the findings through remote coaching sessions and through delivery of their presentation at the 2019 International Health Economics Association Congress.Supplement 2).Alongside dissemination of the evidence, the country teams developed EPP action plans to support the uptake of the cost evidence after the exit of the researchers. Each country’s EPP plan summarized information related to each process step, clarified needed action for policy translation, and specified the technical support and resources required to ensure those actions were carried forward. For example, in Vietnam, the action plan specified a subnational dissemination plan along with information on a vaccine introduction pilot based on the results. In the case of Tanzania, the action plan included additional analyses that the national immunization program planned to commission to inform the next comprehensive multiyear plan and upcoming policy guidelines . Over the course of the project, we revisited the use case for the cost evidence several times and identified new opportunities even as we were finalizing the analysis. New ideas also emerged regarding the usability of the evidence as additional stakeholders were reached as part of the dissemination effort.We learned that more rapid research might occasionally be acceptable, even if it means making some methodological sacrifices.,–,Earlier studies have emphasized the need for research to clearly highlight key messages, as well as the need to summarize policy recommendations.We also found that to tailor messages effectively, a variety of stakeholder perspectives is useful for interpreting the results and identifying policy and management implications. During our third cross-country meeting, national immunization managers and budget officers had an opportunity to review the data and analyses before the final results became available. They asked for new and different ways of presenting the data, which the researchers took onboard. For example, the MOH budget officer in Tanzania struggled to see how shared costs allocated to the immunization program could be combined with cost data in different formats from other programs. To ensure that the evidence was most useful to the immunization program and broader MOH, different cost components were classified as immunization-specific or shared depending on the budgeting process.,As found in other studies, the timeliness of making research outcomes available is critical.There are myriad opportunities for use of evidence that crop up continually, and a longer time horizon allows for bigger thinking and more creativity regarding the use of the findings.Moreover, cost evidence for routine planning and program management should not only be produced and used once, but rather, should be used iteratively with adjustments reevaluated and streamlined over several budget cycles. Ideally, one-off costing studies should be followed and complemented by the continuous collection of costing data, such as through National Health Accounts exercises.Supplement 3 provides common examples of transformations that may be required to facilitate the use of cost evidence.Cost evidence often may not be used due to a lack of understanding on how to transform it to answer policy or program questions. Policy translation and planning often require additional analyses and transformations to cost findings to be able to answer specific questions and be usable for budget requests. Policy translation and planning often draw on a different skillset than that of the researchers that generate the evidence, and immunization program managers and MOH budget officers have competing priorities and do not always have the right skills for this either. For example, at the national level, immunization program teams would benefit from increased capacity to use cost evidence for scenario planning, modeling, and projections. At the subnational level, managers would benefit from support in interpreting cost evidence and transforming it to their local context . ,Previous research has found that the bridge between researchers and policy makers is a key barrier to use of evidence.Nevertheless, even more capacity strengthening to support policy translation is needed for researchers, including practicing presentation skills to deliver results and key messages. Building in extra time to support capacity strengthening in health economics was also necessary for immunization program managers and MOH budget officers. In response to their demand, the project team offered a health economics introductory workshop, but more capacity enhancement in health economics was needed. When the results were becoming available, government officials expressed a desire to have been included in the data collection effort and to gain expertise in policy and budget advocacy. To support policy translation in Tanzania, the immunization program developed a wish list for additional support which included technical assistance in developing briefers and advocacy messages for use with nonfinance stakeholders and support from health financing technical advisors to ensure use of the evidence.,,,Although the impact of decision space on improving the delivery of primary health care services such as immunization is still being explored, the positive relationship between decision space and institutional capacity and accountability is largely recognized.,,Previous research has shown that personal contact and relationships are important in improving the uptake of evidence.A small multistakeholder group comprised of government officials , donors, development partners, and a civil society organization representative may have been more successful in guiding and championing the research and in facilitating dissemination and use. In Indonesia, the project concluded during a leadership change at the MOH when future priorities were uncertain. There was no clear champion within government to take the work forward, which a multistakeholder research steering committee may have protected against.Simply making data and evidence available does not result in its effective use. The EPP facilitated process represents a journey that countries and their development partners can embark on to increase the likelihood that health policy makers will use cost evidence. Although each country’s journey is unique, in all 3 project countries, the EPP process led to increased recognition of the importance of using cost evidence—an outcome that will most likely be sustained. At the beginning of the EPP process, country stakeholders intuitively knew that having cost evidence was better than not, but some had trouble articulating how it could be used. Because country stakeholders were involved throughout the EPP process and because messages were tailored to various audiences, the value of the generated cost evidence became clearer.Although each country’s journey is unique, in all 3 project countries, the EPP process led to increased recognition of the importance of using cost evidence—an outcome that will most likely be sustained.Nevertheless, costing to inform routine planning and budgeting is often still considered by in-country stakeholders as a “nice to have” not a “need to have.” To ensure that costing and economic evidence will continue to be used in routine management and planning processes, additional capacity strengthening is needed to increase not just the capacity to generate evidence but also increase decision makers’ capacity to interpret and use evidence. Research projects need to build in more support to bridge the divide between those who typically generate cost evidence and those that translate evidence for policy decisions and planning. Going further, we recommend building policy dialogue and policy advocacy work more explicitly into the EPP process.Just as has been the limitation of other studies focused on improving the evidence base around EPP,


ADHD is the most common neurodevelopmental disorder presenting to child and adolescent mental health, paediatric, and primary care services. Timely and effective interventions to address core ADHD symptoms and co-occurring problems are a high priority for healthcare and society more widely. While much research has reported on the benefits and adverse effects of different interventions for ADHD, these individual research reports and the reviews, meta-analyses and guidelines summarizing their findings are sometimes inconsistent and difficult to interpret. We have summarized the current evidence and identified several methodological issues and gaps in the current evidence that we believe are important for clinicians to consider when evaluating the evidence and making treatment decisions. These include understanding potential impact of bias such as inadequate blinding and selection bias on study outcomes; the relative lack of high-quality data comparing different treatments and assessing long-term effectiveness, adverse effects and safety for both pharmacological and non-pharmacological treatments; and the problems associated with observational studies, including those based on large national registries and comparing treatments with each other. We highlight key similarities across current international clinical guidelines and discuss the reasons for divergence where these occur. We discuss the integration of these different perspective into a framework for person/family-centered evidence-based practice approach to care that aims to achieve optimal outcomes that prioritize individual strengths and impairments, as well as the personal treatment targets of children and their families. Finally, we consider how access to care for this common and impairing disorder can be improved in different healthcare systems. Attention deficit/hyperactivity disorder (ADHD) is a neurodevelopmental disorder defined by persistent, impairing and developmentally inappropriate inattentive/disorganized and/or hyperactive/impulsive behaviors that lie at the far end of a normally distributed continuum.  ADHD is ADHD results in considerable burden at an individual, family, and societal level and has an important impact on quality of life and daily functioning , 6. PatiWhile there is now considerable evidence to support the efficacy and safety, at least in the short term, of pharmacological  and someThis work did not involve primary research, and therefore, no ethical approval was required. The focus is on children and adolescents, because a range of both pharmacological and non-pharmacological interventions has been evaluated in this age range, while the evidence in adults is limited largely to pharmacological interventions. While most clinical trial data have focused on core ADHD symptoms as the primary outcome of interest, an increasing number of studies have recognized the importance of a broader range of outcomes. These include common co-occurring symptoms such as mood lability and those related to coexisting disorders , functioWe will first consider some of the methodological challenges that we encountered when conducting systematic reviews and meta-analyses of ADHD interventions and highlight how these may have affected interpretation of the evidence and discuss some of the approaches that may mitigate these methodological challenges. We then compare different sets of guidelines in relation to ADHD management, with a particular focus on the common features and highlighting important differences, including possible reasons for these discrepancies. We then discuss how the evidence and guidelines can be translated into high-quality care, and how the clinician can take account of individual symptom profiles, treatment targets, and personal circumstances to provide person- and family-centered approaches to intervention within the current evidence base. Finally, we consider the implications for systems of care and how adherence to evidence-based practice can be implemented across a range of care settings.In this section, summarized in Table low for non-pharmacological approaches and low-to-moderate for pharmacological interventions [Randomized-controlled trials (RCTs) remain the gold standard to assess the efficacy, effectiveness, and safety of interventions. In the last couple of decades, there has been a greater appreciation of the importance of rigorous trial design and analyses to ensure that treatment evaluations are not biased. Standardized instruments, such as the Cochrane Risk of Bias tool  and GRADventions . Of noteventions  and asthventions ).A major potential source of bias and therefore for the quality of RCT evidence relates to the presence and adequacy of blinding—whether patients or assessors of outcomes are aware of who received what intervention. There is potential for unblinding even during well-designed and carefully conducted pharmacological trials, which it has been suggested could undermine confidence in all of the evidence for efficacy . While Patient selection entry criteria also represent a potential source of bias and limit generalisability of findings. Most clinicians are aware of the issues relating to non-inclusion in most RCTs of those more complex comorbid cases that are common within their clinical practice. There are, however, several other issues that are not so obvious. For example, many recent medication trials have included enriched samples, for example, excluding participants who have previously not responded to or had adverse effects from one of the study drugs. Although these criteria were implemented for ethical reasons, they are likely to introduce bias, usually in favor of the study drug. Non-pharmacological trials also typically recruit less complex and hence easier to treat patients for reasons of convenience and this can affect trial findings . In non-http://bit.ly/1O2XRPp). Several of these randomized discontinuation trials have now been published although few have exceeded the 1 year duration required by the regulatory authorities [While there is a wealth of data regarding the short-term efficacy of treatments for ADHD, high-quality data on longer term outcomes are still lacking and the long-term effectiveness of ADHD interventions, both pharmacological and non-pharmacological, continues to be debated , 46, 47.horities –51.A number of studies, including the Multimodal Treatment of ADHD (MTA) study  and obsehttp://bit.ly/1O2XRPp) and have directed that these studies focus on growth and puberty, cardiovascular safety, as well as psychiatric and neurological adverse effects. However, the lack of a comparison group in these studies  post-licensing  makes is  is a relatively recent development in the field of evidence synthesis that allows comparison among different interventions, to provide a ranking of interventions based on, e.g., efficacy and tolerability. Supported by the WHO, Cochrane, GRADE, and NICE , NMAs haMultiple systematic reviews and meta-analyses have been published assessing the efficacy and/or tolerability of both pharmacological and non-pharmacological treatments for ADHD and these data have been further scrutinized in the development of evidence-based guidelines.The most recent and comprehensive appraisal of pharmacological treatments is the network meta-analysis (NMA) conducted by the EAGG, with stringent inclusion/exclusion criteria (such as excluding enrichment designs and RCTs with add-on treatments) as well as including both published and unpublished data. The main findings on efficacy and tolerability are presented in Table In relation to adverse events and safety, both individual reviews, meta-analysis, and network meta-analysis demonstrate good short-term tolerability of ADHD medications once the events are equally common in treatment and placebo arms are taken into account. Common short-term adverse effects are well described and routine monitoring is part of standard care as discussed in many clinical guidelines , 79. TheSeveral high-quality systematic reviews and meta-analyses have also investigated the evidence for the efficacy and effectiveness, but not acceptability or adverse events, of non-pharmacological approaches to treating ADHD Table . For behClinical guidelines are an important mechanism for increasing evidence-based clinical practice, and improving quality of care and resource allocation. Several national guidelines from Europe and North America, compared in Table The recommendations of the cited guidelines (apart from the Canadian CADDRA guidelines which did not include a systematic review of the evidence) were developed by combining evidence summaries , with eDespite being largely reliant on the same evidence base, differences in recommendations are especially likely when high-level evidence  is lacking. In such circumstance, different groups have taken different approaches—the UK NICE  and the All guidance specifies who can diagnose and prescribe; European  guidance makes specific reference to specialist ADHD training among paediatricians and psychiatrists, while North American guidance includes primary care physicians. For prescribing, UK NICE guidance also allows shared care plans between specialists and primary care physicians.All guidelines agree that the clinical interview and direct observations, including assessments of impairment, physical and psychiatric comorbidity, and family history, provide the basis to establish a diagnosis.Rating scales are recommended as auxiliary diagnostic tools and as systematic outcome measures in treatment monitoring, rather than as the gold standard for the diagnosis.Neuropsychological assessment is not considered to be essential to establish the diagnosis, even though it can provide useful information to better tailor the management.In relation to treatment:oYoung children: under 5 years (NICE) or under 6 years (German and Spanish Guidelines); school-aged children: 5/6-18 years; and adults: >18 years. The Dutch guidelines further differentiated between children 6 and 12 and adolescents 12 and 18 years.All guidelines take account of developmental differences, but some specifically stratify treatment recommendations according to age, with three groups identified:routine treatment in this age group, but medication is an option that could be considered in selected cases.The RCT evidence for efficacy and safety of medication in young children is considered too limited to recommend medication for When pharmacological treatment is indicated, most guidelines suggest that stimulants are the preferred first-line medication. The Spanish Guidelines only indicate that any of the approved medications can be used without offering any recommended order of priority.All agree on the need for psychosocial interventions, but differ in their timing in the management of ADHD. In contrast to the other guidelines, the NICE 2018 Guideline no longer recommends parent training as first-line treatment in school-aged children because of the lower effect sizes and poorer quality of evidence compared to medication, but ADHD-focused parent training remains the first-line intervention for pre-school children.In summary, in relation to assessment and diagnosisGuidelines differ in whether they consider ADHD severity as a factor when making recommendations. NICE (2018) currently do not, while the German and Dutch Guidelines, adopting the DSM-5 definitions of severity, do. Both the Dutch and NICE guidelines also take co-occurring ODD/CD into account. As the main aim of the severity specifier is to avoid overuse of medication in less severe cases, these different approaches may reflect variation in administrative diagnostic rates, which, for example, are higher in Germany (3.2–6.1%) –95 than Apart from the consensus-based CADDRA guidelines , these gWith respect to pre-medication investigations Table , all guiwithin each approach, selecting the interventions most suitable for individual patients. While all of the licensed ADHD drugs primarily target a reduction in ADHD symptoms, impact on other symptoms such as emotional dysregulation and mind wandering as well as indirect effects on associated mood [Careful and close application of evidence-based protocols, such as those employed in the MTA trial, often lead to better outcomes for patients in terms of ADHD symptoms  and therted mood , conductted mood , and othted mood  have beeIn terms of duration of action, while the non-stimulants atomoxetine and guanfacine have lower effect sizes on ADHD core symptom reduction , they haPatient and parent priorities; It is important to ensure that patient and parental priorities, preferences and choices are taken into account through the provision of information, followed by a process of shared decision-making [n-making . Familien-making , which mn-making , 109. Itn-making . Young pn-making , 111.Much of the broader management aims for ADHD can be achieved through high-quality psychoeducation offered at the beginning of the treatment process and topped up at intervals thereafter . EvidencAfter psychoeducation, a shared management plan should be created, prioritizing problems and translating them into aims and action plans. Parents and children may have different priorities for treatment targets and these differences should be identified and discussed to build consensus and focus .While we strongly support treatment optimization and the use of measurement-based care , there aCo-occurring disorders commonly present a complicating factor when managing ADHD . In prioWhen developing clinical services for children and adolescents with ADHD, a multi-disciplinary team is preferable to facilitate comprehensive evaluations including comorbidities, and to provide a wide range of therapeutic options , 132. PaClinicians and commissioners of services need to consider and seek to reduce many barriers that limit access to ADHD care , 138. ThThe dominant model of care is provision in a clinical setting, one-on-one, with highly trained professionals. However, this model can be expensive and limit access . Non-cliADHD is the most common neurodevelopmental disorder in children and adolescents. Although ADHD presents with varying levels of severity, a substantial proportion of affected individuals experience persistent symptoms and substantial impairment from their core ADHD and/or co-occurring symptoms. Therefore, the accurate identification of ADHD and implementation of effective interventions is essential. There is a substantial evidence base, at least in the short term, for a range of pharmacological and, for non-core symptoms, some non-pharmacological treatments, with considerable consensus across different national guidelines emanating from societies that are diverse in terms of their rates of acceptance and identification and their organization of healthcare. However, as in many areas of healthcare, there are significant methodological problems with much of the research underpinning these guidelines and the clinicians should be aware of these and how they may limit interpretation of the evidence. Despite these methodological concerns, clinicians should be confident that ADHD medications are effective, well-tolerated, and safe. Parent training provides an important complement, but largely in relation to reducing additional behavioral problems and improving parent–child relationships. High-quality evidence for neuromodulation—including neurofeedback and cognitive training—is more limited, and as a consequence, these should not currently be recommended as the first-line interventions for core symptoms.Management of ADHD extends beyond the implementation of guidelines and should be seen as a partnership with patients and their parents, underpinned by psychoeducation and a shared, agreed-upon management plan that considers individual treatment priorities and preferences. While standard care is delivered in clinical settings through one-to-one patient/clinician relationships, schools and third sector settings have much to contribute to a comprehensive approach. Future research should evaluate the utility of digital methods for monitoring outcomes and delivering psychological interventions , but it In summary, ADHD is an important disorder to treat and manage over time, in relation to core symptoms, co-occurring problems, and improvement of real-world outcomes. Clinicians should be robust in expecting substantial improvement with evidence-based treatment but should undertake this in partnership and with a focus on outcomes that are meaningful for patients.


Health and welfare technologies (HWT) are becoming increasingly employed in the Nordic countries, and in Sweden in particular. The amount of HWT public procurement is likely increasing at a similar rate, but requirements for evidence for effectiveness placed on bidders during this process may be lacking.This study investigated the use of evidence as a requirement in public sector tendering process of HWT, and how it affected bidder attributes and procurement outcomes. A novel type of systematic review and content analysis of requests for tenders for HWT announced prior to June 2021 was therefore conducted in Swedish public procurement databases.Ninety requests for tenders for 11 types of HWT met the inclusion criteria for review, accounting for potential contracts worth 246 to 296 million EUR. Criteria requiring evidence for effectiveness were used in 16 requests for tenders, accounting for 183 million EUR in potential contracts. Eight of the requests referred to an established independent standard to confirm such evidence, such as CE standard of conformity, MDR and/or MDD. This prevalence appears to cut across all types of procuring organisations and all types of HWT. The use of any evidence criteria, or lack thereof, does not appear to affect the outcomes of the tendering process.Criteria requiring evidence for effectiveness are used in less than a fifth of all public procurements of health- and welfare technologies in Sweden, and less than 10% refer to some form of independent standard as confirmation of such evidence. The procurement process therefore risks creating a legacy of sub-optimal technologies in health- and social care services. More prevalent and specific requirements for evidence and its continual generation in the procurement process are highly recommended. Recommendations for decision makers, procurement managers, and developers are provided. Health and welfare technologies (HWTs) are technology-based interventions that aim at maintaining or promoting health, wellbeing, quality of life and/or increasing efficiency in the operational delivery of welfare, social and health care services, while improving working conditions of the staff . These iIn Sweden, the current national vision for healthcare and social services  is that Logically, the public procurement of HWT has likely increased at a similar rate. Swedish procurement legislation is, as with other European Union (EU) member countries, based on EU directives (2014/24/EU being the most relevant for HWT) and EU primary law. All public bodies, including municipalities, regions, and national agencies, must follow this legislation to ensure that EU internal market competition as well as the free movement of goods and services are uninhibited, and that environmental and social considerations are taken into account. The principles of transparency, equal treatment, non-discrimination, proportionality, and mutual recognition must be followed during the procurement process. The Swedish National Agency for Public Procurement and the Swedish Competition Authority are agencies that nationally support and supervise procurement, respectively. However, neither these agencies nor any other independent organisation systematically compile the estimated amount of HWT purchased in Sweden or its outcomes. This may be due to the absence of a common definition, nomenclature, and Common Procurement Vocabulary (CPV) codes for such technologies within procurement administration.SALAR has prioritized the implementation of four HWTs in particular: Global Positioning System (GPS)-based alarms, digital nocturnal surveillance, digital locks, and automated digital drug dispensers . While HWhile the lack of evidence for desirable HWT effects may conceivably result from suboptimal implementation and/or use, there are upstream factors that may also play a significant role. HWT are most often purchased through an open public procurement process that is regulated by law  and thatComprehension of the concept of evidence and its related terminology is broad in academic, social care, health, and medical professions, but in public procurement administration it may be less understood. While health- and medical specialists’ knowledge of evidence may be involved in forming the desired specifications for HWT to be procured, the translation of such knowledge into administrative requirements in legally binding documentation is uncertain.What types of evidence for effectiveness and value do Swedish public sector authorities require from bidders when procuring health and welfare technologies?Are their differences in bidder attributes and procurement outcomes for those that require evidence for effectiveness and value compared to those that do not?This study investigated the use of evidence as a requirement in public sector tendering process of health and welfare technologies, with the following questions:To conduct the study, a novel type of systematic review and content analysis of requests for tenders for HWT in public procurement databases was used. Systematic reviews are exhaustive and reproducible searches of existing studies or background material in relation to a research question to summarize their findings quantitatively or qualitatively. Common in the health and medical fields, such reviews can also be used in other areas such as policy or methodology. This study nonetheless appears to be the first to apply such review methodology in the analysis of publicly procured health-related technologies using a procurement database.The included population in the study were all public sector authorities within the 290 municipalities and 21 regions in Sweden or their owned subsidiaries, as well as their representative member association (SALAR) or its owned subsidiaries. Requests for tenders that were included had to involve HWT that was in accordance with the definition stated in the Pharmaceutical- or medical technologies used for in-patient or hospital-care settings  and technologies that did not have the health or welfare of individual or organisational users or providers as a primary function  were excluded. Administrative systems such as electronic health records and databases/registries, interoperability applications and infrastructure, identification systems and similar were also excluded. Pre-announcements, dialogues, and requests for information were excluded, as were requests for tenders that were interrupted or cancelled by the procuring organisation prior to completion. Requests for tenders where documentation was not available were also excluded, as were duplicates found in the same or separate databases.www.opic.com [www.e-Avrop.com, www.KommersAnnons.se, and www.offentligaupphandlingar.se to check if additional HWT tenders could be found. All databases were operated by private sector companies.The Mercell Opic procurement announcement database (opic.com ) was theEnglish: welfare tech*; digital health tech*; digital health*; e-health*, safety alarm*; GPS-alarm*; security camera*; night camera*; nocturnal camera*; digital nocturnal surveillance; self-monitor*; remote monitor*; door alarm*; floor alarm*; absence alarm*; digital care; remote meeting*; digital lock*; robot*; incontinence sensor*; medication reminder*; medication alarm*; medication dispenser*; digital game*; digital rehab*; digital training*; smart*; health app*; digital activation*; epilepsy alarm*; seizure alarm*; digital learning*; digital education*; digital competency*;Swedish: välfärdsteknik*; digital hälsoteknik*; digital hälsa; e-hälsa; trygghetslarm; GPS-larm; tillsynskamera; nattkamera; digital nattillsyn; egenmonitorering; fjärrmonitorering; larmmattor; dörrlarm; rörelselarm; avvikelselarm*; digital vård*; distansmöte*; digital* lås; robot*; inkontinens sensor*; medicinpåminnare*; läkemedelspåminnare*; läkemedel dispens*medicinlarm*; läkemedelslarm*; läkemedelsautomat*; läkemedelsdispenser*; digitalt spel*; digital rehab*; digital träning*; smarta*; hälsoapp*; digital aktivering*; epilepsilarm*; digital *lärning; digital *läromedel; digital kompetensutvecklingSearches were conducted in Swedish and English. The following search words and combinations were used to identify HWT-related requests for tenders:The search words were in some cases adjusted  such that several overlapping search terms were included.Two researchers (MXR and SLS), hereafter referred to as reviewers, conducted the review process which had five steps:The request for tender announcement titles, summaries and in some cases attached documents of the obtained records were screened for relevance by one reviewer (MXR) with random checks by the other reviewer (SLS). If the reviewer(s) voted that the record fulfilled the inclusion criteria, then the record was saved in the procurement database and sorted by the search term used to identify it. Any conflicts were resolved through dialogue until consensus was achieved.For requests proceeding to this step, all tendering documents related the request for tender announcement were downloaded and screened for confirmation of inclusion by one reviewer (MXR) with random checks by the other reviewer (SLS). If the reviewer(s) voted that the request for tender still fulfilled the inclusion criteria, then the documentation was compiled and proceeded to data extraction. Any conflicts were resolved through dialogue until consensus was achieved..A summary of the results of this selection process can be found in Fig. 1www.voyant-tools.org [The textual content of all documents for included requests for tenders, including eventual awarding decisions and documentation, were then compiled in a text analysis tool  and seEnglish: evidence; science; scientific; sensitivity; specificity; study; studies; intervention*; benefit*; effectiveness; effect*; certification*; MDR; CE conform*; MDD; 2017/745.Swedish: Evidens*; vetenskap*; känslighet*; specificitet*; studie*; utfall*; intervention*; *nytta; nytto*; effekt*; certifier*; bevis*, MDR; CE-märk*; MDD; 2017/745.A context for all found terms, consisting of the 15 words to the left and right of identified search term, were copied and exported to a spreadsheet.utvärderingskriterier or tilldelningskriterier, often in the form of “shall” or “should” requirements) were used in the tendering documentation regarding evidence for effects or value, consisting of collected or objective data, a certification that is only achieved upon provision of such data, or similar andClear evaluation- or awarding criteria  with random checks by the other reviewer (SLS). The context was included as an evidence-related criterion if it met the following inclusion criteria:orthe requirement was solely related to the bidder’s overarching organisation ; orthe requirement is solely related to administration of the intervention ; orthe requirement did not address a defined outcome (e.g. “improve effectiveness” without stating what that effectiveness involves or means for the organisation); there was no request for documentation, certification or data to demonstrate fulfilment of the requirement (e.g. a simple “Yes/No” response sufficed as fulfilling the requirement).The context was excluded as a requirement if any of the following exclusion criteria were met:The criteria confirmation was conducted by one reviewer (MXR), with random checks by the other reviewer (SLS). Any conflicts were resolved through dialogue until consensus was achieved.Essential information regarding the type of HWT, procuring organisation, duration of the tender, number of bidders and related information was extracted for all included requests for tenders. The use of evidence criteria was also denoted with a description of what manner it was used . If the tender was successfully awarded, then the economic value of the tender and the attributes of the winning bidder were also extracted; if the tender was appealed then this was also noted, along with information regarding the outcome of the appeal, if available.the prevalence and type of terminology in the tendering documentation related to evidence for effects and/or valuethe prevalence and type of evidence-related criteria when evaluating bidsthe attributes of the procuring organisation, bidders, type of HWT and procurement result in relation to the above outcomesThe main outcomes of interest for the analysis were:There were 343,343 requests for tenders available in the main procurement database that was searched. No tenders were found in additional procurement databases that could not be found in the main database.After applying the HWT-related search terms, 479 requests for tenders were identified that could include HWT Fig. . Of thesNinety requests for tenders announced between January 2008 and May 2021 progressed to the text analysis stage, containing 11 different types of HWT , most could have been interpreted as referring to the earlier 1993 Medical Device Directive  despitEven if requirement of compliance with the newly implemented MDR becomes more prevalent in public procurement, evidence requirements may still need to be taken more seriously by procuring stakeholders. The “outcomes that matter” for any HWT application need to be formed by the procuring organisation in terms that make them relevant to local conditions and preferences. The structures and processes to accumulate evidence for HWT effects – both prior to and after implementation - in such local conditions need to be spelled out in clear terms to be able to maintain the integrity of a contract with any provider. Such requirements should be based on well-defined intentions at the start of a tendering process such that the integrity of evidence-based criteria, when being translated into legally binding procurement documentation, occurs successfully and in line with the intention.The procurement of pharmaceuticals and medical technology is, in comparison to current HWT procurement, vastly more evidence based. Scientific and statistical validity, analytical and clinical performance, and peer-review acceptance pertaining to pre-defined intended benefits are required by most continental and national authorities to achieve certification, with results from standardized clinical trials often necessary to achieve this. HWT currently lacks a similar national framework for assessment of effectiveness. Development of such a framework, with a national authority to oversee its use, may thus promote a broader use of evidence-based criteria in public procurement and minimise unintended costs and risks.National and/or regional policy makers should task appropriate authorities to develop and oversee a framework for evidence based HWT use, to support public sector procurers and developers achieve a common understanding of evidence use. The MDR should be a foundation in this framework, but also provide guidance adjustable to local conditions and user preferences.Decision makers that initiate procurement processes should clearly define intentions and outcomes that matter – such as health, welfare, and efficiency in operational care delivery outcomes - for any HWT application, and in terms of the local conditions and known or expected user preferences. These intentions and outcomes will form the core of what pre-existing evidence is necessary, and what evidence needs to be generated during the lifecycle of the HWT.Procurement officials must establish an evidence-based procurement process based on the intentions and outcomes prioritized by decision makers. Input should be obtained from researchers, developers and users when establishing this process. The resulting process should include a list of adaptable criteria for what evidence must be provided by bidders and how, as well as criteria for real world evidence generation after purchase and implementation. Evaluation of bids should also place significant weight on assessing such evidence, and contracts with winning bidders should include clauses regarding continual evidence generation as a condition for contract fidelity and/or extension. The use of MDR as a standard criterion should be vastly increased.For HWT developers, evidence for application effectiveness as well as real world evidence generation after implementation are now engrained in the EU MDR. High-quality continual evidence generation should be planned for when developing applications and be adaptable to expected purchasers’ local conditions and needs and user preferences. For small- and medium-sized enterprises, partnership with both academia and presumptive purchasers and their users to assist in meaningful data collection and evidence summarization may be necessary in many cases.Academia can play a significant role in creating a neutral place where stakeholders on the demand and supply sides can meet for mutual learning, sharing and co-creation of both innovations and evidence generation for HWT.The market for health and welfare technology is growing rapidly, particularly in Sweden. At the same time, evidence requirements during public procurement of HWT in Sweden occur in only a small portion of all requests for tenders, which may result in a legacy of ineffective applications, risks, and excess costs. Policy makers and procurement officials should take steps to establish transparent and rigorous processes for evidence requirement and assessment with the MDR as a starting point. These stakeholders should also, together with HWT developers and researchers, work together to achieve high-quality, locally relevant evidence generation when purchasing and employing HWT.


Digital evidence, such as evidence from CCTV and event data recorders, is highly valuable in criminal investigations, and is used as definitive evidence in trials. However, there are risks when digital evidence obtained during the investigation of a case is managed through a physical hard disk drive until it is submitted to the court. Previous studies have focused on the integrated management of digital evidence in a centralized system, but if a centralized system server is attacked, major operations and investigation information may be leaked. Therefore, there is a need to reliably manage digital evidence and investigation information using blockchain technology in a distributed system environment. However, when large amounts of data—such as evidence videos—are stored in a blockchain, the data that must be processed only within one block before being created increase, causing performance degradation. Therefore, we propose a two-level blockchain system that separates digital evidence into hot and cold blockchains. In the criminal investigation process, information that frequently changes is stored in the hot blockchain, and unchanging data such as videos are stored in the cold blockchain. To evaluate the system, we measured the storage and inquiry processing performance of digital crime evidence videos according to the different capacities in the two-level blockchain system. Recently, digital evidence has been used as definitive evidence in criminal investigations and trials. For instance, in 1985, a murder occurred at a rest stop in Oklahoma, USA, but a strong suspect could not be found. However, DNA from cigarette butts smoked by the murderer at that time was stored until 35 years later, when the criminal could be charged . Thus, iHowever, because CCTV and EDRs have been proven to provide important evidence in recent criminal investigations, the necessity of safely managing digital crime evidence videos is also increasingly clear . When laThe rest of this paper is organized as follows: Digital forensics deals with legal evidence left in computers and digital recording media. This is a procedure and method for legally identifying and demonstrating the facts of a specific activity that occurred using a digital device. Digital forensics tools are used daily by inspectors and analysts in local, state, and federal law enforcement agencies. More and more organizations encounter data that cannot be analyzed with today’s tools, because of format incompatibility, encryption, or a lack of education . CurrentCurrently, the criminal investigation process in Korea for digital evidence management consists of seven steps: initial investigation, internal investigation, arrest, completion of investigation, request for analysis, transmission of digital evidence and investigation documents, and trial . The firStudies on ways of improving the original identity, integrity, and reliability of criminal digital evidence obtained in the investigation process in Korea have been conducted ,5. HowevHowever, in previously proposed integrated digital evidence management systems in Korea, a server was operated as a centralized system in an integrated server/client environment. A centralized system is a system that processes all data from each region on a central computer. In other words, it refers to a system in which one specific computer processes data, instead of using all the computers in a server system connected through a communication network to process data. A centralized system has the advantages of easily integrated data management, and low management costs, because the data are concentrated on a central server. However, in such a system, when a server is attacked, important operations and investigation information of the relevant institution may be leaked, and the continuity of storage cannot be maintained for digital evidence. Therefore, digital evidence and investigation information must be managed reliably and transparently in a distributed environment, not in the existing centralized digital integrated management system.With the recent developments in the field of big data, many studies have investigated the storage and management of large amounts of data, such as videos and images, in blockchains in various fields, such as medicine, forensics, and education ,17,18,19However, a blockchain is basically a structure in which all nodes record the shared data on the network. In other words, each node has a repository that stores data, called a distributed ledger, which is placed on multiple nodes. When there is a written request from the user, it broadcasts the status to all nodes so that each node stores the same state. A transaction in a blockchain means that a certain state and data are written onto the blockchain. Currently, one of the methods of sharing the same state for each system, as well as making contracts and agreements, is blockchain technology. However, in the blockchain, even if a number of physical computers are connected, the calculations are concentrated on a single node, and the scalability of block capacity becomes a problem ,23. To address this problem of blockchain scalability, research on a hierarchical blockchain model has been studied . Each leResearch also has recently been attempted on on-chain scaling, which records data within the blockchain, and off-chain techniques, which record data outside the blockchain ,27. On-cCurrently, most methods of storing and managing large amounts of data in the blockchain do so using the IPFS (InterPlanetary File System), which is an off-chain technique. The IPFS is a distributed file system designed to improve the speed of the request/response method, which is a problem with traditional HTTP Web communication. The IPFS allows large data files to be exchanged using peer-to-peer communication on nodes without centralized servers. Unlike HTTP Web, the IPFS can maintain a system with small nodes whether the system is compromised or not. Moreover, because it uses a file system based on the Merkle DAG , its filA blockchain is a decentralized system in which each distributed computer handles tasks and exchanges its content or results, in contrast to focusing on a centralized computer to process all data as in a centralized system . In otheHyperledger Fabric, one of the consortium blockchains, is a Linux Foundation Hyperledger project. It is an open-source framework for building blockchain infrastructure for business-to-business and business-to-consumer transactions . Unlike Hyperledger Fabric consists of a blockchain network, a certificate authority (CA) server, and a client, as shown in Our proposed two-level blockchain system for efficient crime evidence management consists of two layers, managed by hot and cold blockchains . In the Algorithm 1 Save and Search of Hot & Cold BlockchainInput: Hot_Tx // Investigation and identity transaction    Cold_Tx // Evidence video and identity transaction    K // User Private keyoutput: Save Success or Failure     Search Result of Hot_Tx and Cold_Tx02 Cold_ledger  ← NULL // Cold Blockchain Ledger03 D  ← List // User Digital Certificate04 function Save05   I ← 0 // User index06   V ← 0 // Valid flag07   F ← 2 // Full flag08   while D[I] do09    if D[I] ∋ K && !is Empty(HoT_Tx)10      V ← V + 111      Append12    end if13    if D[I] ∋ K && !isEmpty(Cold_Tx)14      V ← V + 115      Append16    end if17    I ← I + 118    end while19   while true do20    if V = F return true21    else return false22    end if23   end while24 end function25 function Search26   I ← 0 // User index27   R1 ← 0 // Hot_Tx28   R2 ← 0 // Cold_Tx29   while D[I] do30    if D[I] ∋ K31      if Hot_Tx[I].ID and RNN= Hot_Ledger[i].IDand RNN32       R1 ← Hot_Ledger[I]33      end if34      if R1. EVI_ID = Cold_Ledger[I].EVI_ID35       R2 ← Cold _Ledger[I]36      end if37     I ← I + 138   end while39   return R1, R240 end function01 Hot_ledger  ← NULL // Hot Blockchain LedgerInstitutions corresponding to the national police agency, local police agency, cyber analysis team, prosecutor’s office, and courts—which are the peers in the channel—would form a consortium in order to participate in blockchain channels and share the identity and investigation information in the hot blockchain using the process shown in To manage digital crime evidence videos, we can access the cold blockchain using the key used to verify the identity of the police agency peer in the hot blockchain. After identity verification, the on-site investigator obtains the right to create and distribute blocks by storing the original digital crime evidence video in the cold blockchain. Then, frequent transactions during the investigation process can access the hot blockchain in order to request the identity and investigation information, as well as accessing the cold blockchain through the evidence ID in order to retrieve evidence. In addition, it is impossible to modify a digital crime evidence video that is not allowed to be modified. In this way, the data stored in the hot and cold blockchains are created in blocks and cannot be modified or deleted, and they are shared with all peers in the blockchain system, thus enhancing transparency and reliability.In the system proposed in this paper, we use Hyperledger Fabric, a blockchain construction framework, and Docker, a software virtualization framework, to implement the two-level blockchain. When managing a blockchain, a number of nodes are connected within the communication network in order to exchange data, so physical computers for each configured peer are required. To do this, a number of peers, authentication servers, orderers, and blockchain state databases are created as containers using Docker, and the blockchain infrastructure is maintained by mounting ledgers, blocks, transactions, and other components to the built container. In the two-level blockchain system, PKI-based encryption keys are issued and used as credentials, so that peers corresponding to the local police agency, cyber analysis team, prosecutor’s office, police agency, and courts can form a consortium and participate in the same channel. To enable the on-site investigator to store investigation and identity information and digital crime evidence videos in the hot and cold blockchains in the two-level blockchain system, the identity is registered in containerized peers and orderers. To this end, containerized peers participate in the channel to form a consortium, and an identity registration request is made to the authentication server using a PKI-based administrator digital certificate and private key. Subsequently, the verified on-site investigator can access the two-level blockchain and obtain the authority to store investigation and identity information, as well as crime evidence videos. The on-site investigator delivers the data to the peer of the national police agency to which he or she belongs, in order to store investigation and identity information and video clips of criminal evidence in the blockchain. At this time, the investigation and identity information are transferred to the hot blockchain, and the crime evidence video is transferred to the cold blockchain chaincode factor to execute the contract. The cold blockchain, which is a core part of the system proposed in this paper, is accessed using the same digital certificate and key used when requesting identity registration for the hot blockchain. Subsequently, the on-site investigator stores the crime evidence video by passing the digital crime evidence video and part of the identity information stored in the hot blockchain as chaincode factors of the cold blockchain. In the chaincode of the cold blockchain, the crime evidence video delivered by the on-site investigator is sequentially delivered, and the chaincode is executed. When investigation and identity information, along with crime evidence video transactions, are shared with all of the peers in the two-level blockchain, all of the nodes of each blockchain reach consensus to create a block. At this time, all peers participating in the channel share the same transaction and block. In contrast, if the two-level blockchain is accessed using an unregistered identity, the two-level blockchain system cannot verify the identifier, and the chaincode cannot be modified, as shown in The environment and system configuration for testing the performance of the two-level blockchain system are shown in Hyperledger Caliper’s performance evaluation index supports tests on transaction read throughput, latency, CPU and memory use, and network I/O resource use. Components include the benchmark layer, interface and core layer, and adapter layer. The benchmark layer defines the backend blockchain network and tests arguments to measure performance, while the interface and core layers query the ledger status of the backend blockchain, invoke smart contracts, and generate performance measurement results in HTML format. The adapter layer measures the experiment by interacting with the blockchain and Caliper . To measInit, Run, and End functions. The Init function consists of a blockchain object and context, as well as user-defined arguments, and is called at the beginning of each experiment round. The Run function transfers crime evidence videos, investigations, and transactions of identity information to the system, while the End function is called at the end of each experiment round to initialize the corresponding function. At this time, all functions in each calling code are processed in an asynchronous manner in order to control response requests and delays occurring between nodes.Next, to deliver multiple transactions to the blockchain and receive responses, a rate controller module is defined in order to deliver transactions at fixed intervals, designated as transactions per second (TPS). In addition, the calling code is implemented so that the experimental module, acting as a real client, can create and deliver transactions. The calling code consists of the To evaluate the performance of the two-level blockchain system, we evaluated the performance of the storage function and inquiry function of the hot and cold blockchains. To this end, crime evidence videos with capacities of 100 MB to 1 GB and 1 GB to 5 GB, which were selected only in the areas necessary for the facts of presumption, were stored and searched for on the cold blockchain. In the case of crime evidence videos to be stored in the cold blockchain, the capacity was increased in units of 100 MB from 100 MB to 1 GB, and in units of 1 GB from 1 GB to 5 GB, and the blocks were created by repeatedly submitting 100 transactions for each capacity. In addition, for the identity and investigation information, for which transactions in the hot blockchain are frequently stored, the amount of data registered when investigating a criminal case in Korea was assumed to be 1 KB. The transmission rate was increased, and 100 transactions were repeatedly submitted in order to create a block. At that point, the TPS was measured by calculating the total number of transactions submitted, the time required, and the transaction processing delay time of the crime evidence videos, as well as identity and investigation information stored in each of the hot and cold blockchains. The TPS is the number of transactions that can be processed in one second, and is a representative blockchain performance evaluation index that indicates how many contracts the corresponding blockchain system can process. The transaction processing delay time refers to the delay until a transaction is received and the chaincode is executed. This is used to measure the number of transactions processed per second. Transaction throughput per second calculates the number of transactions processed per second using the first transaction submission time and the last transaction submission time, which is the transaction time, the number of successful and unsuccessful submissions, and the transaction processing delay time. Caliper’s rate control module can control the transaction submission rate, number of submissions, and performance calculations. Subsequently, if the experimental module acting as a client repeatedly submits each transaction, including crime evidence videos and identity and investigation information, to the hot and cold blockchains 100 times for 5 rounds, the two-level blockchain system will, upon request, execute the chaincode and create 500 blocks in the configured node. At this time, information such as the number of successful and unsuccessful transaction submissions, submission time, and response time is communicated with the adapter layer in real time during the series of processes from transaction request to block creation. The experimental results shown in According to the experimental results for the hot blockchain, the storage function increases in steps of 100 TPS, from 100 TPS to 500 TPS, as shown in Currently, the typical blockchain systems, Bitcoin and Ethereum, operate at 7 TPS and 20 TPS, respectively, for cryptocurrency transactions for an unspecified number of people. This is numerically superior to the performance of the proposed two-level blockchain system. However, because of the nature of a crime evidence management system, this system can provide similar performance to Bitcoin while storing frequent transactions and large amounts of data separately if the blockchain is accessed by a limited number of users. In addition, because of the technical characteristics of a blockchain, the original crime evidence video can be safely stored and managed inside the blockchain.The digital crime evidence data obtained during the investigation of a criminal case is transmitted and managed on a physical hard disk until it is analyzed and submitted to the court, and there is a risk that it can be damaged or manipulated by an attack. Therefore, digital crime evidence videos that were difficult to obtain and analyze are not adopted as court evidence, because the continuity of storage cannot be guaranteed. Criminal evidence data are not modified once stored, but investigation information is frequently modified. In this paper, we proposed a two-level blockchain system to increase the integrity of digital crime evidence, so as to efficiently manage criminal evidence. In the proposed system, only authorized participants can access the hot and cold blockchains, in a decentralized environment to separate, store, and share the original information of the investigation, identity information, and digital crime evidence videos. The two-level blockchain system stores the investigation and identity information, as well as digital crime evidence videos, by on-site investigators with verified identities. In addition, by separating data into two blockchains, the same transaction can be stored in the ledgers of all institutions participating in the channel, and the same block can be generated by executing a smart contract. Investigation and identity information, as well as digital crime evidence videos, once created in blocks, cannot be deleted by any user. In addition, because the block is shared with all institutions in the two-level blockchain system, transparency and reliability are enhanced. We also evaluated the performance of the proposed system. According to the experimental results, when the storage performance of the cold blockchain increased from 100 MB to 1 GB in units of 100 MB, the average performance decreased by 0.11 TPS. In addition, it was confirmed that when the capacity increased by 1 GB from 1 GB to 5 GB, the decrease was 0.7 TPS on average. The query performance of the cold blockchain had an average decrease of 1.4 TPS when increasing in steps of 100 MB, from 100 MB to 1 GB, and an average decrease of 6.8 TPS when increasing in steps of 1 GB. In addition, the experimental results of the hot blockchain revealed that the storage function decreased the TPS by 3.82 when the transmission rate increased from 100 TPS to 500 TPS in units of 100 TPS, and the inquiry performance increased by 79.8 TPS. This is the difference according to the capacity width of the digital crime evidence video used in the experiment, and the transmission rate of the identity and investigation information, and it was confirmed that the actual uniform capacity and the TPS according to the increase in transmission rate were predictable.The proposed system is suitable for a system that can search while storing data with large files for recording, and ensure the integrity of the data. It can be applied to other applications considering data with similar characteristics, but there is a limitation in applying it to various applications in general. Additionally, for the two-level blockchain system to be used as an actual digital evidence management system, further performance improvements will be required. To this end, as future work, we must first perform an experiment to show the effect of the number of peers in the network. We also intend to research a two-level blockchain system that can respond to high transaction transmission speeds. We also plan to lower the code complexity within the cold blockchain. In the future, it is expected that the proposed two-level blockchain system will contribute to lowering the threat that exists in the transmission and management of digital evidence in Korea, and increasing reliability as a result.