Étude d’évaluabilité et évaluation des processus et des effets d’un programme de prévention de l’usage de substances psychoactives lors de la transition primaire-secondaire
Implementation of school-based substance use prevention activities is a widespread strategy to reach out to a significant number of young people. These activities are based either on existing programs or innovations dictated by the context of implantation or the existence of research data. In one case or another, evaluation of these programs is the best way to measure their effectiveness and/or to know how they operate. It is this need that motivated a school board in Quebec to recommend the evaluation of the Intervention Network (IR), a program developed to delay the age of substance use onset and to reduce problematic use of substances among students. This program takes an innovative approach with a patient navigator (PN) as the main implementation actor who keeps track of students from grade 5th to 9th. Inspired by such models in the fields of health prevention and the Healthy Schools Approach (HSA), the role of the PN here differs from those fields. S/he serves as an interface between the various stakeholders and students but in the IR program, the PN is integrated in primary and secondary schools involved in the program. The PN is also the one who ensures mobilization of other actors for the implementation of the program activities. This thesis aims to provide an account of the evaluation process and the results obtained. The evaluation approach is participatory and collaborative and combines both quantitative and qualitative data collected through questionnaires, interviews, focus groups, a logbook and meeting notes. Data were analyzed in the framework of three articles, the first being an evaluability assessment (EA) of the program. Participants of this EA are key-informants involved in the program implementation (n=13). A document review (report, logbook) was also conducted. The EA was helpful to clarify the intentions and objectives of the initiators of the program. It helped provide explicit information on the program theory and develop the logic model, two elements that facilitated subsequent evaluation operations. The second article is a process evaluation that uses Actor-Network Theory (ANT) as analytical framework, through the four moments of innovation translation (problematization, interessment, enrollment and mobilization of allies), controversies analysis, and the role of human and non-human actors. After analysis of the 19 interviews and available documents, results showed that the program implementation phases adequately follow the four moments of the ANT, that management of controversies by negotiation and support was needed for the mobilization of some human actors. This process evaluation also highlighted the role of non-human actors in the implementation process.  The last article is a combined evaluation of the effects (quantitative component) and the process (qualitative component) of the program. For the quantitative component, a quasi-experimental design was used and data were collected longitudinally by questionnaires from 901 students of 5th and 6th grades and their teachers from 2010 to 2014. Data analysis indicated that the program had no effect on the accessibility and perceived risk, problematic use of alcohol and use of both alcohol and cannabis among participants. On the other hand, results suggest that the program could help reduce the level of consumption and delay the age of onset of alcohol and cannabis. These results also suggest a potentially positive effect of the program on alcohol intoxication among students. As for the qualitative component, stakeholder interviews (N = 17) and focus groups with high school students (N = 10) were conducted. Program documents were also analyzed. Results of this second process evaluation showed that the program is highly viewed and positively appreciated by the different actors involved in the program. However, frequent staff turnover and the number of schools to follow can be obstacles to the smooth running of the program. Nevertheless, the leadership role and support from the principals, teacher’s collaboration, the quality of the PN and flexibility in the implementation process are identified as factors contributing to the success of the program. All the results and their implication for program implementation and evaluation are discussed. Finally, to ensure greater access to the results, this thesis concludes with a plan for transferring knowledge drawn from the evaluation research.
Program evaluation Evaluability assessment Process evaluation Impact evaluation Actor-Network Theory Sociology of translation Mixed-methods Psychoactive substances Prevention Primary and secondary schools Knowledge transfer