This technology-mediated intervention increased parents' confidence and motivated them to speak more freely about HPV vaccination in-person and online with others in their social networks. Participants identified prevalent misinformation about HPV vaccination and learned how to effectively craft messages to address concerns related to safety and side effects, gender, understanding of risk, and sexual activity. Objective measures and qualitative open-ended assessment showed high intervention engagement and treatment satisfaction. All participants (100%) indicated that they enjoyed participating in the intervention. The effectiveness of this feasibility study suggests that social media is an appropriate platform to empower parents to counter vaccine hesitancy and misinformation through HPV vaccination information that is simple and shareable in-person and online.Human papillomavirus (HPV) vaccination prevents 6 HPV-related cancers in men and women. Yet, rates of HPV vaccination among adolescents in the United States lag behind other developed nations, revealing a significant public health issue. This feasibility study tested a collaborative online learning environment to cultivate HPV vaccination champions. A 3-month training program recruited parents to serve as proponents and social media influencers to identify solutions to overcome barriers to HPV vaccination. A mixed methods study design included a pretest survey, three online asynchronous focus groups, a posttest survey, as well as a longitudinal follow-up survey at 6 months. Participants included 22 parents who self-identified as female (95.4%) and white (90.9%). Overall, there was a statistically significant difference in knowledge of HPV and HPV vaccination between pretest and posttest ( The human papillomavirus (HPV) is the most prevalent sexually transmitted infection (STI) in the United States, with 79 million Americans currently infected with the virus . The majThe HPV vaccine is critical to reduce HPV infection rates and HPV-related cancers. A vaccine to prevent HPV has been available in the U.S. since 2006. Gardasil®9  has been offered in the U.S. since 2016 and is currently the only HPV vaccine available in the U.S. The Centers for Disease Control and Prevention (CDC) recommend that all children ages 11 and 12 receive two doses of HPV vaccine. Adolescents who receive the first dose of the HPV vaccine at age 15 or older or who are immunocompromised require three doses. The HPV vaccine is recommended for all men and women up to age 26 and is approved for some people up to the age of 45  of Facebook and Twitter. A screening tool was used prior to enrollment in the study. For this feasibility study, we recruited participants who were committed to increasing HPV vaccination and dedicated to starting conversations (online and in-person) and answering questions about HPV vaccination in their social networks. Participants were recruited through word of mouth, email messages, social media posts, and at relevant meetings and events. As a result, snowball sampling occurred when participants recommended additional participants. Participants received an incentive for their time and effort in the study, including $100 after completing the 3-month training and $20 for completing the longitudinal follow-up at 6 months. Informed consent was obtained through Qualtrics by all parents prior to participation.Participants included parents living in South Carolina who were committed to increasing HPV vaccination and were The development of the intervention was informed by best practices in implementation science , 30 and The goals of the 3-month online training were to help HPV vaccination champions explain HPV vaccination recommendations, discuss the importance of HPV vaccination and the risks of HPV-related cancers and disease, describe ways an ambassador can increase HPV vaccination by supporting vaccination and overcoming barriers, provide examples of activities to engage in as an ambassador , and understand resources to support champions. The webinars were designed to respond in real time to the questions and concerns participants expressed during online focus groups and the private Facebook group.Participants completed a questionnaire at baseline (pretest) and post-intervention at 3 months (posttest) and 6 months . The questionnaires investigate awareness and knowledge of HPV and the HPV vaccine, attitudes and beliefs about HPV and the HPV vaccine, and behavior/behavioral intention regarding HPV and the HPV vaccine. The measures were drawn from a variety of sources and have proven to be reliable and valid, including the Health Information National Trends Survey (HINTS), the Behavioral Risk Factor Surveillance System (BRFSS), TTM, HBM, and surveys of young adult populations –38, whicEngagement was assessed objectively on web-based platforms (Facebook and MailChimp). Engagement with bi-weekly emails was defined as the number of participants who opened the email, which was obtained from MailChimp metrics. Engagement with the private Facebook group was measured by the number of interactions, including post likes, comments, and original posts.The online focus groups and the posttest and longitudinal follow-up questionnaires assessed how satisfied participants were with the training by rating its overall usefulness and likelihood of recommending it to a friend. Level of satisfaction with specific intervention components  was also reported. All items were rated on an agreement-oriented 7-point Likert scale anchored with strongly agree and strongly disagree.T-test was used to compare the pretest and posttest composite score for the overall scale, given the continuous distribution of these data. Wilcoxon Signed-Rank tests were used to compare pretest and posttest responses of HPV and HPV vaccination attitudes. Responses were ranked in order of 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree, and 6 = don't know. For analysis, agree included responses of “strongly agree” and “agree.” Disagree included responses of “strongly disagree” and “disagree.” For analysis of HPV vaccination influence, influence included responses of “strongly influenced” and “influenced.” For analysis of participant intervention experience, extremely included Likert scale responses of “4” and “5.” A p-value ≤ 0.05 was used to determine statistical significance for all analyses. All data analyses were conducted using SAS statistical software version 9.4 .All data were reported as frequencies and response rates were reported as percentages of the total sample population. Basic descriptive statistics were used to describe the sociodemographic characteristics, participants' use of technology and participant retention rates. For analysis of technology use, cell phone included responses of “receive a text message on a cell phone,” “send a text message on a cellphone” and “use a cellphone to make or receive a voice call.” Hourly included responses of “about once an hour” and “more than once an hour.” Daily included responses of “about once a day” and “several times a day.” Weekly included responses of “up to about once a week” and “a few times a week.” Friedman's test was used to compare pretest, posttest and 6-month follow-up responses within participants' intention to vaccinate their child against HPV. Responses were ranked in order of 1= “I am unsure about my intention to get my child vaccinated” to 11= “My child has received all three shots of the HPV vaccine.” Exact McNemar's tests were used to compare pretest and posttest responses for each question of HPV and HPV vaccination knowledge. Responses were recoded as binary to designate correct answers as “1” and incorrect answers or “don't know” as zero. The overall average of correct responses was analyzed from composite scores calculated from the sum of correct responses per individual. A paired Participants were invited to participate in three online asynchronous focus groups. Focus groups allow researchers to better understand socially constructed understandings of HPV and HPV vaccination. The first focus group addressed knowledge, attitudes, and beliefs about HPV vaccination. Questions included myths/misunderstandings about HPV vaccination. The second focus group addressed communicating about HPV vaccination, including challenges and opportunities to improve in-person conversations and constructing media messages. The third focus group asked specifically about the HPV vaccination champions intervention, including visibility of messages in the community, perceptions of messages, needs/preferences for future messaging, successes and opportunities for improvement.Research shows that online focus groups include many advantages, such as convenience, accuracy of data, low costs, expanded geographic range, and increasing access to specific types of participants , while preserving the quantity and quality of data collected during in-person groups , 40. In Qualitative data analysis of the online focus groups was conducted using a constant comparative method . ResearcThis study was approved by the Institutional Review Board at the College of Charleston.Participants included 22 parents with a median age of 40.2 ± 6.6 years. Most participants self-identified as female (95.4%) and white (90.9%). Participants lived in counties across South Carolina with representation from each of four regions in the state, including the Upstate Region, Midlands Regions, Lowcountry Region, and Pee Dee Region. All participants reported private health insurance coverage. Participants' education ranged from a high school diploma (4.5%), some college (9.1%), an undergraduate degree (27.3%), some graduate education (13.6%), and a graduate degree (45.4%). All participants reported an annual household income above $30,000 with the majority reporting $70,000 or more (63.6%). Among participants, 62.6% reported using a cell phone every day and 30.3% reported using it every hour. Almost half of participants (45.5%) reported using a computer every day with 9.1% reporting hourly use. The majority of participants reported using multiple devices every day (59.1%) with 22.7% reporting hourly use of multiple devices. All participants completed the posttest survey and the retention rate at the 6-month longitudinal follow-up was high (82%) see .n = 11; 50%) reported “my child has received all shots of the HPV vaccine series,” 9 participants (40.9%) indicated “I plan to get my child vaccinated at the recommended age,” and 2 participants (9.1%) reported “I am unsure about my intention to vaccinate/I do not plan to get my child vaccinated in the next 6 months”   knew that HPV can be spread through sexual intercourse, HPV can cause an abnormal Pap  test, and some types of HPV can cause cervical cancer. Fewer participants were aware that HPV can be spread through contact other than sexual intercourse (77.3%) and that some types of HPV can cause oral cancer (81.8%). Participants reported changes in knowledge from pretest to posttest, particularly learning that “some types of HPV can cause anal cancer,” “condom use does not fully protect against the spread of HPV” and “an HPV infection cannot be cured.” Overall, there was a statistically significant difference in the average of correct answers from pretest and posttest ( 0.0042) .Participants reported high perceptions of HPV vaccination benefits, barriers and severity; however, they reported low susceptibility. Parents' attitudes about HPV and HPV vaccination mirrored the constructs of the Health Belief Model except related to perceptions of susceptibility. There were no statistically significant differences between responses .At baseline, participants identified factors that influenced their HPV vaccination decision. “Concerns about my child getting other HPV-related cancer” and “concerns about my child getting HPV” were the most frequently identified statements .Parents supported the HPV vaccination as cancer prevention. According to one participant, “my daughter was immunized for HPV: Vaccine vs. cervical cancer. The choice is clear.” Another parent said, “I support the HPV vaccine because I'm all for cancer prevention.” Among these supporters of HPV vaccination, the parents agreed, “any opportunity to prevent cancer is foolish not to take advantage of.” Some participants shared a personal connection to HPV or HPV-related cancer that increased their commitment to vaccination. According to one participant, “…I've seen how devastating cervical cancer is. My aunt's MIL got it in her forties. She had so many complications from the cancer and treatments and ended up dying within 2 years of being diagnosed.” Another parent wrote, “…being someone who has HPV, I am definitely for the vaccine. My daughter has already had the vaccine and my son will when he is old enough.” Prior to the intervention, participants already knew that HPV was ubiquitous and that it caused precancerous cervical lesions, as well as cancer. For example, one participant wrote, “I support the vaccine because I think it is important! HPV is so prevalent and yet easy to protect yourself.” Another parent supported the vaccine because, “I've known too many young women scared because of precancerous cells due to HPV.” Among this group of parents, HPV vaccination was common sense. According to one parent, “it seems obvious to me that if you have a means to protect yourself and your children then you should take advantage of it, therefore, vaccinate.”Parents discussed conversations they had in their everyday lives about HPV vaccination. Participants revealed the most common barriers to HPV vaccination among parents in their social networks. Misinformation about HPV vaccination emerged related to safety and side effects, gender, understanding of risk, and sexual activity. A number of participants described vaccine hesitancy toward specific vaccines, including the HPV vaccination. According to one parent:A lot of people I talk to seem to break vaccines down into two groups, the ones that they consider absolutely necessary  and the ones that they consider less important/optional . Often HPV is grouped in the second category and the reasoning is often based on fear of side effects combined with a lack of appreciation of the impacts of the diseases.Another participant concurred, “while they are generally pro-vax they've heard that this one has a lot of documented injuries associated with it.” Parents described hearing comments about injuries and deaths related to the HPV vaccine.Understanding the HPV vaccine as a gender-specific vaccine was also a prominent theme among parents in the participants' social networks. According to one participant, “parents of boys don't see how it applies to their son at all.” Participants also believed that many parents continued to link the vaccine with sexual activity. Parents explained how this opinion impacted vaccine decision-making, “it isn't relevant for preteens since it is sexually transmitted,” and “their kid is too young for sex yet anyway.” According to one participant, “I usually hear it is because their kid is not sexually active or out of fear that will condone their younger child to become sexually active.” Another parent offered a compassionate response to this concern:I have heard many mothers say they didn't want their child to get the vaccine because they weren't sexually active and didn't plan to become sexually active for years. A pediatric nurse practitioner that I work with has always had the best response to that with “Your daughter is a princess but she may not marry a prince 1 day.” That usually resonates with parents and they end up choosing to vaccinate.Participants suggested parents who have opted for the vaccine have not been as vocal about it as those with concerns. The prevalence of the virus itself and HPV-related cancers was also overlooked in parents' social networks. According to one participant, “as a parent, I know cancer is a worry for pretty much all parents. But I don't think parents realize how common HPV related cancers are.”The electronic newsletter showed moderate penetration with an average of 47.2% of participants opening the bi-weekly email see . The majOverall, participants rated the intervention positively see . All parThrough the online focus groups and open-ended responses on the posttest and longitudinal follow-up surveys, parents unanimously described the benefits of the intervention. Participants appreciated the ease of use and convenience of the private Facebook group, which streamlined seamlessly with their existing social media habits. According to one participant, “I liked the interaction and instruction on Facebook.” Participants demonstrated an increase in knowledge about HPV vaccination. One participant wrote, “I definitely know a lot more about HPV and the HPV vaccine .” Parents described improved confidence and the ability to talk more freely with other parents. According to one participant, “one of the main benefits to me was getting confident with the facts about the HPV vaccine and HPV vaccination rates in S.C.” Another parent wrote, “it has helped to open conversations and given me the opportunity to educate others.” The longitudinal follow-up showed that participants were still using the training 6 months later by filming videos as advocates, joining advocacy groups, and holding many discussions about HPV vaccination with the people in their lives. The training also offered unexpected opportunities for participants to improve communication about HPV vaccination with their children and their patients. According to one participant, “I am a nurse practitioner and going through this program helped me relay to my patients the importance of the HPV vaccine.” Participants expressed gratitude for the program and emphasized how useful it was to them personally. They particularly enjoyed the aspects of the program focused on improving online and in-person communication and suggested the program include more of this practical skills-based training, such as crafting social media posts.p = 0.0042)  . In line . In linParticipants' intervention engagement and treatment satisfaction indicate that this approach provides utility and scalability among parents. All participants (100%) indicated that they enjoyed participating in the intervention. The electronic newsletter showed moderate penetration with an average of 47.2% of participants opening the bi-weekly email. However, this information was repeated in the private Facebook group, offering two ways for participants to engage with the material at their convenience. All Facebook posts had one or more interactions by participants with an average of 3.3 comments per post. Parents reported that Facebook was easy to use, convenient, and provided an optimal platform for instruction and interaction. This finding expands existing research demonstrating that interactive forums empowered parents to express vaccine concerns and offered opportunities to provide answers in real time .Participants described how other parents in their social networks displayed vaccine hesitancy toward the HPV vaccination despite an overall pro-vaccination attitude. This finding builds on limited research aimed at determining different types of vaccine hesitancy . ParentsMost parents 86.4%) believed that the intervention resulted in improved confidence and the ability to talk more freely about HPV vaccination with other parents in-person and on social media. This finding supports the effectiveness of connecting vaccine-interested parents with those who are like-minded in order to assist them in countering vaccine hesitancy and misinformation on social media .4% belie. ParticiThis mixed methods study offers an innovative approach to reach parents to overcome barriers to HPV vaccination. Participants represented counties across four regions of South Carolina. However, the homogeneity of the population and small sample size limit the generalizability of the results. Future studies should purposively sample diverse parents, especially in terms of race and ethnicity. Researchers may consider less stringent screening criteria, increased participant incentives, more targeted recruitment, and a budget for recruitment to improve diversity in future studies. Another limitation of the study is the largely single gender population of mothers. Although future studies may seek to incorporate more fathers, it is important to continue to target women since mothers serve as the primary decision makers for adolescents receiving HPV vaccination –15. It iThis feasibility study offers a model of cultivating HPV vaccination champions in a community setting and demonstrates potential for scalability and dissemination of this intervention approach –27. In tA technology-mediated intervention for parents increased their confidence and motivated them to speak more freely about HPV vaccination in-person and online with others in their social networks. The collaborative online learning environment cultivated HPV vaccination champions through a 3-month training program that supported parents to serve as proponents and social media influencers to overcome barriers to HPV vaccination. Participants identified prevalent misinformation about HPV vaccination and learned how to effectively craft messages to address concerns related to safety and side effects, gender, understanding of risk, and sexual activity. Objective measures and qualitative open-ended assessment showed high intervention engagement and treatment satisfaction. The effectiveness of this feasibility study suggests that social media is an appropriate platform to reach parents with HPV vaccination information that is simple and shareable in-person and online. This study combined education and health promotion messages with skills-based communication training to empower parents to raise their voices in support of HPV vaccination.BLS@cofc.edu.The datasets presented in this article are not readily available because the informed consent procedure guaranteed participants in this study that only authorized study staff would have access to the data. Requests to access the datasets should be directed to The studies involving human participants were reviewed and approved by The Institutional Review Board (IRB) at the College of Charleston, Charleston, SC. The patients/participants provided their written informed consent to participate in this study.BS and KC contributed to conception and design of the study. NR organized the database and assisted with project management. AW assisted with project management and conducted quantitative data analysis. BS performed the qualitative analysis. BS and NR wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Objectives: To develop and test a human papillomavirus (HPV) vaccination intervention that includes healthcare team training activities and patient reminders to reduce missed opportunities and improves the rate of appointment scheduling for HPV vaccination in a rural medical clinic in the United States.Methods: The multi-level and multi-component intervention included healthcare team training activities and the distribution of patient education materials along with technology-based patient HPV vaccination reminders for parents/caregivers and young adult patients. Missed vaccination opportunities were assessed pre- and post-intervention  by retrospective chart review and compared using Pearson χ2. The patient parent/caregiver and young adult patient population (n = 80) was surveyed following the reminder messages and penalized logistic regression quantified unadjusted odds of scheduling a visit.Results: Missed opportunities for HPV vaccination declined significantly from the pre-intervention to the post-intervention period . Participants who recalled receipt of a vaccination reminder had 7.0 (95% CI 2.4–22.8) times higher unadjusted odds of scheduling a visit compared with those who did not recall receiving a reminder. The unadjusted odds of confirming that they had scheduled or were intending to schedule a follow-up appointment to receive the HPV vaccine was 4.9 (95% CI 1.51–20.59) times greater among those who had not received the vaccine for themselves or for their child.Conclusions: Results from this intervention are promising and suggest that vaccination interventions consisting of provider and support staff education and parent/caregiver and patient education materials, and reminders can reduce missed opportunities for vaccinations in rural settings. Improving human papillomavirus (HPV) vaccine uptake could prevent tens of thousands of cancer cases each year. HPV is a sexually transmitted infection that causes cervical, anal, penile, vaginal, vulvar, and oropharyngeal cancers, and genital warts. While HPV vaccination can prevent most HPV-related cancers, more than 20,000 women and 14,000 men are diagnosed with HPV-associated cancers each year in the United States . ImproviDeveloping and deploying effective strategies for improving rural HPV vaccine coverage is critical.Higher HPV vaccine uptake in rural communities could improve health outcomes for this unique population. The United States non-metropolitan residents have higher cervical cancer incidence, late-stage diagnoses, and death rates than metropolitan residents . Data frRural barriers to HPV vaccination are multi-factorial. A robust body of literature explicates barriers in receiving preventive health services, such as vaccination for HPV –11 and uComplicating the situation further, in March 2020, the WHO classified COVID-19  as a global pandemic leading to an unprecedented strain on the U.S. healthcare system ; neverthMulti-level interventions are needed to reduce HPV vaccination disparities. Solving complex public health issues requires consideration of factors at multiple levels, such as the individual , clinician , clinic , community, and society . Current studies find that multi-level interventions display a synergistic effect, with interventions targeting parents and providers achieving higher levels of HPV vaccine uptake than interventions targeting each group alone. However, many of these past studies have not been rigorously tested in rural populations.This study developed and tested a multi-level and multi-component intervention in a rural Telluride, Colorado, United States. The intervention consisted of healthcare team training activities to strengthen and increase strong and consistent HPV vaccination recommendations and evidence-based patient-directed HPV vaccine education materials as well as technology-based parent/caregiver reminders to make vaccine appointments for their age-eligible children. This study sought to fill important gaps in the literature, including exploring the impact of combining technology-based interventions with human-delivered communication. Few previous studies have tested the combination of strong provider recommendations for the vaccine with automated reminders to patients.® (eCW) for the electronic health record (EHR). They have prioritized increasing HPV vaccination rates in 2017 when they began collaborating with Dr. Kepka, an associate professor at the University of Utah, United States and an investigator at Huntsman Cancer Institute, and the Intermountain West HPV Vaccination Coalition (led by Dr. Kepka)  serves rural patients in the western United States. Telluride, Colorado is a rural community of <2,500 residents. In addition, Telluride is a mental health and primary care Health Professional Shortage Area (HPSA) and a Medically Underserved Area (MUA). Located in a box canyon of the Colorado Rocky Mountains in San Miguel County, Telluride is surrounded by mountains, receives over 167 inches of snowfall annually, and driving conditions are often treacherous. The nearest city to Telluride is Montrose  and is approximately 1.5 h drive along over 65 miles of mountainous roads. TMC includes 5–8 primary care clinicians and approximately 7–10 other clinic staff members. They provide primary care services to all ages, from birth to death and are about 100 miles from the nearest hospital. The clinic uses eClinicalWorks. Kepka) .via virtual meeting. A step-by-step documentation and guide for EHR implementation developed by the University team were provided offline to the providers and support staff at TMC.Dr. Kepka assembled a University team to guide the development and implementation of the components of the multi-level HPV vaccination intervention. Team members included two clinical informatics experts, a graduate student in nursing, a program manager and data analyst, and Dr. Kepka . The development of the parent/caregiver and young adult reminder intervention included three video calls between the University team and TMC providers and support staff at the end of 2020. Calls consisted of an environmental scan of existing HPV vaccination efforts, a discussion on possible approaches for improving HPV vaccination rates and delivery, and conversations around the facilitation of TMC provider and support staff vaccination efforts by the University team. The video calls also included a walkthrough of the EHR workflow and immunization preparation activities at TMC. The team determined prioritization of the target population, the development of a vaccination patient reminder campaign and messaging content, and campaign implementation within the EHR. The team also completed a walkthrough of running the campaign with TMC team members as completed Human papillomavirus vaccination training for the healthcare team included two 1-h early morning video calls that focused on training providers and support staff at TMC on evidence-based HPV vaccination systems, vaccine recommendations, and patient education materials relevant to their patient population. Healthcare teams were taught how to deliver a strong, brief, and consistent provider recommendation for the HPV vaccine to their patients . They we2.Before and after the two training activities, an online survey that included HPV and HPV vaccination knowledge questions and barriers to vaccination was completed online by providers and support staff at TMC using Research Electronic Data Capture (REDCap) tools hosted at the University of Utah , 30. Thevia patient preferred method  using patient outreach capabilities provided by the EHR system available at TMC .With the assistance of the University team, TMC providers and support staff developed a HPV vaccination reminder campaign for patients/caregivers with age-eligible children for the HPV vaccine (children ages 11–17) and young adults (ages 18–26) who are also age eligible for the HPV vaccine. The reminder message was branded as coming from TMC instead of sending reminders from their individual provider. HPV vaccination messages reminding patients or their parents/caregivers to schedule an appointment were sent from the Medical Center n = 44) in August 2020. Then, the team improved the design of the reminder campaign and expanded the target group to parents and caregivers of children ages 11–17 who are age-eligible for the HPV vaccine and to young adults ages 18–26 who are also age-eligible for the vaccine. The final round of the HPV vaccination reminders was sent to the larger group of participants in October 2020.Human papillomavirus vaccination reminder messages were designed using evidence-based recommendations highlighted by the American Cancer Society's HPV Vaccination Roundtable . First, via text or email to parents/caregivers at TMC during the reminder campaign is listed as:An example reminder message that was sent You'd do anything to protect your child.Now is the time to give [NAME] the gift of cancer prevention.The HPV vaccine protects boys and girls against up to 6 types of cancer.Our records indicate that [NAME] has turned 11 since the beginning of the COVID-19 pandemic. We recommend TDaP, HPV, and Meningitis routine vaccines at the 11-year-old visit. Don't delay, get your child scheduled today for a well-child visit by calling (###) ###-####.We are taking special precautions to ensure all well-child visits are safe during these challenging times.n = 402) and a shorter period of time following the intervention  was performed. Age at the time of the visit was collected as a continuous variable and included in analyses as whole years (with no rounding). All other collected variables were binarized into yes/no categories and Pearson's χ2 with probabilities were calculated for pre- and post-intervention comparisons. To determine the robustness of χ2 estimates, sensitivity analyses were performed to examine the effects of partially overlapping samples. A dataset of non-overlapping individuals between pre- and post-interventions was created, and p-values compared for pre-intervention, post-intervention, and post-intervention without overlaps. Age distributions were compared using 95% confidence intervals for the median with the null-hypothesis locational parameter equal to the pre-intervention median. Analyses were performed using SAS software .A cross-sectional retrospective chart review of HPV immunization of all vaccine-eligible patients (ages 11–26 years) visiting the clinic prior to intervention .A short online survey was conducted among the TMC parent/caregiver and young adult patient population about 3 weeks following the vaccination reminder intervention. Participants received an email invitation to the survey with a URL link that was used to take the survey. The survey asked patients, parents, and/or caregivers if they had received a reminder; what mode  of reminder they received; if they scheduled, or planned to schedule, an appointment; if they or their child had received the HPV vaccine; two open-ended qualitative response questions about scheduling the appointment and comments about the campaign; and demographic questions. Participants had the option to receive a $10 gift card as a thank you gift for their time upon completion of the survey. The survey was designed and administered using the online SurveyMonkey application  were calculated for the odds of scheduling an appointment by age, gender, receiving a reminder, mode of reminder, and receipt of the vaccine for either self or child. Analyses were performed using SAS software .Survey data were analyzed descriptively using frequencies, distributions by scheduling (or intent to schedule) an appointment, and Pearson's χThe study was considered to be exempted by the University of Utah Institutional Review Board as a primary care quality improvement study.2 estimates for all factors, except for “other visit”  and being up to date on the HPV vaccine at the beginning of the visit , indicating the use of the full post-intervention dataset to be appropriate (p = 0.05), and normality assumptions appeared to be reasonable in both the pre- and post-intervention populations . Higher proportions of younger (ages 10–14) patients were seen prior to the intervention (34.3 vs. 21.2%), and higher proportions of older (ages 19–28) patients were seen after the intervention (49.5 vs. 33.6%). Having any HPV vaccine records on file at the time of visit differed from pre- to post-intervention . Those without a vaccine record on file and were also not categorized as up-to-date, had an “unknown” HPV vaccination status . Neither the number of wellness visits, patients up-to- date on an HPV vaccination schedule, patients receiving an HPV vaccination during the visit, nor patients declining a vaccination at the time of visit varied between pre- and post-intervention. A lower proportion of patients following the intervention were due for the HPV vaccination initial dose and/or booster compared with those due prior to the intervention . Missed opportunities for HPV vaccination declined significantly from the pre-intervention to the post-intervention (p = 0.002).Less than 27.3% of individuals in the post-intervention sample overlapped with the pre-intervention sample. Sensitivity analyses demonstrated robustness of χropriate . The ageropriate . The medrvention   and equally distributed between those 18–45 years and those over 45 years old. A little more than one-third received or remembered receiving a vaccination reminder . Of those reporting the mode of reminder, most of them reported receiving an email message vs. a text message . Just under one-third of the parents/caregivers reported getting their child the HPV vaccination  and slightly over one-third reported getting the HPV vaccine for themselves .Parents/caregivers of HPV vaccine eligible patients and young adult patients at TMC who responded to the survey were primarily female (p = 0.0086), receipt of a reminder from TMC (p = 0.0002), and by vaccination status for their child (p = 0.0074) or themselves (p = 0.0116)  , although, due to cell sizes n < 5 for both covariates, estimates were considered as unreliable.Those who scheduled or intended to schedule an appointment differed by age ( 0.0116) . Parents11–0.77) . Howevern = 17). Survey responses were not paired from pre- to post-intervention and results represent changes at the clinic level. Most respondents agreed or strongly agreed to utilize education and resources in the future , and that the education and materials provided were helpful in treating every visit like a vaccination visit . The number of providers and support staff identifying a lack of educational materials as a barrier to vaccination decreased more than 17% points following the intervention, as did lack of time to discuss vaccination (The number of providers and support staff completing the pre-intervention and post-intervention surveys did not change (cination . The moscination  barrierscination .via free-text responses (data not shown), three general themes emerged as helpful: framing HPV vaccination as cancer prevention, education and materials provided, and education and discussion on HPV and vaccinating both male and female patients prior to the 15th birthday of the patient. An example of a statement of framing HPV vaccination as cancer prevention was “We have really taken on the approach as presenting this vaccine as part of cancer prevention, which parents are much more likely to receive well”. A characterization of the helpfulness of the education and discussion of HPV and vaccination was “We also have been able to provide valuable statistics on the prevalence of HPV once a patient becomes sexually active, and by stressing that we vaccinate early when children can mount a strong response long before they are exposed has helped parents understand why we vaccinate early against HPV”. Feasibility measures and usability among providers and support staff at TMC were favorable overall in both the pre- and post-intervention surveys (p > 0.2 for each).When examining what had been helpful in facilitating HPV vaccination by providers and support staff  surveys , with mo surveys  followinOverall, the multi-level and multi-mode HPV vaccination intervention performed well in a rural setting, even during the COVID-19 pandemic. Our findings indicate that patients who did not receive the HPV vaccine at the time of visit, although they were due for the vaccination , declined significantly following the HPV vaccination intervention period. Parental vaccination hesitancy or refusal, the most commonly reported vaccination barrier by providers and support staff, also declined by 41.2% points.While most barriers decreased in the frequency of selection by providers and support staff, there were a few vaccination barriers that were selected more frequently following the campaign. These included: lack of EHR use to track vaccination; personal fear of adverse and/or side effects; and being unsure of the need for vaccination. The lack of a system for provider reminders to discuss immunization decreased during a time of clinic overburden from COVID-19, while lack of an EHR to track vaccination and being unsure of the need for vaccination both increased following the education intervention. This likely characterizes sentiments expressed by providers and support staff in person during the educational campaign of a lack of a national immunization registry and a largely seasonal and/or transient patient-population that is often unsure of their own and/or child's HPV vaccination status. Future work for enhancing immunization uptake, particularly in rural settings, could include better integration of immunization registries at a national level with enhanced optimization of provider reminders which would improve provider and support staff confidence for the need of vaccination among transitory patient populations.These early results are promising but without a control group, or comparison to another similar clinic during the same time period, we cannot characterize the effect or the degree of effect the intervention had vs. other external factors. Future analyses could include difference-in-difference calculations to extend these findings from association into causal. Small cell sizes, due largely to unequal pre-post intervention population sizes, were present. At this time, we are unable to accurately characterize the impact of the COVID-19 pandemic on HPV vaccination in this population. The COVID-19 pandemic paused regular in-person appointments for many and may also have resulted in fewer people getting the HPV vaccine; counts may be underestimated as some people may get vaccinated once the pandemic is over; or counts may be accurate as some people may forego vaccination. This would be consistent with previous research indicating a preference for a single vaccine per visit, and many may prioritize the COVID-19 vaccine over the HPV vaccine.CIs) and unstable estimates of probability and odds for gender, mode of reminder, and a child's receipt of the HPV vaccine. It is unclear why so many people reported not receiving the reminder messages. This could be due to limited patient recall, or inaccurate records of patient contact information or contact preferences. Future studies could examine this issue in an attempt to increase acceptance, receipt, and/or memorability of messages.The results of this study suggest that, among rural patients aged 18 and older, those receiving a clinic HPV vaccination reminder, regardless of the mode, are 6.96 times more likely to schedule or intend to schedule a follow-up visit for vaccination for either their child or themselves . Small numbers (n < 5) in several of the cells in the contingency table of scheduling a follow-up visit contributed to imprecise (i.e., wide The population of providers and support staff, while sizable for a single-clinic rural intervention, suffered from small cell sizes issues and some unstable estimates were present. Additionally, the responses were unpaired and we were unable to measure individual changes in responses or knowledge improvement from prior to and following the facilitation and education campaign. None of the comparisons of pre- to post-intervention survey responses from providers and support staff were statistically significant at α = 0.05, however, many of the outcomes demonstrated improvement and have important clinical implications for HPV administration and championing vaccination for the future. Caution should be taken in generalizing our results to all rural areas and the unique regional, cultural, economic, and medical landscape should be considered before implementing any public health campaign among vulnerable populations. We are unsure how the pandemic will influence HPV vaccination over time.Results from this multi-level and multi-model intervention are promising and suggest that vaccination interventions consisting of provider and support staff education and training activities, parent/caregiver and young adult reminders, and evidence-based patient education materials can reduce missed opportunities for HPV vaccination in a rural setting. Future research should assess the implementation of larger scale multi-level and multi-model HPV vaccination interventions in rural primary care settings across the United States to reduce inequities in HPV vaccination rates and the incidence and mortality of HPV-related cancers among these vulnerable patient populations.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.The studies involving human participants were reviewed and approved by University of Utah Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.DK designed the study, led the data collection and analysis activities, and interpreted the results. EM and AW designed the HPV vaccine training and intervention activities. GF and BG are our clinical informatics experts and contributed to the design of the digital intervention. SA, HB, SM, and AP assisted with interpretation of findings. GC provided senior input on study design and interpretation of findings. DK, KC, and SA drafted the manuscript. All authors reviewed and provided edits to the manuscript.DK receives a small portion of her salary from a grant that is provided and supported by the American Cancer Society, who received funding from Merck, for the purpose of the “Mission: HPV Cancer Free Quality Improvement Initiative.” EM receives a small portion of her salary from a memorandum of understanding with the Huntsman Cancer Institute. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Testing for Human Papillomavirus (HPV) is currently being implemented as part of cervical cancer (CC) screening in several countries. However, infections with all but one of the HPV types classified as possibly carcinogenic cannot be detected by the assays used for CC screening today. The aim of our study was to demonstrate the use of a targeted next generation sequencing (NGS) HPV panel for CC screening—both general practitioner-collected samples and self-samples. We here show that the targeted HPV panel can detect HPV with a sensitivity and specificity similar to commercial HPV assays, one of which is used for CC screening today. However, the targeted HPV panel possess several advantages compared to the screening assays, as it enables specific detection of all relevant HPV types and can identify viral integration, variants in the HPV genome, and dominant HPV types in multi-infected cases.® 4800 HPV DNA test (Roche) and CLART® HPV4S assay (GENOMICA). Interestingly, of the samples that tested positive with the NGS panel, three (6.4%) of the GP-collected samples and four (9.1%) of the self-samples tested positive exclusively for HPV types only included in the NGS panel. Thus, targeted HPV sequencing has great potential to improve the HPV screening programs since, as shown here, it can identify additional HPV positive cases, cases with HPV integration, variants in the HPV genome, and which HPV type is dominant in multi-infected cases.At present, human papillomavirus (HPV) testing is replacing morphology-based cytology as the primary tool for cervical cancer screening in several countries. However, the HPV assays approved for screening lack detection for all but one of the possibly carcinogenic HPV types and do not genotype all included HPV types. This study demonstrates the use of a targeted HPV next generation sequencing (NGS) panel to detect and genotype all 25 carcinogenic, probably carcinogenic, and possibly carcinogenic HPV types as well as the low-risk types HPV6 and HPV11. The panel was validated using a cohort of 93 paired liquid-based cytology samples -collected cervical samples and cervico-vaginal self-samples (SS)). Overall, the targeted panel had a sensitivity  and specificity  similar to the commercial HPV assays, Cobas Persistent infection with high-risk human papillomavirus (HPV) is the main cause of cervical cancer development . SeveralThe causal link between persistent infection with high-risk HPV and cervical cancer has led to the development of multiple HPV detection assays . HPV det® 4800 HPV DNA test  and the CLART® HPV4S assay  [Traditionally, PCR-based amplification has been used for HPV detection with most tests detecting the viral capsid protein L1, including the Cobas, Spain) . Former , Spain) . Previou, Spain) .In this study, the small targeted NGS panel has been further optimized, and we aimed to examine the feasibility of the NGS assay for HPV detection and genotyping in liquid-based cytology (LBC) samples -collected cervical samples and cervico-vaginal self-samples). Furthermore, we aimed to examine if the designed NGS panel can assess the viral integration status as well as detect variants in the HPV genome.® 4800 HPV positive and 46 Cobas® 4800 HPV negative GP-collected cervical samples with paired cervico-vaginal self-samples collected at home were selected . Both GP-collected samples and self-samples from each woman in the study cohort had been analyzed with Cobas® 4800  and CLART® HPV4S  for use in previous publications, and the recruitment procedure as well as the sample collection, processing, storage, and analysis have been described elsewhere [The study cohort was extracted from a cohort of 213 women aged 30–59 years, diagnosed with atypical squamous cells of undetermined significance (ASC-US), and reflex HPV tested within the Danish cervical cancer screening program between June 2015 and December 2016 ,34. Fromlsewhere ,34.® 4800 can individually genotype HPV16 and HPV18 and can detect HPV31, 33, 35, 39, 45, 51, 52, 56, 58, 59, 66, and 68 as a pooled result, where the specific HPV type is not specified. CLART® HPV4S can individually genotype all HPV types included in Cobas® 4800 and can, in addition, genotype the low-risk HPV types HPV6 and HPV11.The Ion Torrent NGS panel was designed using the AmpliSeq Designer  and included the 25 IARC-classified carcinogenic , probably carcinogenic (HPV68), and possibly carcinogenic  HPV types. Furthermore, the panel included the two most common low-risk HPV types HPV6 and HPV11 [® 4800 and CLART HPV4S, one amplicon was designed for L1. Additionally, three amplicons were designed for E2, whereof two were in the E2 region called the hinge-region [For each of the 27 HPV types included in the NGS panel, eight amplicons were designed. Two amplicons were designed for each of the E6 and E7 oncogenes and, for comparison with the L1-detecting analyses such as Cobase-region . E2, ande-region ,37.To confirm the presence of DNA in each sample, an amplicon for each of five human reference genes  were included in the panel as well. How to select appropriate human reference genes is described in a previous publication . The pan® 4800 and CLART® HPV4S assays [The NGS panel was validated using DNA from paired GP-collected cervical samples and cervico-vaginal self-samples. The DNA had been purified for a previous study using the Cobas x480 Instrument  and analyzed using the CobasS assays ,34.® 4800 as per routine in the cervical cancer screening program, was used for the NGS assay. The samples were prepared for sequencing using the Ion AmpliSeq Library Kit 2.0  according to the manufacturer’s protocol, except that the reactions were performed in half volume. Input DNA was in the range of 10–60 ng for the self-samples. Due to limited material, the input DNA concentration was not measured for the GP-collected samples. However, the DNA concentrations were measured for six comparable GP-collected samples, not used for NGS, from the original cohort of 213 women. The DNA concentrations of these samples were very low , indicating very low DNA concentrations of the GP-collected samples included in this study as well. Therefore, 6 µL of undiluted DNA was used for library preparation of the GP-collected samples. The GP-collected samples were labelled with Ion Xpress Barcode Adapters , but in an attempt to reduce the amount of background, the self-samples were labelled with Ion Torrent Dual Barcodes . Before pooling the libraries for template preparation and sequencing, the libraries were quantified using the Ion Library TaqMan Quantitation Kit  and diluted to 50 pM. Template preparation was performed using the Ion 510 & Ion 520 & Ion 530 Kit—Chef  on the Ion Chef System . Sequencing was performed using the Ion GeneStudio S5 System  with the coverage analysis plugin set to a minimum amplicon length of 50 base pairs.Excess DNA, purified for Cobas® 4800 and CLART® HPV4S assays, since the HPV16 reads present in these samples would correspond to background reads. The cut-off was defined as the mean of the HPV16 E6 and E7 amplicons’ limit of detection (LoD). The LoD for each HPV16 E6 and E7 amplicon was calculated as follows: The sequencing data from each sample were aligned to a custom reference sequence composed of the human reference genome (hg19) and the genomes of 286 HPV types. The applied HPV reference genomes have been used in a former study , where a® 4800, and CLART® HPV4S assays were compared regarding sensitivity and specificity for HPV detection in general as well as for HPV16 detection. Each assay was compared to the two other assays to assess a “true/consensus” HPV result, which means that when at least two of the assays showed the same HPV result, this result was interpreted to be correct. For example, if the NGS assay did not detect any HPV types/HPV16, but the two other assays detected HPV/HPV16, the result of the NGS assay was interpreted as being false negative. The three assays were compared only for the HPV types detectable with all three HPV assays . Sensitivity, specificity, and corresponding 95% confidence intervals (CI) were calculated for each of the three HPV assays for both GP-collected samples and self-samples.The NGS, Cobas® HPV4S. The Cobas® 4800 assay was excluded from this comparison, as this assay only genotypes HPV16 and HPV18. The NGS and CLART® HPV4S assays were compared only for the HPV types detectable with both HPV assays . The degree of agreement between the NGS assay and the CLART® HPV4S assay was categorized as follows: Agreement was defined as detection of the exact same HPV types in the NGS assay and in CLART® HPV4S. Partial agreement was applied in samples infected with multiple HPV types and defined as detection of at least one of the same HPV types in the NGS assay and CLART® HPV4S. Disagreement was defined as detection of none of the same HPV types between the NGS assay and CLART® HPV4S. The percentage of agreement between NGS and CLART® HPV4S was calculated as the number of samples with agreement divided by the total number of samples.The specific HPV types identified by the NGS assay in the study cohort were compared to the HPV types identified by CLARTAccording to the EUs General Data Protection Regulation, the project was listed at the record of processing activities for research projects in the CDR . The study was approved by the local Ethical Committee of the Central Denmark Region .In total, 93 paired GP-collected samples and self-samples were genotyped using the NGS assay. All human reference genes were amplified, and the mean number of reads were 28,425  for the HPV negative GP-collected samples and 28,242  for the HPV negative self-samples, confirming the presence of suitable quality DNA material. Using the NGS assay, a total of 15 women had discordant results. Nine women (9.7%) were GP-collected HPV positive samples/self-sample HPV negative samples. Six women (6.5%) were self-sample HPV positive samples/GP-collected HPV negative samples. Several different HPV types were identified in the study cohort: HPV6, 16, 18, 30, 31, 33, 35, 39, 45, 51, 52, 53, 56, 58, 59, 66, 67, 68, 69, 70, 73, and 82 . Twenty-® 4800 or CLART® HPV4S assays . However, two of the three mentioned GP-collected samples had a Cobas® 4800 HPV positive result anyway. For these two samples, one of them tested HPV16 positive using Cobas® 4800, but since HPV16 was not detected neither by any other assay nor by Cobas® 4800 itself in the self-sample; this is most likely a false positive result. The other sample tested positive for an unspecified HPV type using Cobas® 4800. Since the self-sample from this patient tested HPV positive using the three different HPV assays, it is difficult to conclude whether this is a false positive sample or not. However, Cobas® 4800 has been observed to cross-react with HPV types not included in the assay [Several possibly carcinogenic HPV types only included in the NGS assay were identified  . One or he assay . This in® 4800, and CLART® HPV4S assays for HPV detection was 97.7% (95% CI: 87.7–99.9%), 93.0% (95% CI: 80.9–98.5%), and 88.4% (95% CI: 74.9–96.1%), respectively. The specificity of the same samples using the three assays was 98.0% (95% CI: 89.4–100.0%), 86.0% (95% CI: 73.3–94.2%), and 98.0% (95% CI: 89.4–100.0%), respectively. For the self-samples, the sensitivity was 92.1% (95% CI: 78.6–98.3%), 94.7% (95% CI: 82.3–99.4%), and 97.4% (95% CI: 86.2–99.9%), respectively, and the specificity of the same samples was 96.4% (95% CI: 87.5–99.6%), 90.9% (95% CI: 80.1–97.0%), and 100.0% (95% CI: 93.5–100.0%), respectively  were categorized as agreement, ten (10.8%) as partial agreement, and eight (8.6%) as disagreement. For the self-samples, 78 (83.9%) showed agreement, eight (8.6%) showed partial agreement, and seven (7.5%) showed disagreement , 100.0% (95% CI: 81.5–100.0%), and 83.3% (95% CI: 58.6–96.4%), respectively. The specificity of HPV16 detection using the same assays was 98.8% (95% CI: 95.8–99.9%), 95.8% (95% CI: 91.6–98.3%), 100.0% (95% CI: 97.8–100.0%), respectively  of the GP-collected samples and four (9.1%) of the self-samples tested positive exclusively for HPV types only included in the NGS assay .Our NGS assay is a semi-quantitative method that can detect and genotype HPV and includes all 25 carcinogenic, probably carcinogenic, and possibly carcinogenic HPV types as well as the low-risk HPV types HPV6 and HPV11. When comparing the sensitivity and specificity of HPV detection using the NGS, CobasSince multiple studies have shown carcinogenicity of the above-mentioned HPV types HPV53, 67, and 70 as well as the HPV types HPV26, 73, and 82 ,44,45 wh® 4800 and CLART® HPV4S in sensitivity and specificity for HPV detection while being semi-quantitative and being able to assess the viral integration status as well as detect viral variants.Since an increase of colposcopy referrals have been observed with the shift to HPV detection in cervical cancer screening ,48, an i® 4800 and CLART® HPV4S, as well as several other HPV assays, detect only a region of the L1 gene [® 4800 and CLART® HPV4S detect L1 poses the possibility that these two assays might be more likely to agree with each other compared to the NGS assay detecting several regions of the HPV genomes. This is a disadvantage of the comparison method used in this study where the consensus HPV results were based on HPV detection agreement between at least two of the assays. This could potentially result in a false negative consensus HPV result if both Cobas® 4800 and CLART® HPV4S do not detect any HPV due to L1 deletion. However, this does not seem to be the case in our cohort, which could be due to a low number of integrated cases. Since viral integration often occurs during progression of cervical dysplasia to cancer [Besides the above-mentioned features, our NGS assay detects several genomic regions of each included HPV type in the oncogenes E6 and E7 as well as E2 and L1. Both Cobas L1 gene . However L1 gene  which cr L1 gene ,55,56. To cancer ,57 and ao cancer ,42. Thero cancer . DespiteAs previously mentioned, because the NGS assay can detect and genotype all relevant oncogenic HPV types, it enables establishment of whether a patient has same genotype persistence or genotype switch in successive cervical samples . This co® HPV4S was thus caused by only three HPV16 false negative samples, and the study cohort should therefore be expanded for more precise sensitivity and specificity estimations, especially regarding HPV16 detection.A limitation of our study is the relatively small study cohort, which resulted in few HPV16 positive samples even after pooling the GP-collected samples and the self-samples, whereby each HPV16 positive sample had a huge impact on the calculated sensitivities for HPV16 detection. The relatively low sensitivity of 83.3% (95% CI: 58.6–96.4%) observed for CLARTBesides the aforementioned features, a strength of our NGS assay is its ability to work on both self-samples and GP-collected samples. Previous studies have shown an increase in cervical cancer screening attendance using self-sampling, and several countries have already implemented or are in the process of implementing self-sampling in their screening programs ,29,58,59® 4800 and CLART® HPV4S assays in sensitivity and specificity for HPV detection and HPV16 detection in both GP-collected cervical samples and cervico-vaginal self-samples. The NGS assay could, thus, potentially decrease the number of false negative screening results caused by infections with HPV types not currently screened for whereby more cervical dysplasia and cervical cancer cases could be avoided. Moreover, the NGS assay gives valuable additional information about the dominating HPV type in cases with multiple HPV infections, integration status, and variants in the HPV genome which are features that could be valuable in future stratification of HPV positive women. Thus, we believe the additional information provided by NGS-based HPV analysis outweighs the longer analysis time (2–3 days) compared to PCR-based HPV analysis (hours).In conclusion, the NGS assay enables detection and genotyping of all carcinogenic, probably carcinogenic, and possibly carcinogenic HPV types while performing as well as the Cobas


Presence of a firearm is associated with increased risk of violence and suicide. United States military veterans are at disproportionate risk of suicide. Routine healthcare provider screening of firearm access may prompt counseling on safe storage and handling of firearms. The objective of this study was to determine the frequency with which Veterans Health Administration (VHA) healthcare providers document firearm access in electronic health record (EHR) clinical notes, and whether this varied by patient characteristics.The study sample is a post-9–11 cohort of veterans in their first year of VHA care, with at least one outpatient care visit between 2012–2017 . Demographic data, veteran military service characteristics, and clinical comorbidities were obtained from VHA EHR. We extracted clinical notes for outpatient visits to primary, urgent, or emergency clinics . Natural language processing and machine learning (ML) approaches were used to identify documentation of firearm access. A taxonomy of firearm terms was identified and manually annotated with text anchored by these terms, and then trained the ML algorithm. The random-forest algorithm achieved 81.9% accuracy in identifying documentation of firearm access.The proportion of patients with EHR-documented access to one or more firearms during their first year of care in the VHA was relatively low and varied by patient characteristics. Men had significantly higher documentation of firearms than women  and veterans >50 years old had the lowest (6.5%). Among veterans with any firearm term present, only 24.4% were classified as positive for access to a firearm (24.7% of men and 20.9% of women).Natural language processing can identify documentation of access to firearms in clinical notes with acceptable accuracy, but there is a need for investigation into facilitators and barriers for providers and veterans to improve a systemwide process of firearm access screening. Screening, regardless of race/ethnicity, gender, and age, provides additional opportunities to protect veterans from self-harm and violence. In 2020 42% of United States (US) households reported owning a firearm.8While access to firearms is associated with increased risk for injury and death, safe firearm storage is associated with decreased risk.14Screening veterans for firearms ownership and safe storage is needed to prevent unnecessary injuries and deaths. Healthcare providers are in a position to screen and counsel patients on safe firearm storage.26To understand how current practice may be adapted, more information on the frequency with which healthcare providers document firearm screening is needed. In this study we present results of Veterans Health Administration (VHA) healthcare providers’ documentation of firearm access screening in electronic health record (EHR) notes among VHA patients in outpatient primary care, urgent care, and emergency department (ED) settings.The study is a cross-sectional examination of the frequency of documentation of screening for veterans’ access to firearms across several healthcare settings using natural language processing (NLP), which refers to automatic computational processing of human language.What do we already know about this issue?United States Veterans are more likely to own a firearm and to be at risk for firearm injuries and death than civilian populations.What was the research question?Our study aimed to determine how frequently VA healthcare providers document firearm access screening.What was the major finding of the study?Documentation of firearm access for Veterans by healthcare providers was low but higher in men than women.How does this improve population health?Identifying barriers and facilitators to help healthcare providers increase screening for firearms and counsel safe storage could support prevention efforts.International Classification of Diseases, 9th and 10th revisions, Clinical Modification [ICD-9-CM and ICD-10-CM]) and Current Procedural Terminology  or ≥1 inpatient code for the condition. This methodology has been used for the identification of psychiatric disorders in administrative dataFor the information extraction process, we developed a coding manual for chart review and a taxonomy for firearms for annotation. A taxonomy was created by by searching existing vocabularies  and the literature for published ontologies used for guns, gunlock, and firearms. A Cochrane review on gunshot wounds contained terms such as trauma* or injur* or penetrat* or wound*or perforat* or stab* or gunshot or shot, and the Medical Subject Headings (MeSH) database included the following: “Wounds, Gunshot”[Mesh]) OR “Wounds, Penetrating”[Mesh:NoExp]) OR“Multiple Trauma”[Mesh])) OR “polytrauma.” This list of terms was supplemented with over 120 candidate terms and phrases contained in a national databaseWe identified 2,584,607 notes with one or more of the phrases, and annotated 1856 text snippets randomly selected from notes that contained any of the search terms. Each snippet contains a 35-word span before and after a firearm-relevant phrase. The annotation classifications for firearm access were the following: positive (ability to determine that the veteran had current access to at least one firearm); negative (language that the veteran did not have current access to any firearms); and ambiguous (there was insufficient evidence for either a positive or negative classification from the note – an example might be that the veteran owned a firearm but it was somewhere else). Each snippet was annotated by two of the authors and disagreement adjudicated by their consensus. An inter-annotator agreement was calculated. The annotated snippets served as the reference standard in training and testing.patient, owns, a, and shotgun. In the same phrase, patient_owns, owns_a, a_shotgun are unique bigrams. Alpha or numeric tokens (discrete words and numbers) were counted in the unigrams and bigrams. The features included unique unigrams with a frequency greater than 34, and unique bigrams in the annotation spans with a frequency greater than four. These threshholds are empirically chosen to filter out the less prevalent n-grams and reduce overfitting. The training features for the model (for each document) consisted of binary indications of the presence of each of the identified unigrams and bigrams, along with the offset location of the keyphrase in the snippet.We used n-grams as features. In clinical text, unigrams are single words, and bigrams are two words that occur in a sequence. For example, in the phrase “patient owns a shotgun” the unique unigrams are We used the annotated snippets to train a random forest model with 200 estimators or trees. The random forest model maximum depth was set to 15, with maximum features automatically determined by the model and the gini split criterion. Hyperparameters were determined through gridsearch and other testing. We split the 1856 text snippets into 85% for training and 15% for testing. The model performance was measured by accuracy.For validation, we annotated an additional 238 clinical notes on the note (instead of the snippet) level, with 175 negtive and 63 positive for firearm access. The random forest model was applied to these notes, based on the snippet identified in each document. We conducted all statistical analyses using SAS software version 9.4 . Baseline characteristics of veterans include frequency (percentages) and means (± standard deviations) or median (interquartile range), and differences by age, race, ethnicity and gender were examined using chi-squared test or Student’s t test, as appropriate. We used a multivariable logistic regression model to assess firearm mention and adjust for potential confounding based on the literature. Among those with firearm mention, a logistic model was run to assess firearm access. We examined model fit using quasi-information criterion and residual plots. Hypotheses were tested at a two-sided significance level of α = 0.05.The Cohen kappa score measuring inter-annotator agreement among the review team members identifying screening documentation incidents was 80%. On the testing dataset (15%), the accuracy was 81.0%. On the final validation dataset, the random forest model achieved 81.9% accuracy, 90.9% specificity, 57.1% sensitivity, and positive predictive value of 69.2% in classifying the 238 test notes. We included data during the first year in VHA care for 762,953 veterans in the analytic sample. Results demonstrate documentation of firearm access in clinical notes for less than 10% of contemporary veterans within the first year of enrollment in VHA healthcare, and that nearly one quarter of those with documentation were identified as having access to a firearm. There was a significantly lower rate of documented access for women veterans, despite data that show high rates of both men and women veterans who live in homes with firearms, and increasing rates of fiream-related suicides among women veterans.Discussions must be acceptable to providers and to patients for it to be effective. Roszko and colleagues’ review of 53 studies of non-veteran clinician firearm attitudes and practices found that positive attitudes toward firearm discussions were higher than actual documented discussions, with low firearm discussions across all disciplines.While providers may have positive attitudes toward firearm screening and counseling, recent studies show mixed support by gun owners and veterans for healthcare provider initiation of gun safety conversations.41Perhaps related to the reasons specified above, the evidence for the effectiveness of this firearm safety conversation in the clinical setting is mixed.48Specific to the VHA, an appropriate clinical response to the public health problem of firearm suicide in the veteran population is needed. Further research within the VHA is needed to determine the healthcare setting(s) and provider types most appropriate for firearm screening and counseling interventions. This step will require a participatory approach among health services and informatics researchers to improve the feasibility, acceptability, relevance, and sustainability of interventions.Given retrospective studies may introduce sampling biasNatural language processing methods are able to determine the prevalence of documented firearm screening and safety counseling across a large population of US military veterans. We identified low prevalence of firearm access screening documentation and believe that further investigation into facilitators and barriers is necessary. This work should inform the process for development of systemwide practices to reduce firearm suicide and injury among US veterans, a large group at elevated risk.


The World Health Organization’s Western Pacific Region is responsible for one-fourth of the global cervical cancer burden, and nearly 90% of that burden is concentrated in middle-income countries (MICs). Applying a conceptual model of implementation of population-based interventions, we synthesized the current implementation status of human papillomavirus (HPV) vaccination and national plans that form the basis of its implementation in 17 MICs. We gathered information from a range of governmental documents, published studies, and global databases. For all available national cancer-related plans and immunization plans, we examined the description of HPV vaccination. We found that, as of July 2021, only four countries (24%) had a mature HPV vaccination program with a high first-dose coverage; three (18%) had introduced HPV vaccination, but needed further efforts to scale it up, seven (41%) had not been able to introduce it after conducting demonstration projects, and three (18%) did not have any experience in HPV vaccination. In the national plans, most of the countries recognized the importance of HPV vaccination, but only 10 (59%) provided an implementation strategy on how it would be introduced or scaled up. Countries with a mature program were more likely to have their implementation strategy detailed in their national cancer control plan. Successful implementation of HPV vaccination requires overcoming known challenges and having a clear national plan. Positioning HPV vaccination clearly in the overall national cancer control plan may be key to accelerating its nationwide implementation. The World Health Organization (WHO)’s Western Pacific Region (WPR) comprises 27 countries and 10 areas, home to 953 million women . Among wCervical cancer is a devastating disease caused by persistent infection with oncogenic human papillomavirus (HPV) . HoweverIn this review, we synthesized the implementation status of HPV vaccination and national plans that form the basis of its implementation in 17 MICs of the WPR. These countries are highly diverse in terms of population size and health system structure, but all are in the midst of socioeconomic development and rapid aging  1,3,11,,11,12,13To synthesize and present the implementation status of HPV vaccination, we applied a conceptual model of implementation of population-based interventions ,16,17. TUsing this model, we first reviewed cervical cancer burden and screening practices as essential background. We then examined, for each country, whether they had conducted any demonstration project for gaining experience in delivering the HPV vaccine prior to nationwide introduction ,19. WhilWe gathered information from a range of documents and databases between May and July 2021.The estimated cervical cancer incidence and mortality and their estimation method for 2020 were extracted from the GLOBOCAN Cancer Today database . The proTo understand the practice of cervical cancer screening, information on the presence and coverage of the national cervical cancer screening program was extracted from the WHO Global Health Observatory . This reInformation on HPV vaccination demonstration projects and national programs was extracted from governmental documents and published studies. For the latter, PubMed was searched using the keywords “papilloma”, “HPV”, “vaccine”, and “immunization”, and the name of the country. When a discrepancy in information was observed between different sources, the information in governmental documents was prioritized. Documented challenges and barriers to HPV vaccination implementation were also extracted.In countries with a national program, program coverage of first and final doses for the years after introduction to 2020 were obtained from the WHO/UNICEF HPV Immunization Coverage Estimates . The metConsidering the nature of HPV vaccination, two types of multi-year national plans issued by the government  were reviewed: cancer-related plan and immunization plan. The former was defined as any document covering strategies to reduce cervical cancer burden, and included a national noncommunicable disease (NCD) plan, national cancer control plan (NCCP), and national cervical cancer control plan, depending on their availability of each country . An immuFor all available plans of each country, we examined whether HPV vaccination was (i) mentioned with an implementation strategy, i.e., time-bound activities and targets for introduction, scale-up, or maintenance; (ii) mentioned without an implementation strategy ; or (iii) not mentioned. Two reviewers (R.H. and S.O.) independently reviewed and rated the plans fitting the abovementioned criteria 2, 1, and 0, respectively. Discrepancies that occurred were discussed within a wider research group to arrive at a consensus.We compiled all data in Microsoft Excel sheets by country. The 17 MICs were initially classified as Asian and Pacific Island and organized by population size. To present the implementation status of HPV vaccination, we then categorized them into the following four groups: scale-up, i.e., those running a national HPV vaccination program for over five years ; delivery, i.e., those who had introduced HPV vaccination  and were in the process of scaling up; preparation, i.e., those who had completed at least one demonstration project, but not yet introduced HPV vaccination; and others.Future projections show that, if the current situation continues, both incidence and mortality will rise in all the eight countries for which complete data was available. Their total number of deaths is expected to increase by over 30% from 67,992 in 2020 to 90,407 by 2040. Among the 17 MICs, 15 (88%) had a national screening program running in 2019 . The maiFour countries  were placed in the scale-up group, having a national HPV vaccination program for over five years. In these countries, nonavalent or bivalent vaccine was provided to girls through an existing school health program on a semi-annual  basis ,29,47,48Three countries  were placed in the delivery group. In the Philippines, an HPV vaccination program was introduced in 20 priority provinces in 2015 and later expanded to 48 provinces, covering nearly 60% of the total, but not scaled up nationwide, with program coverage remaining relatively low ,34. QuadAmong the 10 countries yet to introduce HPV vaccination, seven  were placed in the preparation group, having completed one or more demonstration projects. The projects’ outcomes were generally reported as good with high vaccination coverage of the target population, barring Mongolia where community resistance was observed during the vaccination campaign  19,32,3,44,45,46Challenges and barriers to HPV vaccination introduction and scale-up were obtained from nine countries  29,37,4,46,49,50Of the 17 MICs, cancer-related plans were obtained for 16 (94%) and immunization plans for nine 53%)  [28,29,3%  28,29,76,77,78When we looked at which type of national plan contained an implementation strategy for HPV vaccination in those 10 countries, it differed greatly by country. An interesting relationship was observed between the implementation status and the type of national plan. Among the four countries in the scale-up group, three  had described their implementation strategy for HPV vaccination in the NCCP as one of the primary prevention actions to reduce cancer risk; the fourth, Fiji, had a national immunization plan providing actions for achieving HPV vaccine delivery and coverage target, but HPV vaccination was mentioned only in passing in the NCD plan.In the delivery group, two countries’ (Lao PDR and Solomon Islands) implementation strategies for HPV vaccination were detailed in the national immunization plan as one of the “new vaccines” along with costing and financing projections, but not in the cancer-related plans. It was therefore unclear to what extent the NCD or cancer program was involved in the implementation process.Three countries in the preparation group  had described their implementation strategy for HPV vaccination in the national cervical cancer control plan. Their strategies were similar and entailed building political advocacy, raising public awareness, training providers, exploring financing mechanisms, and improving monitoring systems; however, notwithstanding these detailed strategies, there was little to no information in the immunization plan as well as in the broader NCD plan or NCCP. Furthermore, Vietnam’s immunization plan stated that the HPV vaccine would not be introduced within its policy cycle, given the limited government funding and prioritization of other vaccines (measles/rubella and rotavirus) . In PapuThree Pacific Island countries  did not mention HPV vaccination in any of their national plans. As smaller countries might not need disaggregated plans, we further reviewed their overall health strategic plans, but did not find any mention of HPV vaccination.In this review, we synthesized the implementation status of HPV vaccination and national plans that form the basis of its implementation in 17 MICs in the WPR. As of July 2021, only four countries (24%) had a mature HPV vaccination program with a high first-dose coverage; three (18%) had introduced HPV vaccination, but needed further efforts to scale it up, seven (41%) had not been able to introduce it after conducting demonstration projects, and three (18%) did not have any experience in HPV vaccination. In their national plans, most of the countries recognized the importance of HPV vaccination, but only 10 (59%) provided an implementation strategy on how it would be introduced or scaled up. Countries with a mature program were more likely to have an implementation strategy detailed in their NCCP.In WPR, as reported globally and from other regions, less-resourced MICs bear the majority of cervical cancer burden . RecogniTo synthesize and present the current implementation status of HPV vaccination, we applied a conceptual model of implementation of population-based interventions. This was because we found that a number of studies on these countries described experiences of demonstration projects and national program introduction, but the information was often scattered in isolation, making it difficult to understand where each country currently stands in the implementation process—this includes whether demonstration projects have been conducted, when vaccination was introduced, and how the coverage has changed over time. Categorizing the countries into four groups showed that only four (24%) had a mature HPV vaccination program , three (18%) had introduced HPV vaccination, but needed further efforts to scale it up (delivery group), seven (41%) had not been able to introduce it after conducting demonstration projects (preparation group), and three (18%) did not have any experience in HPV vaccination (as of July 2021). Not only introduction but maintaining high vaccination coverage is important to ensure target girls are well protected and unvaccinated adolescents and young adults also benefit from herd immunity.The documented challenges to implementation were five-fold: (1) difficulty in vaccine delivery, (2) low community awareness and resistance, (3) limited coordination between stakeholders, (4) concern about cost and sustainable financing, and (5) insufficient vaccine supply. These challenges are consistent with those reported in previous studies, and none have been identified as unique to the WPR ,82,83,84The first three challenges  can be explained by two unique features of the HPV vaccine that make it different from traditional childhood vaccines ,84: firsIssues regarding cost and financing are other common challenges that MICs face . The pri®, Xiamen Innovax Biotech Co., Ltd., Xiamen, China) has obtained the WHO prequalification [Insufficient vaccine supply remains a global concern, including in the 17 MICs ,92. Recefication .To overcome these challenges, particularly with regards to multisectoral collaboration and financing, high-level commitment and clear policy are vital ,95,96. TBecause the primary objective of HPV vaccination is to reduce cervical cancer cases, it is essential that countries monitor its incidence trend alongside vaccination coverage. To monitor incidence, a functional national or regional population-based cancer registry (PBCR) is needed . HoweverThe limitations of this review should be noted. First, we did not examine whether national plans were actually implemented as planned, but rather aimed to provide a snapshot of the current situation and the national plans in place. We did our utmost to gather the most updated national plans of the 17 MICs, but there were a few outdated documents and there may be some that we could not obtain. There were also national plans that were mentioned to exist in a document, but were invalid or did not actually exist when we inquired with the MoH of those countries. Second, there may be a case where HPV vaccination has been implemented at the subnational level without it being mentioned in a national plan. Finally, we only studied MICs in the WPR. It would therefore be interesting to carry out a similar analysis of MICs in other countries and regions.Although HPV vaccination is vital for the elimination of cervical cancer, only four of the 17 MICs in the WPR have a mature HPV vaccination program; the other 13 need accelerated efforts to proceed with implementation. Successful introduction and scale-up of HPV vaccination is only possible by overcoming known challenges and having a clear national plan with a viable implementation strategy. Positioning HPV vaccination clearly in the overall NCCP, as an essential part of cancer prevention, may be key to accelerating its nationwide implementation.


Allele G*01:01:02 showed to be protective against any- and high-risk (HR) oral HPV . Men having allele G*01:01:01 showed a reduced risk for incident  and persistent  oral infections. Allele G*01:01:03 was associated with increased risk for urethral HR-HPV infections . Among self-reported demographic data, genotype G*01:01:01/01:01:03 was associated with an increased risk for oral warts  and allele G*01:03:01 increased the risk of pollen and/or animal allergy . To conclude, HLA-G polymorphism in men largely impacts the outcome of an oral HPV infection and seems to associate with self-reported allergies.The host genetic factors that influence the natural history of human papillomavirus (HPV) infection in men are not well known. Our aim was to evaluate the role of human leukocyte antigen (HLA)-G polymorphism in oral and genital HPV infection in men. Altogether, 130 men from the Finnish Family HPV Study, with a 6-year follow-up, were included in the analyses. HLA-G alleles were tested by direct sequencing. Oral, urethral, and semen samples were collected and analyzed for 24 different HPV genotypes. Unconditional logistic regression was used to determine associations between HLA-G alleles and genotypes with HPV infection and its outcomes. Overall, eight different HLA-G alleles were identified with 15 different HLA-G genotype combinations. The most common HLA-G allele among the men was G*01:01:01 (86.2%, The online version contains supplementary material available at 10.1007/s10096-021-04362-8. Human papillomavirus (HPV) infections are typically asymptomatic, transient, and cleared by the host immune system. However, persistent infection with certain high-risk (HR) HPV types is associated with HPV-related cancers, most importantly cervical cancer which is the fourth most common cancer in women worldwide . In men,Genital HPV prevalence in men ranges widely depending on the population and type of sampling sites, and at its highest is shown to be 72.9% but more usually ≥ 20% . CompareThe human leukocyte antigen (HLA)-G has been proposed as a possible immunological co-factor in the pathogenesis of HPV as in other viral infections. HLA-G, as a member of the non-classical human leukocyte antigen (HLA) class Ib, acts as a negative regulator of immune responses through interactions between natural killer (NK), T, and antigen-presenting cells . CompareThe Finnish Family HPV (FFHPV) Study is a longitudinal cohort followed up at Turku University Hospital, University of Turku, Finland. The study design was to evaluate the dynamics of HPV infection among family members: mothers, fathers, and their offspring as described previously , 13. A tIn the present study, the previously published data on HPV infection in men was utilized –18. At eHPV DNA was extracted from samples by using the high salt method as described previously . HPV ampDNA for HLA-G typing was extracted from the frozen whole blood samples by using the MagNAPure 96 System (Roche). Determination of HLA-G alleles was done by direct DNA sequencing exploring exons 2–4 (1718 bp) of HLA-G gene regions as described previously .The demographic data of the men were collected by a structured questionnaire at baseline. The mean age of the men was 28.9 years. The baseline demographic data have been reported in detail previously . The queAll statistical analyses were run by using STATA SE15.1 . The present study included overall 130 men, who had both completed HLA-G allele testing and had a ≥ 1 oral HPV genotyping result available. Baseline samples for two men were missing, leaving 128 men with HPV result of their semen, urethra, and oral baseline samples for analyses. Only those HLA-G alleles and genotypes that were ≥ 3% prevalent among men were included in the analyses. HPV species were classified as either LR or HR groups . These gFour oral HPV outcome variables were defined: always HPV negative, incident HPV, clearance, and persistence. Always HPV negative denoted men who tested negative for HPV at baseline and every follow-up visit. Incident HPV included men who tested negative at enrolment and then positive at some point during follow-up. Clearance included men tested HPV positive at baseline or at some point in the follow-up, then turned HPV negative before the end of the follow-up and remained negative until the end of follow-up. Persistence was defined whenever a man tested positive for the same HPV genotype with two or more consecutive visits. This definition also included the men who tested consecutively positive with the same HPV genotypes as part of a multiple infection.p < 0.05 was regarded as statistically significant.Unconditional logistic regression was used to determine associations (OR) between HLA-G: alleles as genotypes, and oral HPV infection outcomes. Those men in whom the allele was absent were used as reference. Associations of possible risk factors with HLA-G alleles and genotypes were analyzed by using unconditional logistic regression. All the statistical tests were performed two-sided and the value n = 112) followed by G*01:01:02 . The most common genotype found was G*01:01:01/01:01:01  followed by G*01:01:01/01:01:02 , respectively.Overall, eight different HLA-G alleles with 15 different HLA-G genotype combinations were identified. Altogether, seven different HLA-G alleles and genotypes  were included in the analyses. The allele and genotype distributions are shown in Fig. n = 86), urethral (n = 122) and oral (n = 128) samples is shown in Table At baseline, the frequency (%) of different HPV genotypes in semen  with oral HPV outcomes  are shown in Table The association of the self-reported demographic data of the men and their association with HLA-G alleles and genotypes are given in Table ′UTR polymorphisms in prostate cancer susceptibility [To our knowledge, this is the first study reporting on the association of HLA-G polymorphism and HPV infection in men. To date, most studies have focused on HLA-G polymorphism and HPV-related diseases in women , 23–29. tibility .The male cohort of the present study represents a Finnish, Caucasian, male population which has a quite restricted and homogenous gene pool due to historical isolation. The relatively low number of different HLA-G alleles observed is in line with our recent observations of the women in the same cohort  and is eOur main finding was that the presence of wild-type *G01:01:01 allele in men was shown to be protective for incident and persistent oral HPV infections. Moreover, the *G01:01:02 allele was associated with a lower risk for any and HR oral HPV infections. Interestingly, part of our results is contradictory to that reported in studies on HLA-G polymorphism and HPV infection among women, which are focused on genital infections , 36, 37.We found that G*01:01:01 allele decreased the risk for incident and persistent oral HPV infection by any viral type. In contrast, Metcalfe et al. reported that G*01:01:01 allele was associated with an increased risk of genital LR-HPV infection . FergusoIn addition, we found that allele G*01:03:01 in men was significantly associated with an increased risk for allergies to pollen and/or animals. Several studies have shown that HLA-G is associated with allergic diseases as reviewed by Murcada et al. . In geneThe main limitation of our study was the rather low number of men with uncommon HLA-G alleles. This might have interfered with the detection rate of significant associations between HLA-G and HPV infections in men. The demographic data was also self-reported, which can create some bias especially with the self-reported wart diagnosis. We had no similar follow-up data for genital samples as we had for oral samples. Thus, the evaluation of the role of HLA-G in HPV infection outcomes had to be restricted to only the oral site. The strength of the present study is a representative cohort with the same marital status and ethnic background which also reflected the genotype distribution that we observed. Although the follow-up time was relatively long, 76.2% (99/130) of the men completed the 3-year follow-up and 42.3% (55/130) the full 6-year follow up with oral samples available from each follow-up visit. To conclude, this is the first study to disclose that HLA-G polymorphism seems to play a role in oral HPV infections in males. Thus, the role of HLA-G gene in oral and genital HPV infections needs further investigations, particularly among men.Supplementary file1 (PDF 83 KB)Below is the link to the electronic supplementary material.