The National Diabetes Education Program, cosponsored by the National Institutes of Health and the Centers for Disease Control and Prevention, employs mass media communications, public-private partnerships, and dissemination of information and education tools to address the diabetes epidemic in the United States. The program's goal is to help reduce the morbidity and mortality from diabetes and its complications by improving the treatment and outcomes for people with diabetes, promoting early diagnosis, and preventing onset of diabetes. Evaluation is an integral component of the National Diabetes Education Program's planning and implementation process. The program's evaluation is based on the Centers for Disease Control and Prevention's Framework for Program Evaluation in Public Health, which has guided program planners and evaluators in developing measurable short-term, midterm, and long-term outcomes. We describe how the National Diabetes Education Program has applied the evaluation framework, demonstrating how multifaceted health communications programs can design program evaluations to answer key questions about program processes and outcomes. Diabetes was the seventh leading cause of death in the United States in 2006; approximately 233,619 deaths were attributed to the disease  in 1997 to improve diabetes management and to help reduce the morbidity and mortality from diabetes and its complications. The leading federal government program that promotes diabetes prevention and control, NDEP is cosponsored by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health (NIH) and the Division of Diabetes Translation of the Centers for Disease Control and Prevention (CDC). The aim of NDEP is to improve the treatment of diabetes and its complications, to promote early diagnosis, and to prevent the onset of diabetes . To reacIncrease awareness of the seriousness of diabetes, its risk factors, and strategies for preventing diabetes and its complications among groups at risk.Improve understanding about diabetes and its control and promote better self-management behaviors among people with diabetes and their social supporters.Improve health care providers' understanding of diabetes and its control and promote an integrated approach to care.Promote health care policies that improve the quality of and access to diabetes care.Reduce health disparities in racial and ethnic populations disproportionately affected by diabetes.Four central principles guide NDEP's planning, implementation, and evaluation activities. These principles are based on effective approaches used by the National High Blood Pressure Education Program, the National Cholesterol Education Program, and other education programs during the past 30 years . The plan formulated a statement of need for the program and presented key objectives, target audiences, messages, strategies, and tactics it would employ.The NDEP Evaluation Work Group provided guidance on developing outcome measures for evaluating the program by prioritizing a 3-tiered list of outcome measures for each target audience. From the list of prioritized outcome measures, the work group developed a conceptual program evaluation framework that included an overview of program resources, program activities, process goals, and intermediate and long-term goals .The evaluation work group and staff used the conceptual framework to develop the more explicit logic model. For example, in looking at the program's glucose control component, the conceptual framework enabled NDEP to develop a logic model that included key strategies to increase knowledge, influence attitudes and beliefs, and change behaviors through its mass media messages and educational materials and activities. As shown in As a national education program, NDEP focuses its evaluation efforts on monitoring and assessing what the program's mass media campaigns, educational materials, and other promotional activities can plausibly influence; for example, increasing knowledge of the importance of controlling risk factors for cardiovascular disease in people with diabetes, improving a person's feeling of self-efficacy to take charge of diabetes self-management, or increasing the practice of self-monitoring of blood glucose levels. At the same time, NDEP monitors long-term outcomes, such as prevention of diabetes-related complications, by tracking incidence and prevalence of blindness, kidney failure, amputations, and cardiovascular disease as reported by the National Health Care Surveys conducted by CDC's National Center for Health Statistics (NCHS). The program monitors intermediate outcomes, such as the health status of people with diabetes who have major risk factors that contribute to diabetes complications, using data from the National Health and Nutrition Examination Survey (NHANES) and the National Health Interview Survey (NHIS) conducted by NCHS and CDC's Behavioral Risk Factor Surveillance System (BRFSS). NDEP monitors these trends and uses the results to determine what messages and educational activities the program needs to emphasize.The outcomes specified in the NDEP evaluation lend themselves to survey data collection. Because of budget and regulatory constraints on survey research by federal government programs, NDEP devised a multifaceted approach for measuring program outcomes using survey data. The program uses a combination of existing data available from ongoing federal government and partner organization surveys related to diabetes and supplements these data by conducting its own survey research.CDC's NCHS coordinates several surveys that incorporate diabetes-related questions, including NHANES, NHIS, and BRFSS. NDEP has used data from these surveys to track many of its outcome measures, including epidemiologic trends in diabetes and cardiovascular risk factor control as well as public knowledge, attitudes, beliefs, and health behaviors regarding diabetes.To fill gaps in national data collection on key NDEP outcome measures, the program successfully negotiated adoption of several questions into the NHANES and NHIS diabetes question sets. Responses to these questions will provide NDEP with data about people with diabetes and people at high risk for diabetes.Since the program's inception, NDEP has worked closely with the American Diabetes Association (ADA), the leading health professional and voluntary organization concerned about diabetes, to coordinate evaluation activities. ADA has a strong market research program that includes annual surveys of people with diabetes. The association also commissions periodic physician surveys and national omnibus surveys to track knowledge, attitudes, and practices related to diabetes among the general public and people with diabetes. ADA routinely consults with NDEP about questions to include in these surveys and shares nonproprietary data with the program. This collaboration has provided NDEP with baseline and benchmark data for tracking changes in knowledge, attitudes, and practices for several short-term outcome measures.Other partners, such as the American Academy of Nurse Practitioners (AANP), conduct routine surveys of their members about knowledge, attitudes, and practices related to various health conditions. The AANP shared their findings related to diabetes management with NDEP and agreed to assist NDEP with conducting survey research among its members.NDEP applied for supplementary funding from the NIH Evaluation Branch, explaining the need for the research and how it would be used in program evaluation. Concurrently, NDEP sought permission from the federal Office of Management and Budget (OMB) to conduct original survey research. Federal government regulations restrict agencies from fielding survey research among 10 or more people without OMB approval. These restrictions are to minimize the "paperwork burden" on the public.After NDEP received funding from NIH and approval from OMB, a national telephone survey was conducted in 2006. NDEP targeted adults aged ≥45 years, the population at highest risk for diabetes and prediabetes. Because diabetes is more prevalent among minority populations, African American and Hispanic/Latino households were oversampled. The survey results will fill most of the remaining gaps on several key evaluation questions about the public's knowledge of diabetes and prediabetes and will provide direction for shaping educational messages and materials.NDEP supplements the extensive outcome data monitoring detailed above with an ongoing system of process evaluation, including the following:Tracking television, radio, and print public service advertising placements .Tracking results of press release dissemination though a clipping service .Tracking the number of publications ordered from the National Diabetes Information Clearinghouse, NDEP's fulfillment organization, and the number of public inquiries to CDC's Division of Diabetes Translation.Gathering monthly Web statistics on the number of unique visits to the NDEP Web site and the number of downloads of the top 100 most popular publications.Another key element of NDEP's process evaluation involves monitoring and tracking NDEP-related activities conducted by members of the partnership network through a semiannual Partner Activities Survey. This online survey asks partners to report on the types of promotion and dissemination activities they have conducted in the past 6 months for each NDEP priority campaign message. This survey also gathers feedback on partners' satisfaction and challenges with NDEP's campaigns, educational materials, and operations.NDEP reports the results of its process and outcome evaluations to program stakeholders in many ways. One is through NDEP's annual report on evaluation activities and the latest data on outcome measures, which is presented to the program's steering committee. A summary of this report is incorporated into the steering committee meeting minutes, which are disseminated to the entire partnership network. Similarly, a report on NDEP evaluation is included on the agenda of the partnership network's meetings held every 2 years.The results of the Partner Activities Survey are compiled and disseminated to all members of the partnership network. For example, recent surveys indicate that most partners who respond to these semiannual surveys conduct activities to promote NDEP's awareness campaigns. Many of them exhibit at conferences and meetings, participate in health fairs, make presentations, or conduct training sessions and workshops to promote NDEP's messages and materials. Nearly all the partners promote NDEP's Web site resources to their colleagues and constituents.The survey reports also provide data on partner activities to the various work groups, who use the information to assess progress in implementing their strategic plans. NDEP has begun preparing process measure reports on media placements and publications dissemination related to each work group's strategic plan. For example, the report to the Hispanic/Latino work group provides process measures on all of the public service announcements and article placements, the number of Spanish-language publications distributed by the program's clearinghouse, and the number of Web downloads of Spanish-language materials.In 2001 and 2004, NDEP completed and disseminated progress reports on the program's accomplishments, covering 1997 through 2003. To mark NDEP's 10-year anniversary, the program released a 1997-2007 report in September 2007  — is a key measure of progress in controlling diabetes. Findings in the 2 most recent NHANES surveys indicate a significant increase in people with diabetes taking medications to control cholesterol and hypertension .To inform the diabetes, health professional, and public health communities about NDEP's activities, accomplishments, and lessons learned, NDEP writes and submits journal articles. NDEP staff members and partners make presentations about the program at health professional and public health meetings. The program has been on the agenda of periodic prevention summits of the US Department of Health and Human Services, where representatives have had the opportunity to address public health professionals about NDEP's activities and progress.NDEP uses the evaluation activities and reports to assess progress toward reaching program goals and to make midcourse corrections in program activities. NDEP also updates strategic plans, including those for each work group, every 3 years. Program staff members work closely with each work group to review previous plans, assess the latest outcome and process measures, and then develop updated plans for the next 3-year period.NDEP updates its data as new data from NHANES, NHIS, and BRFSS become available each year. The program has created a database for tracking results from these surveys and charting trends on key intermediate outcomes. These trend data are reviewed and shared with the evaluation work group, the executive committee, and the operations committee to keep them abreast of changes in outcome measures. NDEP also used the data to help set priorities and allocate program resources for messages and campaigns that need to be reinforced and disseminated to selected target audiences. Similarly, results of the NDEP survey of the public's knowledge, attitudes, and practices related to diabetes are assisting NDEP in understanding its target audiences, specifically people with diabetes, people with prediabetes, and people at risk for diabetes, and to inform the program of key issues to consider in program planning.NDEP analyzes the results of the semiannual Partner Activities Survey to determine what, if any, changes are needed in program operations to better serve partners' needs. The Partner Activities Survey also has provided insights about the role partners play in promoting and disseminating NDEP messages and what gaps the national program staff needs to fill.The CDC Framework for Program Evaluation in Public Health is an effective tool for structuring an evaluation process that has a high probability of producing findings that programs can use for improvement. It has given the NDEP a roadmap for asking key evaluation questions and for identifying practical solutions to obtaining answers. The process also has been helpful in educating NDEP's funders, partners, and other stakeholders regarding useful, feasible, appropriate, and accurate measures for evaluating a multifaceted public and professional education program such as NDEP, which is significantly different from a randomized clinical trial or preintervention/postintervention study, the research model that is more familiar to stakeholders.Engaging stakeholders in the design and implementation of NDEP's evaluation has been a critical strategy. Without the support, guidance, and contributions of the many individuals and organizations that have been involved in the evaluation process, NDEP would have many more gaps in its evaluation framework.Developing a logic model of the program and identifying the essential evaluation measures needed to assess program outcomes has helped to streamline the evaluation design and eliminate many irrelevant measures. As the program evolves, NDEP will add new components to the logic model, building on previous elements.Most importantly, NDEP has taken to heart the iterative nature of evaluation as represented by the CDC framework. The true value of this approach for NDEP will be its attention to continuous refinement of the program's efforts based on evaluation findings — that is, closing the loop between data and action.


In this article, we describe the National Cancer Institute (NCI) Community Cancer Centers Program (NCCCP) pilot and the evaluation designed to assess its role, function, and relevance to the NCI's research mission. In doing so, we describe the evolution of and rationale for the NCCCP concept, participating sites' characteristics, its multi-faceted aims to enhance clinical research and quality of care in community settings, and the role of strategic partnerships, both within and outside of the NCCCP network, in achieving program objectives.The evaluation of the NCCCP is conceptualized as a mixed method multi-layered assessment of organizational innovation and performance which includes mapping the evolution of site development as a means of understanding the inter- and intra-organizational change in the pilot, and the application of specific evaluation metrics for assessing the implementation, operations, and performance of the NCCCP pilot. The assessment of the cost of the pilot as an additional means of informing the longer-term feasibility and sustainability of the program is also discussed.The NCCCP is a major systems-level set of organizational innovations to enhance clinical research and care delivery in diverse communities across the United States. Assessment of the extent to which the program achieves its aims will depend on a full understanding of how individual, organizational, and environmental factors align  to achieve these improvements, and at what cost. Oncology, like many other medical specialties, is in an era of profound change. The emergence and implications of genomics, proteomics, immunology, and synthetic biology, to name a few fields, will affect the way science is practiced and the way health care is provided . SimilarThe objective of this paper is to describe the NCCCP pilot and the evaluation designed to assess its role, function, and relevance to the research mission of the NCI, as well as its contribution to improving patient care in a non-profit community hospital setting. The program itself is viewed as an organizational innovation and its evaluation as an effort to map the factors that facilitate or impede its ability to meet objectives within a community environment. The evaluation presents a unique opportunity for NCI to focus on program evolution to assess proof of concept as well as on specific indicators of program improvement to assess proof of performance.We begin by describing the developmental trends that provide the context and rationale for the NCCCP pilot. We then describe the conceptual framework used to organize the evaluation for the NCCCP. This framework, together with the NCCCP objectives and components, define the key analytical questions underlying the implementation and sustainability of the program. The paper ends with a discussion of the implications for the research agenda of the NCI within a changing service delivery environment.Two developmental trends within the larger environment provided the rationale for the NCCCP initiative -- NCI's growing commitment to reconfiguring clinical research and the need to improve access to state-of-the-art cancer care in community settings.In 2002, the National Institutes of Health (NIH) launched the NIH Roadmap  as a meaThe NCCCP responds to these initiatives through its emphasis on establishing new partnerships of research and care delivery with organized patient communities, community-based health care providers, and academic researchers. Both the NCI strategic plan  and the Clinical research and care delivery have entered a new era involving an increasing amount of economic, service, and research activity across, rather than within, the boundaries of traditionally defined organizations. Evidence suggests that cancer patients diagnosed and treated in a setting of coordinated multi-specialty care and clinical research are more likely to receive state-of-the-art care -7, and fThis is particularly evident for racial/ethnic minorities, people of lower socioeconomic status, residents of rural areas, and members of other underserved populations who face an unequal burden of cancer . AlthougThe evaluation of the NCCCP pilot incorporates elements of both formative and summative evaluation research and requires an interdisciplinary, recognized, theoretical framework for organizational change, as well as a mixed methods approach using both qualitative and quantitative data collection strategies. The NCCCP evaluation is an unprecedented initiative for the NCI, given its focus on changes in cancer service delivery and research capacity at the community level, its assessment of multiple levels of analysis across multiple timeframes, and its recognition of differing sets of 'initial conditions' across the various sites.In order to capture the essential elements of organizational change/innovation adoption and implementation over time as well as multiple levels of influence on that developmental process, we have combined two major conceptual models from organizational theory: the innovation life-cycle model, which emphasizes stages of implementation ,15, and Figure Using stage models of innovation, the structure, functioning, and performance of the NCCCP pilot can be conceptualized as a process of organizational innovation, unfolding within a multi-layered context of environmental effects that will influence how the pilot sites develop over time. This type of innovation is typically defined as any technology or practice that an organization uses for the first time regardless of whether or not other organizations have previously used the practice or technology. The NCCCP involves a variety of organizational innovations at various phases of implementation. These well-documented phases ,15 inclue.g., CCOP, Community Networks Program, Cancer Centers Program). It was originally designed to address four key goals: enhance community cancer center infrastructure and resources to address health disparities and improve access to evidence-based cancer care for underserved populations; improve the research infrastructure in community settings by supporting increased participation in clinical trials ; encourage the adoption of electronic medical records for care delivery and research, and integrate these research activities with the cancer biomedical informatics grid (ca-BIGR); and assess the feasibility of standardized collection of biospecimens for NCI-sponsored research .The objective of the NCCCP pilot is to test a public-private partnership that is designed to bring state-of-the-art cancer care  to all cancer patients in the community, using linkages with other NCI-sponsored research programs  are less likely to successfully implement and achieve the aims of the NCCCP  than pilot sites embedded within community hospitals that are dominant within their local markets.Hypothesis one: Pilot sites embedded with community hospitals that are in relatively weak market positions , and market influences that might constrain community hospital support of NCCCP activities. The more competitive the local hospital market, the less likely a host-site is to have flexible resources available to support NCCCP activities.Hypothesis two: Pilot sites embedded within highly competitive local cancer services markets  are less likely to successfully implement and achieve the aims of the NCCCP than pilot sites embedded within less competitive local cancer services markets.This hypothesis focuses on the specialized market for cancer services within the community, again recognizing that a competitive environment often constrains organizational focus and resources to 'the bottom line,' and away from innovative programming. However, competition for scarce resources can sometimes push organizations to connect cooperatively to other actors through strategic alliances to reduce uncertainty. Further, the development of strategic linkages to other cancer service providers may be more advantageous at different stages of implementation, depending upon other characteristics of context, or histories of pre-existing linkages .The application of our combined theoretical perspectives requires an evaluation design that brings into focus the ongoing structures and processes within the participating organizations and the environment within which they function, and how these structures and processes evolve over time. The evaluation involves a phased longitudinal assessment of the pilot program over a three-year period. Figure The initial phase of the evaluation will map inter-organizational relationships within programs to project activities as well as the emergence of organizational linkages across pilot sites and between pilot sites and external organizations. Documenting these organizational relationships involves the development of what Miles and Huberman ,25 have Building on the initial assessments, evaluation metrics will be identified that correspond to site-specific work plans in the core components of the program. Special attention will be given to the appropriateness of the metrics for the evaluation questions, and the feasibility of site implementation and data collection in a manner consistent with cross-site evaluation.Based on the information collected in these two phases, a plan has been created that outlines in detail the qualitative and quantitative methods, measures, and data collection protocols that will guide the formal evaluation of the pilot program. This evaluation will involve both a process assessment and an impact assessment of the implementation, operations, and performance of the NCCCP pilot sites. Assessing change in accrual, practice patterns and adherence to evidence-based guidelines within the limited three-year time frame of the pilot is a challenge. However, other community-based initiatives have documented significant changes within a similar time frame including increased accrual with the launch of the minority based - CCOP  as well The process assessment will evaluate the implementation experience of the specific NCCCP pilot sites, and in subsequent data collection activities through individual site assessments and comparative research. It also will assess the program improvements, best practices, and the sites' relationships to NCI-designated cancer centers and other community and national program resources. These process assessments will be supplemented with information from patient and family member focus groups and a cross-site patient survey to elicit the performance of the program from the patients' and families' experience.The impact assessment will address a traditional set of evaluation objectives that should be fully answered and understandable once the early stages of the NCCCP and the pilot formative stages are clearly understood. The following evaluation questions will guide that analysis. They are in large part derived from the conceptual model described above:1. What changes in practice patterns, trial accrual, and adherence to evidence-based practice are attributable to the NCCCP pilot?e.g., NCCCP pilot activities, related hospital organizational factors, local medical staff relationships, NCI partnership, NCCCP network collaborations) are associated with these changes?2. What factors  are necessary to facilitate the expansion of the NCCCP to other community-based cancer programs?5. What is the potential for replicating these results in similar community-based cancer programs that did not participate in the NCCCP pilot? What factors  funding will be tracked.et al. [The cost assessment will include a macro-cost component  that distinguishes between what Leatherman et al.  have teret al. , we hypoNCI increasingly recognizes the critical role that multi-level systems interventions will play in improving health, both in clinical research and in clinical care. Federal research institutions are scrutinized and criticized for the limited existing initiatives that facilitate a rapid translation of research findings into clinical community and public health practice. The NCCCP, initiated as a pilot program, represents the implementation of a major systems-level set of organizational innovations to enhance clinical research and care delivery in diverse communities across the US. Its success will depend, in large part, on inter- and intra-organizational collaboration and cooperation in multiple spheres. Assessment of the extent to which the program achieves its aims will be challenging in a three-year pilot, and will depend upon a full understanding of how individual, organizational, and environmental factors aligned  to achieve these improvements, and at what cost. Current theories of organizational innovation and change provide useful perspectives to guide evaluation design and to help identify why certain results were achieved or not achieved, and options to enable community cancer centers to build on this experience in their efforts to work with NCI to deliver research and evidence-based care to cancer patients where they live.The authors declare that they have no competing interests.All authors contributed to the design, coordination, drafting and review of the manuscript. SBC, ADK, DMO and MLF contributed to the manuscript conceptualization. MJ, JMB, and DMO prepared the tables for the manuscript, as well as figure • Discrete cancer center with medical, surgical, and radiation oncology under one administrative and medical structure• A strong oncology practice leadership group committed to providing vision, oversight, and plans for growth and research support• Physician director with cancer expertise• A clinical trials program with at least 25 patients enrolled annually• At least 1,000 annual new cancer cases• Cancer screening programs• Multi-disciplinary cancer committees• Use of evidence-based clinical guidelines• Patient navigation services• Infrastructure and programs for community outreach to underserved populations and a policy that all patients screened for cancer will receive treatment for cancer• An electronic health record or implementation plans underway• Commission on Cancer accreditation• College of American Pathology, or Joint Commission Accreditation for Laboratory• Hospital Chief Executive Officer (CEO) support• Supplemental funding to support the public/private partnership• No more than $3 million dollars in NCI funding per year


Physical activity is essential for maintaining health and function with age, especially among women. Strength training exercises combat weakness and frailty and mitigate the development of chronic disease. Community-based programs offer accessible opportunities for strength training.The StrongWomen Program is an evidence-informed, community-based strength training program developed and disseminated to enable women aged 40 or older to maintain their strength, function, and independence. The StrongWomen Workshop and StrongWomen Tool Kit are the training and implementation tools for the StrongWomen Program. Program leaders are trained at the StrongWomen Workshop. They receive the StrongWomen Tool Kit and subsequent support to implement the program in their communities.Program dissemination began in May 2003 with a three-part approach: recruiting leaders and forming key partnerships, soliciting participant interest and supporting implementation, and promoting growth and sustainability.We conducted site visits during the first year to assess curriculum adherence. We conducted a telephone survey to collect data on program leaders, participants, locations, and logistics. We used a database to track workshop locations and program leaders. As of July 2006, 881 leaders in 43 states were trained; leaders from 35 states had implemented programs.Evidence-informed strength training programs can be successful when dissemination occurs at the community level using trained leaders. This research demonstrates that hands-on training, a written manual, partnerships with key organizations, and leader support contributed to the successful dissemination of the StrongWomen Program. Results presented provide a model that may aid the dissemination of other community-based exercise programs. Physical inactivity and poor nutrition are leading contributors to chronic disease and premature death throughout the United States and abroad -3. As thResearch has shown that many age-related physiologic declines are not inevitable. Laboratory and home-based studies have demonstrated that strength training — also referred to as progressive resistance training or weight lifting — confers numerous health benefits, particularly for women as they age. Strength training is an activity in which muscles move dynamically against weight (or other resistance) with small but consistent increases in the amount of weight being lifted over time. Done regularly, these exercises build bone and muscle and help to preserve strength, independence, and vitality -16. For Healthy People 2010 guidelines is to increase to 30% the proportion of adults who perform physical activities that enhance and maintain muscular strength and endurance (Despite compelling scientific research and recommendations from the government and the American College of Sports Medicine (ACSM), only 17% of adult women and approximately 12% of all adults aged 65 or older participate in strength training exercises -25. One Exercise: A Guide from the National Institute on Aging  was growing, increasing the number of potential program participants. From 1990 to 2000, the number of women aged 40 or older grew by 23.3%, compared with a 13.2% growth in the total population ,42. WomeHealthy People 2010 report presented data on strength training practices in the late 1990s and goals for 2010, including goals for the proportion of older adults to participate in strength training exercises  includes an easy-to-read information sheet that participants can give to their health care provider, along with a packet of peer-reviewed research articles detailing the scientific basis for the program. We distribute contact information for our staff and each site's leader to foster communication between health care providers and program administration and leaders -13,16-22StrongWomen Program classes last approximately 1 hour and consist of 5 minutes of warm up , 40 minutes of strength training, 5 minutes of balance training, and 5 minutes of cool down . The StrongWomen Program is a 12-week session with two 1-hour classes per week on nonconsecutive days. Generally, eight to 15 participants per class participate in the 12-week session as a group. Most program leaders operate subsequent sessions as a maintenance program for a group that has completed a 12-week session while initiating separate sessions for new groups. We encourage participants to perform the strength training exercises on their own on a third nonconsecutive day of the week. To assist them, leaders distribute copies of the exercise descriptions and illustrations as well as a list of local resources for other physical activity opportunities . Program leaders may choose to make minor adjustments to the recommended program. For example, they may schedule a 10-week class instead of a 12-week class.The greatest variation in program logistics is in participant cost and equipment. The out-of-pocket cost to participants ranges from none (when no class fee is charged and equipment is provided by the program) to $120 for 12 weeks of classes ($5 per class twice per week), plus the need to bring their own equipment. A typical fee for a session in which all equipment is provided by the program is $48 to $96 for 12 weeks of classes (or $2–$4 per class). Ultimately, the program leader or the organization implementing the program determines the fees and how the equipment is acquired and paid for. The equipment per participant includes at least two sets of dumbbells , an adjustable ankle weight (10–20 lb per cuff), and an exercise mat or towel for floor exercises.When participants must purchase their own equipment, it costs approximately $50 to $80 . This estimate is for new equipment and includes shipping and handling fees. Obtaining used equipment and avoiding shipping and handling fees reduce costs substantially.When program leaders provide the equipment, the cost varies but is typically less per participant than when the participants purchase their own equipment because weights and mats can be purchased at bulk discounts up to 50%. For example, the equipment cost for 10 participants ranges from $25 to $40 per participant (and less if used equipment can be obtained). The meeting space and other items that a program leader must provide to participants include an adequately sized, well-lit room; a parking area; sturdy chairs; and bathroom facilities.The foundation of the StrongWomen Program is the written manual (the tool kit) and the hands-on training for program leaders (the workshop). Collectively, the workshop and tool kit form the curriculum for the program. Neither is a stand-alone entity; each potential leader must attend the workshop to receive the tool kit and subsequently implement the program.The StrongWomen Workshop During the full-day workshop , program leaders participate in a series of seminars and hands-on sessions based on the content of the tool kit. During the hands-on sessions, they learn how to instruct participants on the strength training and flexibility exercises. The tool kit describes and illustrates all exercises, and participants model, review, and practice them several times throughout the workshop.Senior Fitness Test  and negotiating discounts from local vendors and organizations. We transcribe notes from the discussion as well as other questions and answers posed during the workshop and distribute them to program leaders at the end of the day. Workshop attendance at Tufts University is $300 per attendee and includes the StrongWomen Program tool kit as well as breakfast and lunch. Cost of attendance at off-site workshops varies, depending on sponsorship and resources, but it never exceeds the $300 fee.The StrongWomen Tool Kit The StrongWomen Tool Kit  is a 200Dissemination began in May 2003 in three parts: recruiting leaders and forming key partnerships, soliciting participant interest and supporting program implementation, and promoting growth and sustainability.Strong Women books and were interested in operating programs within their own organizations on the basis of the research and practical applications presented in the books. They contacted Tufts University with their interest, and the StrongWomen Program began shortly thereafter.The first group of program leaders were members of organizations that have since become key partners with the StrongWomen Program: hospitals, nonprofit outpatient wellness centers, and the National Extension Association of Family and Consumer Sciences branch of the Cooperative State Research, Education, and Extension Services (hereafter referred to as the Extension Service), which is under the direction of the U.S. Department of Agriculture. Individuals from these organizations had seen the Hospitals and nonprofit outpatient wellness centers — in particular, women's wellness centers — are important for the StrongWomen Program because they are often at the core of women's health services within a community. In addition to offering standard medical care, such as family practice, gynecology, and endocrinology, women's wellness centers may offer community programming such as outpatient exercise and nutrition classes. Hospitals are also important for the StrongWomen Program because they provide a large audience of potential program participants and they usually have the space and resources necessary for implementing and maintaining the program. Because the StrongWomen Program is in-house at a hospital or wellness center, physicians perceive the program as a safe and viable option for their patients.Extension Service educators, who are in every county in every state, offer research-based health information and programs to their communities. Collaboration with the StrongWomen Program helps Extension Service agents to bring knowledge, awareness, education, and research-based programming to community members and to increase the Extension Service's reach in underserved and rural locations (a focus area for the Extension Service).Since the inception of the StrongWomen Program, collaboration with clinics and the Extension Service has been vital to disseminating the program. Knowledge of the program has spread within the networks of these organizations by internal newsletters, bulletin boards, word-of-mouth, and formal presentations at professional meetings. At the national conference of the Extension Service in October 2004, we trained 150 program leaders at a StrongWomen Program Workshop. (Most workshops have 15–40 attendees.) The broad geographic range represented by program leaders at this workshop expanded the dissemination of the StrongWomen Program and supported grassroots awareness.Strong Women book series and its related Web site (www.strongwomen.com) and the StrongWomen Program Web site (go.tufts.edu/strongwomen), which can be accessed directly or linked through www.strongwomen.com , this requirement was only a strong recommendation. In addition, program leaders must implement the StrongWomen Program only in nonprofit organizations, such as senior centers, hospital outpatient centers, Extension Service locations, assisted living facilities, or faith-based organizations. A simple preregistration worksheet assesses the qualifications for nonprofit status.The StrongWomen Program highly recommends, but does not require, that program leaders have at least two of the following: an educational background in a field such as health services, nutrition, exercise physiology, physical therapy, or public health; some experience in providing exercise instruction; and certification by a reputable health and fitness organization, such as the National Strength and Conditioning Association, the ACSM, or the American Council on Exercise. As of July 2006, program leaders have ranged in age from 21 to 83 years, with a mean age of 50 years, and have had a diverse range of professional backgrounds .To help new program leaders implement the StrongWomen Program in their communities, we encourage them to follow several steps: 1) read the entire tool kit within 1 week of the workshop; 2) find an organization to host the program and a space to operate the program ; 3) determine the pricing structure, equipment, and schedule for their StrongWomen Program, working with their hosting organization; 4) plan and publicize an informational meeting about the program using the publicity materials provided in the tool kit or other means; and 5) operate their first StrongWomen Program within 3 to 6 months of attending the workshop. New and experienced program leaders can receive support for implementing the StrongWomen Program by e-mail or telephone from the program manager or by networking with other program leaders, many of whom are listed on the StrongWomen Program Web site.Virtually anyone can be a participant in the StrongWomen Program. Program leaders recruit participants through placing advertisements in local newspapers, posting fliers throughout the community, or making announcements through available newsletters and bulletins. The research that provides the scientific basis for the program was conducted with women aged 40 to 91 years; we developed the exercise programs with this audience in mind. However, women may begin to lose muscle and bone mass at an earlier age, and we encourage program leaders to allow all women, regardless of age, to join the classes. As of July 2006, the age range of participants was 21 to 94 years with a mean of 63 years. Strength training is important for men as well; although we encourage program leaders to include men who wish to join, preliminary data show that most participants are women.A variety of mechanisms are in place for long-term maintenance of the StrongWomen Program. Two maintenance objectives are to educate current and potential program leaders using the most up-to-date evidence-informed programming possible and to continue to assist them in implementing and sustaining their programs.The program manager spends approximately 15 to 20 hours per week answering 200 to 300 e-mails and 40 to 60 telephone calls from program leaders. The program manager responds to questions from prospective program leaders who are considering attending a workshop and from current program leaders to support implementation of existing programs. This support includes assisting with space, equipment, and resource issues; helping with incentive and reward programs for participants; working with volunteer assistants to program leaders on class set-up and other issues; and helping program leaders identify modifications for exercises to improve accessibility for some participants. The program manager also publicizes continuing education events and curriculum updates among program leaders through the e-mail list and the StrongWomen Program Web site.teaching process instead of the workshop content. After the second workshop, we require prospective ambassadors to plan, execute, and follow up on their own workshop attended by members of the public and their hosting organization. In addition to allowing the candidate to demonstrate a mastery of the entire curriculum, this workshop provides an opportunity for the candidate to demonstrate support from his or her sponsoring organization; both demonstrations are requirements for becoming an ambassador. The program manager determines the guidelines and protocols in collaboration with each potential ambassador and the hosting organization (because logistics may vary site by site) and attends the workshop to oversee its complete execution.The StrongWomen Ambassador training program is another component of program growth and sustainability. Seven ambassadors conduct workshops in Alaska, Arkansas, Colorado, Kansas, Oregon, and Pennsylvania. These individuals participate in a more extensive training process than the workshop provides, and they are then qualified to hold workshops within their own states to train new program leaders. Ambassadors are also important for program sustainability by serving as local resources for program leaders. To become an ambassador, a program leader must have been actively operating programs in his or her state for at least 6 months. Then he or she must attend a second program leader workshop, which is identical to the first. At this second workshop, we give the teaching materials to the potential ambassadors and instruct them to observe the The aims for long-term sustainability of the StrongWomen Program are to focus efforts on creating supplementary curriculum materials, such as additional evidence-informed exercise programs that participants will require as their strength and fitness increase; on facilitating the leadership and training of additional ambassadors, who are critical to maintaining the reach and momentum of the program's growth; and on creating advanced workshops and educational opportunities for program leaders that will enable them to broaden the scope and capacity of their leadership as agents of positive change in their communities.The program manager conducted site visits at six active StrongWomen Program classes in Kansas, Oregon, and Massachusetts during the first year of dissemination. The primary component of the site visit was observation. During the observation, the program manager observed one or more complete exercise sessions at each site and graded each of the following on a 5-point scale :Adequate space for participants and equipment Equipment safety Location safety and accessibility Execution of exercise program Following the observation, the program manager conducted interviews with participants as a group and with leaders individually. Interviews were related to program logistics , level of participant comfort with their leader, opinions on enjoyment of the program, perceived benefits, and suggestions for changes.The results revealed adherence to the curriculum in terms of space, equipment, location, and exercise program recommendations outlined in the tool kit. Adherence was determined by an average score of at least 4 in all categories. In Kansas, two classes at one site scored an average of 4.25; in Oregon, three classes at one site scored an average of 4.5; and in Massachusetts, three classes at one site scored an average of 4.25.Both program leaders and participants were satisfied with the program logistics and outcomes related to participation. The primary requests from program leaders were related to more guidance on fee structures and scheduling, which is now addressed in greater detail during the workshop. Participant concerns were related to scheduling and the desire for additional nutrition information. Scheduling concerns were subsequently addressed with leaders and expanded upon in the tool kit. Although the took kit already included a chapter on nutrition, a packet of fact sheets on nutrition (similar in content to the information presented to program leaders in the tool kit) is now available for program leaders to distribute to participants.By September 2004, 142 program leaders from 13 states had been trained; 139 were women. Of the 139 women, 31 (22%) were from urban areas, 55 (40%) were from suburban areas, and 53 (38%) were from rural areas. Of 130 program leaders with current contact information, 103 participated in a brief telephone survey . Of the 103 respondents, 72 (70%) had implemented at least one StrongWomen Program, with a mean class size of 11. The mean time between attending the workshop and starting the first program was 12 weeks . On the basis of logistic regression analysis that we performed for a previous study  in this article; most of those data and their analysis  are forthcoming.As of July 2006, 39 workshops had been conducted for 881 program leaders from 43 states. Thirty-eight states have active StrongWomen Programs . On the Our objective to increase middle-aged and older women's access to community-based strength training programs was timely because of the growth of this population, current physical activity recommendations, and increased awareness of the benefits and importance of strength training for women as they age. The monthly StrongWomen newsletter, Web sites, and other correspondence  offered low-maintenance vehicles for regular communication with both existing and potential program leaders as well as leaders from key organizations.Dissemination relied heavily upon establishing solid partnerships within stable organizational networks . An evidence-informed program, a straightforward curriculum, and the flexibility to implement the program in a range of sites facilitated the successful implementation by program leaders. For participants, the credibility, accessibility, and affordability as well as the social and peer support inherent in the community-based structure of the program likely contributed to their participation.The mission of public health is to prevent disease and promote health in the greater population. Among a myriad of important issues, widespread access to exercise opportunities is a public health priority, and community-based programs present a feasible strategy for addressing it. The national dissemination of this strength-training program targeted to middle-aged and older women provides a viable model and systematic method for increasing access to evidence-informed exercise programming in a range of community settings.


A computational program which compares the effciencies of different experimental designs with those of maximum precision (D-optimized designs) is described. The program produces confidence interval plots for a calibration curve and provides information about the number of standard solutions, concentration levels and suitable concentration ranges to achieve an optimum calibration. Some examples of the application of this novel computational program are given, using both simulated and real data.


In 2003, Steps to a Healthier Austin was funded by the Centers for Disease Control and Prevention to implement chronic disease prevention and health promotion activities. We report Steps to a Healthier Austin's partnership with Health & Lifestyles Corporate Wellness, Inc , to provide a worksite wellness program for Capital Metropolitan Transportation Authority , Austin's local transit authority.Capital Metro employs 1,282 people. In 2003, Health & Lifestyles was hired to help promote healthier lifestyles, increase employee morale, and combat rising health care costs and absenteeism rates.Health & Lifestyles provided consultations with wellness coaches and personal trainers, a 24-hour company fitness center, personalized health assessments, and preventive screenings. The program expanded to include healthier food options, cash incentives, health newsletters, workshops, dietary counseling, smoking cessation programs, and a second fitness center.Participants in the wellness program reported improvements in physical activity, healthy food consumption, weight loss, and blood pressure. Capital Metro's total health care costs increased by progressively smaller rates from 2003 to 2006 and then decreased from 2006 to 2007. Absenteeism has decreased by approximately 25% since the implementation of the program, and the overall return on the investment was calculated to be 2.43.Since the implementation of the wellness program in 2003, Capital Metro has seen a reduction in costs associated with employee health care and absenteeism. Capital Metropolitan Transportation Authority  provides public transportation services to the city of Austin, Texas. This organization, established in 1985, has 1,282 permanent employees and 250 contract employees. Approximately half (668) of the permanent employees are bus operators. The organization, with a small workforce, serves a sizeable number of residents  in the growing city of Austin . In 2003, Capital Metro became a self-insured company. By becoming self-insured, Capital Metro was able not only to know exactly what it spent annually on health insurance but also to gain more control over these expenses. From 2003 to 2004, health care costs increased to $13.4 million (a 26.4% increase). Health care costs at Capital Metro are higher for certain people. In 2007, the average medical plan cost for employees with diabetes, hypertension, or high cholesterol was $3,454 per month. Essentially, an employee with a chronic disease incurred expenses 2 to 3 times higher than an employee without a chronic disease.Absenteeism also greatly increased during the early 2000s. Absenteeism costs in 2001 were almost $2 million, and by 2004 they had risen to almost $2.3 million — an increase of almost 15% in only 3 years — during which time the size of the workforce remained fairly static , a private company based in Austin. This company provides comprehensive worksite wellness solutions for employers by using evidence-based programs and practices. Health & Lifestyles created and has been managing the Capital Metro worksite wellness program since its inception in January 2003.Steps to a Healthier Austin is a community-based program, funded by a cooperative agreement from the Centers for Disease Control and Prevention, designed to reduce diabetes, asthma, and obesity by improving nutrition, increasing physical activity, and decreasing tobacco use. The Steps to a Healthier Austin intervention area includes 20 contiguous zip codes that encompass the eastern half of the city of Austin and Travis County. This area has a population of 460,041, of which 41% are Hispanic and 14% are African American .The intervention area population also has higher rates of chronic disease compared with the rest of Travis County. An estimated 12% of adults have been diagnosed with asthma, 7% have been diagnosed with diabetes, 55% are overweight, and 26% are obese. Overall, 21% of the adult Steps to a Healthier Austin population smoke cigarettes, 26% report no leisure-time physical activity, and only 25% meet the minimum recommended daily consumption of 5 servings of fruits and vegetables . In addition, racial/ethnic health disparities are documented in the intervention area; African Americans have higher death rates from heart disease, hypertension, cerebrovascular disease, and diabetes than do whites. Both Hispanics and African Americans in the intervention area are 2.5 times more likely than whites to die from diabetes .To address these issues, Steps to a Healthier Austin partnered with key organizations in the intervention area. Capital Metro was invited to be Steps to a Healthier Austin's worksite wellness partner for the Steps to a Healthier Austin consortium. Capital Metro was chosen for several reasons. First, Capital Metro's workforce is ethnically diverse (33.5% African American and 27.7% Hispanic). Second, Capital Metro's headquarters is in the intervention area, where approximately 47% of Capital Metro employees live . Third, Capital Metro had a promising worksite wellness program and was willing to be a part of the greater Steps to a Healthier Austin program. Finally, Capital Metro's workforce had high rates of chronic disease. In 2005, as part of the partnership between Capital Metro and Steps to a Healthier Austin, Capital Metro received an influx of Steps to a Healthier Austin funds to be used to enhance its wellness program. This increased financial commitment rounded out the offerings of the program and could have been responsible for the improvements in health care cost and absenteeism seen after 2005.Health & Lifestyles launched the Capital Metro wellness program in January 2003. Participants included bus operators, mechanics, and administrative employees. The initial program consisted of 1 full-time employee who was available to provide health information and one-on-one consulting and education, who made brochures on health information available, and who coordinated wellness seminars and demonstrations by local vendors. Over time, the wellness program expanded to include multiple offerings, including on-site fitness centers, healthier food options, cash incentives, health newsletters, workshops, dietary counseling, and smoking cessation programs. In 2005, Steps to a Healthier Austin made Health & Lifestyles, via Capital Metro, a formal consortium partner and since that time has funded the purchase of more equipment and the hiring of 1 part-time personal fitness trainer.The first company fitness center was launched in June 2006 and is open 24 hours per day. The fitness center includes stationary weight lifting equipment, free weights, exercise bands, balls, exercise mats, shower facilities, and lockers. The fitness center is located at the corporate headquarters, and membership is only $5 per month for employees. Free assessments and personal training are provided Monday through Friday from 6:00 am to 6:00 pm.2 max), and other factors. The machine tracks employees' progress and provides data needed to determine their eligibility for cash incentives. It also provides a nutrition and exercise plan, to which the personal trainer and dietitian can refer when assisting each employee with his or her fitness goals. The TriFIT has been an integral part of the fitness component of Capital Metro's worksite wellness program, and Capital Metro recently purchased a second machine to meet the needs of the increasing number of employees participating in this program.Shortly after the new fitness center opened, Steps to a Healthier Austin funded the purchase of a TriFIT machine . The machine provides employees with a full body assessment and plan for achieving fitness goals. A full body assessment includes weight, flexibility, body composition, maximal oxygen consumption  or 126 mg/dL (employees with diabetes).Total cholesterol drops to below 200 mg/dL.Body fat decreases by 3% or more in 6 months.The employee loses 10% of his initial weight.2 max increases by 5 mL/kg/min in 6 months.VOCash incentives are also used to reward employees for joining weight-loss programs, for sustaining use of the on-site gym, and for achieving other health-related goals, such as participation in races and stopping smoking.In addition to offering physical activity opportunities, as part of its wellness program, Capital Metro has improved access to healthy food. Capital Metro now provides what it calls its Healthy Options Cafe. Employees are given weekly discount coupons to purchase healthy cafeteria items; $5 worth of coupons are available each week for each Capital Metro employee and contractor. Capital Metro also instituted a healthy vending contract and stipulated that 60% of offerings had to be healthy choices, and healthy options are cheaper than less healthy options. No snacks can be "value sized," and all snacks must contain less than 5 g fat (nuts and seeds exempt) and less than 30 g carbohydrate (fruit exempt). Beverage choices are 25% water or sports drinks, 25% fruit juices, 25% low-calorie soft drinks, and 25% regular soft drinks.Besides improving physical activity access and providing healthier food choices, Capital Metro offers on-site Weight Watchers classes. Employees are reimbursed up to $180 for the cost of the program if they lose 10% of their initial weight and if they attend 85% of meetings during a specified time period. In addition, employees earn $75 for losing 10% of their initial body weight, $125 for reaching their goal weight, $200 for maintaining their goal weight for 6 months, and an additional $200 each year the weight is maintained for up to 5 years.Capital Metro also has a strong tobacco-cessation component in its wellness program. Employees receive up to $100 in reimbursement for over-the-counter nicotine replacement products, and they can earn $250 for every year they remain tobacco-free for up to 5 years.Other elements of the wellness program include a wellness calendar and newsletter. The monthly calendar details health-related events and highlights health topics related to the national health calendar, with an emphasis on disease prevention. Additional promotions  are planned throughout the year relevant to diseases that are prevalent among the employee population.Health promotion events offered on-site include health screenings, educational programs, cooking demonstrations with the Sustainable Food Center , chair massages, and stretching sessions. Employee health screenings are held throughout the year and include ultrasound stroke screening, vision and hearing screenings, mammograms, and glucose, cholesterol, and blood pressure testing. Screenings are offered at no charge.Once each year, Capital Metro hosts a Health and Wellness Expo that presents information from vendors and conducts games with prizes. Among other activities, this event promotes the annual Texas Round-Up and invites employees and their families to participate in the family mile, 5K, or 10K event. Other programs offered throughout the year include healthy eating workshops presented by dietitians, a bike loan program, a walking club, and stress-management workshops. In 2008, a second fitness center opened at a different site.Participation in the wellness program increased markedly from 2006 to 2007. Currently, 300 employees — approximately 25% of the workforce — are enrolled in the program. Of these, half are bus operators and the rest are administrative or managerial staff. The number of visits to the wellness center increased from 1,571 in 2006 to 3,174 in 2007. The number of fitness center workouts recorded increased from 2,137 in 2006 to 11,176 in 2007. The number of personal training sessions also increased, from 561 visits in 2006 to 2,348 visits in 2007.Cash incentives have been received by 172 participants for achieving their wellness goals. Specifically, Capital Metro has paid $7,800 in weight loss incentives, $7,500 for smoking cessation, $10,200 for gym memberships, $4,500 for TriFIT assessments, and $1,500 for 5K and 10K participation. Total cash incentives paid to date have reached $31,500.Eighty-six of 133 people who had at least 2 TriFIT assessments in 2007 lost weight, a total of 791 pounds or approximately 10 pounds per person and an average loss of 3% body fat. Participants' average blood pressure decreased by 4 mm Hg, but because of the fluctuating nature of blood pressure, we cannot necessarily consider this to be a significant decrease.People participating in the TriFIT assessment can also complete an optional health risk assessment. Unfortunately, many people feel the assessment involves disclosing too much private information, and they do not find it to be a useful tool they should fill out multiple times. The repeat risk assessment data are limited, and no conclusions can be drawn from them at this time. Consequently, the program focuses on goal-oriented programming. Participants receive individual coaching and information to help them achieve individual goals, such as weight loss, increased fitness, and increased flexibility.From 2003, when the wellness program was launched, to 2006, Capital Metro's health care costs actually increased each year , but theA return on investment (ROI) was calculated to show the effect of the program and to help management understand the benefits it provided. To calculate ROI, the health care and absenteeism savings were divided by the cost of the program during the 4 years , for an ROI of 1.86 for health care savings, 0.57 in savings due to reduced absenteeism, and a total ROI of 2.43 from 2004 through 2007 .Since its beginning in 2003, the wellness program at Capital Metro has shown promising results in improving employee health and reducing costs associated with health care and absenteeism, and the financial benefits outweigh the annual investment (2.43 ROI). Employees engage in more physical activity, have better knowledge of disease management (diabetes and asthma), have better eating habits, and smoke less than they did before the program was implemented. Health care and absenteeism costs have been reduced and are continuing to decline, most likely as a result of the program. Managerial staff have reported that employee morale has increased since the program was implemented. Most importantly, however, we believe that the wellness program has the potential to reduce the prevalence and severity of chronic diseases, allowing Capital Metro employees to lead longer, healthier lives.


The purpose of this study was to demonstrate how formative program process evaluation was used to improve dose and fidelity of implementation, as well as reach of the intervention into the target population, in the "Active by Choice Today" (ACT) randomized school-based trial from years 1 to 3 of implementation.The intervention integrated constructs from Self-Determination Theory and Social Cognitive Theory to enhance intrinsic motivation and behavioral skills for increasing long-term physical activity (PA) behavior in underserved adolescents . ACT formative process data were examined at the end of each year to provide timely, corrective feedback to keep the intervention "on track".Between years 1 and 2 and years 2 and 3, three significant changes were made to attempt to increase dose and fidelity rates in the program delivery and participant attendance (reach). These changes included expanding the staff training, reformatting the intervention manual, and developing a tracking system for contacting parents of students who were not attending the after-school programs regularly. Process outcomes suggest that these efforts resulted in notable improvements in attendance, dose, and fidelity of intervention implementation from years 1 to 2 and 2 to 3 of the ACT trial.Process evaluation methods, particularly implementation monitoring, are useful tools to ensure fidelity in intervention trials and for identifying key best practices for intervention delivery. Process evaluation can be used to explain why interventions succeed and fail, and whether there are characteristics or mechanisms involved in the program's implementation that potentially mediate or moderate outcomes. In large-scale trials the importance of monitoring program implementation has been highlighted -10 and tEvaluations of implementation are especially important given that few studies have achieved full implementation in real-world settings. This isProcess evaluation data used for formative purposes during a developed intervention, as described in this study, should be distinguished from process evaluation used for formative evaluation during the developmental phases of an intervention -23. In aA growing literature has included process evaluation as a key element in evaluating success of implementation in large-scale PA trials. The Pathways initiative - a large-scale, multi-site, 3-year study testing a school-based intervention, used process evaluation methods in evaluating implementation of an intervention to lower percent body fat in American Indian children . PathwayOther large trials have reported summative process evaluations which have implications for using process evaluation data for formative purposes. For example, in one investigation of the SPARK program , a multi-component elementary school program which sought to promote PA in elementary children, process evaluation data was obtained to determine success of implementation. The SPAIn "Switch-Play," Salmon and colleagues  sought tth graders per school). The formative data from each year were used to provide corrective feedback to keep the intervention "on track", and was part of a comprehensive approach to process evaluation for monitoring and assessing program implementation in ACT [The purpose of the present study was to demonstrate how program process evaluation was used in a formative manner  to impron in ACT .th grade, 2) have parental consent to participate, 3) agree to study participation and random assignment, and 4) be available for a 6-month follow-up. Adolescents were excluded from participation if they 1) had a medical condition that would interfere with the prescribed PA intervention plan, 2) were developmentally delayed such that the intervention materials would not be cognitively appropriate or, 3) were currently in treatment for a psychiatric disorder.A total of 24 middle schools  in South Carolina were recruited to participate in one of the two after-school programs  over the 4 years (6 schools per year) of the trial implementation. To be eligible, adolescents were required to 1) currently be enrolled in the 6The ACT trial is a group-randomized cohort design with three intervention and three comparison schools per cohort (year). The schools were paired prior to recruitment and randomization to condition to avoid possible bias or confounding by socio-demographic differences. The criteria on which the schools were paired included: 1) school size, 2) proportion of minority versus non-minority ethnicity, 3) proportion of students enrolled in free and reduced lunch program and 4) urban or rural community setting. Baseline psychosocial, PA, and anthropometric measures were obtained prior to randomizing schools in each pair. The measurement team and intervention team maintained separate entities to blind the measurement staff to group conditions. Data was collected by trained measurement staff for each pair of schools on the same days over a period of two weeks in a lagged timeline . This paper reports on years 1, 2, and 3 of the trial.Two phases of recruitment were implemented yearly during the ACT trial. The first phase involved attending parent orientations at school events to provide program information and obtain informed consent. Following the orientation a second phase of recruitment took place during the school day. Pep rallies and homeroom visits were two methods of recruitment implemented during the second phase to increase enrollment and excitement about the programs . Randomization of schools to programs  occurred after recruitment and baseline assessments were completed. The recruitment target was 60 students from each school.The intervention integrated constructs from Self-Determination Theory (SDT) ,30 and SInvestigators and staff defined the "essential elements" for the ACT intervention guided by constructs from the ACT theoretical frameworks (SDT and SCT) . The essThe ACT intervention was implemented on Mondays, Tuesdays, and Thursdays for two hours after school. The ACT intervention was supervised by a team leader who had expertise in implementing physical activities in youth. The team leaders provided the structure for the ACT intervention components including the PA component. Four additional trained staff provided oversight and assisted with facilitating the program components. The program had three main components: snack/homework (30 minutes), a PA component that included activities which the students selected each week of MVPA (60 minutes), and a SCT and motivational component  during which intervention staff taught participants behavioral skills and motivational strategies to increase their PA at home and with friends (30 minutes).The General Health Education Program (comparison program) focused on nutrition, stress management, drug prevention, and school drop-out prevention. The program was held on the same days and times as the ACT intervention program. The health education modules were taught in an interactive format and students typically rotated from one station to then next every twenty minutes .ACT intervention staff and volunteers were trained prior to the beginning of intervention each school year and received one booster training session midway through the intervention period. Training content included: an overview of the ACT trial purpose, an introduction of the behavioral theories and models guiding the ACT intervention, a detailed review of the ACT intervention manual, staff expectations regarding implementing the intervention and record keeping, team building, interacting with students, first aid, and administrative responsibilities and procedures. Training sessions were didactic and interactive. The interactive components provided opportunities for the staff to practice intervention strategies and for training leaders to identify and correct any problem areas for the staff during the training.as implemented during the after-school program. The evaluation questions, presented below, guided the selection of methods and tools: 1) Fidelity - To what extent was the social environment autonomy supportive?, 2) Dose delivered -To what extent were all planned components of the program provided to program participants? and 3) Reach-What percentage of the possible target group attends each week of the program?ACT process evaluation methods were guided by the essential elements framework that defined dose and fidelity or "complete and acceptable delivery" of the ACT intervention ,33,34. TProcess evaluation data were collected by a trained, independent process evaluator using systematic observation of after-school program activities. Through observation and use of a quantitative checklist and ratings scales, the process evaluator assessed the extent to which the ACT after-school social environment achieved the essential elements upon which the program was designed. To assess dose and fidelity, the process evaluator observed the two-hour program for each day of the program for two weeks (3 program days for two weeks) at three points in time, early (weeks 1 and 2), midpoint (weeks 8 and 9) and near the end (weeks 15 and 16) of the 17-week program. It was possible to observe each program in the same phase of implementation because program implementation was staggered by 2 weeks across the three intervention sites.An overview of the fidelity and dose process tools is provided in Tables Many implementation fidelity ratings reported in the literature pertain to implementation or a curriculum or set of program activities and a rating ranging from "poor to excellent" has typically been optimal ,13. In tAs shown in Table After the intervention was completed each year, the process evaluation data were examined to determine areas of strengths and weaknesses and to make adjustments to keep the program "on track" for the next year (cohort). Based on process evaluation data in year 1, changes were made in the subsequent program years to ensure complete and acceptable program delivery and to maximize reach into the target population.Demographic data for participants in years 1, 2, and 3 are provided in Table All but one school met the recruitment minimum goal of 60 participants. As shown in Table In response to attendance challenges in year 1, a tracking system was developed to more easily contact parents whose children had poor attendance at ACT. Detailed protocols were developed for ACT and general health intervention participants. The protocols included detailed phone scripts and follow-up actions for various scenarios . The information was then included in a tracking database that included codes for the various scenarios. Staff attempted to collect updated contact information if it was not readily available from the school or provided by the participant.The goal of implementation monitoring in ACT was to reach implementation criteria/goals and to ensure complete and acceptable delivery of ACT by criteria that were determined prior to program implementation. Dose delivered (completeness of program delivery) and fidelity were captured through 49 observations in year 1, 47 observations in year 2, and 48 observations in year 3. As shown in Table There were both curricula as well as visual and organizational changes made to the manuals. The curricula changes included not repeating any weeks during the program. In addition, some activities were taken out that weren't feasible. For example, a camera activity was taken out because it was not feasible to give each child in each school a camera to complete the activity. Visual and organizational changes were also made to the manual. Each daily sheet was changed to include a "to do" list. A "what's the point?" box was added near the top to reinforce top priorities for each daily activity, and which ACT essential element was being covered that day. Fun and interesting visuals were also added to make the daily sheets more appealing to ACT staff; who were primarily school teachers and staff. Finally, important points conveying the main emphasis of ACT  were bolded and functional definitions were added where appropriate.As shown in Table Significant changes were made in staff training to attempt to improve program dose and fidelity. A core-training with all the schools team leaders was developed and implemented prior to any of the programs start dates. In this training, team leaders spent 20 hours being exposed to all the essential elements of ACT. They participated in hands-on activities that helped them become more familiar with the basic elements of the program. After the core training, team leaders then helped facilitate their school's staff training. The team leaders took on a more active and leadership role in these 12-hour school trainings. Mid-year, a booster training session was held and feedback was given to each team staff member by the ACT project director. Constructive feedback was given based on internal evaluations that had been conducted by the project director. Finally, the external evaluator's criteria sheet was shared with staff members so that they would become familiar with exactly how the essential elements of the program were translated into specific staff tasks and responsibilities.Overall, this study suggests that the formative evaluation contributed to improving the intervention dose, fidelity, and program attendance. The intervention itself was not changed; rather, the changes made enabled ACT staff to do a better job of delivering the planned intervention. Many of the changes were related to staff training and monitoring methodology. Specifically, changes in the staff training, the intervention manual, and tracking of students' participation were associated with reaching the goals for dose, fidelity, and reach when comparing years 1 through 3 of implementation. These findings have important implications for future research and suggest that formative process evaluation procedures can inform and enhance program implementation in on-going trials.Using process evaluation data in a formative manner is frequently recommended; however, there are relatively few reports describing formative compared to summative uses of process evaluation. A commonly cited challenge, particularly in large trials, is the time frame required for data collection, management, synthesis, and reporting . This inIn a review conducted by Durlak and DuPre , it was The authors declare that they have no competing interests.st author) is the Principal Investigator on the study and participated in the work of the whole content of this manuscript including conception, design, data acquisition, interpretation of data, drafting the manuscript, critical revision, and supervision. SG (2nd author) is a Research Associate, Assistant Professor, who was involved in the conception and design, data acquisition, interpretation of data, drafting the manuscript, critical revision, and supervision. RS (3rd author) is a Co-Investigator on the study and participated in the conception and design, data acquisition, analysis and interpretation of data, drafting the manuscript, critical revision, and supervision. HKU (4th author) is a Co-Investigator on the study and participated in data acquisition, drafting the manuscript, critical revision, and supervision. DM (5th author) is a graduate student on the study who participated in data acquisition, and drafting the manuscript. LM (6th author) is a Research Associate who participated in the acquisition of data, data analysis and interpretation, critical revision of the manuscript, and supervision.DKW (1


P=0.02), only peritumoural LVI, and not BVI, was associated with the presence of lymph node (LN) metastases (pperi=0.002). In multivariate analysis, peritumoural LVI was the only independent determinant of LN metastases. Furthermore, the number of vessels with LVI was larger than the number of vessels with BVI (P=0.001) and lymphatic emboli were larger than blood vessel emboli (P=0.004). We demonstrate that it is possible to distinguish between BVI and LVI in BC specimens using specific lymphatic endothelium markers. This is important to study the contribution of both processes to BC metastasis. Furthermore, immunohistochemical detection of lymphovascular invasion might be of value in clinical practice.Recently, peritumoural (lympho)vascular invasion, assessed on haematoxylin–eosin (HE)-stained slides, was added to the St Gallen criteria for adjuvant treatment of patients with operable breast cancer (BC). New lymphatic endothelium-specific markers, such as D2-40, make it possible to distinguish between blood (BVI) and lymph vessel invasion (LVI). The aim of this prospective study was to quantify and compare BVI and LVI in a consecutive series of patients with BC. Three consecutive sections of all formalin-fixed paraffin-embedded tissue blocks of 95 BC resection specimens were (immuno)histochemically stained in a fixed order: HE, anti-CD34 (pan-endothelium) and anti-D2-40 (lymphatic endothelium) antibodies. All vessels with vascular invasion were marked and relocated on the corresponding slides. Vascular invasion was assigned LVI (CD34⊕ or ⊖/D2-40⊕) or BVI (CD34⊕/D2-40⊖) and intra- (contact with tumour cells or desmoplastic stroma) or peritumoural. The number of vessels with LVI and BVI as well as the number of tumour cells per embolus were counted. Results were correlated with clinico-pathological variables. Sixty-six (69.5%) and 36 (37.9%) patients had, respectively, LVI and BVI. The presence of ‘vascular’ invasion was missed on HE in 20%  and 65%  of cases. Although LVI and BVI were associated intratumourally ( Breast cancer (BC) is the most frequent cause of death in women between 35 and 55 years of age  have supported BC metastasis models hypothesising that lymphatic and haematogenous dissemination in BC are two complementary and specific pathways  mastectomy specimens containing invasive BC examined at the Department of Pathology, General Hospital AZ St-Augustinus, Wilrijk, Belgium, were prospectively included in this study after written informed consent. As the investigators had no access to clinico-pathological data, all 109 included resection specimens were processed and analysed as described below. When the anonymisation code was broken, one patient with documented distant metastases and five patients with a local recurrence were excluded. From eight patients with more than one resection specimen, only the first was taken into account. Therefore, 95 patients with operable breast carcinoma were included in this study. One patient refused to have an axillary lymphadenectomy, hence no information on LN status was available in this patient. eviously . A fibroμm sections of all formalin-fixed paraffin-embedded (FFPE) tissue blocks containing invasive carcinoma were cut and numbered. These slides were (immuno)histochemically stained in a fixed order: the first section of each block was stained with HE, the second with antibodies against CD34 (panendothelium marker) and the third with antibodies against D2-40 (lymphatic endothelium marker). The CD34 and D2-40 immunohistochemistry (IHC) were performed on the Dako Autostainer  using the Envision Dual Link + as the detection system (Dako). The CD34  and D2-40  primary antibodies were both incubated for 30 min.After routine histopathological examination, three consecutive 4 HE: tumour cells within an endothelium-lined vessel-like structure; CD34 or D2-40: tumour cells within an immunohistochemically positive vessel-like structure). Every slide was assessed by two investigators (GVdE and CC) without access to the data of other sections of the same resection specimen or other stainings of the same section. Every vessel with tumour cell invasion according to the criteria on one of the three consecutive sections was relocated on the other slides and assigned BVI or LVI based on the immunohistochemical staining profile  were screened for (lympho)vascular invasion using strict criteria  were not normally distributed, the median (25–75th percentile) value is reported and compared between different groups with a Mann–Whitney U-test. For analysing associations between categorical variables , the χ2 test or – when the assumptions of the χ2 test were not met – the Fisher's exact test, was used. To build a multivariate model predicting axillary LN involvement, a logistic regression with backward procedure including intra- and peritumoural ‘vascular’ invasion, LVI and BVI was performed.Statistical analysis was performed with the SPSS 12.0 software package. A ρ=0.40, P<0.001). Lymph vessel invasion (P=0.01) and BVI (P<0.001) were more frequently demonstrated in larger tumours and the number of vessels with LVI  and BVI  was correlated with the size of the tumour. The presence of LVI or BVI was not associated with the number of blocks investigated, but the number of vessels with LVI , not BVI , was correlated with the number of blocks.A total of 3297 vessels  with LVI and 135 vessels with BVI  were demonstrated in consecutive sections of 446 FFPE tissue blocks. The median number of blocks per patient was four (range 1–20). The number of FFPE blocks investigated was significantly correlated with the size of the tumour (P=0.02), but not peritumourally (P=0.11).Lymph vessel invasion was more frequent than BVI. Sixty-six (69.5%) patients had LVI  and 36 (37.9%) patients had BVI . In 28 (29.5%) resection specimens, both LVI and BVI were found and in eight and 38 resection specimens, only BVI or LVI were found, respectively. The presence of LVI was associated with the presence of BVI intratumourally (P=0.004) and the median sizes of the lymphatic and blood vessel emboli were 16 (8–35) and eight (5–11) intravascular tumour cells (P=0.001). If both LVI and BVI were present, the median of the fraction of lymph vessels of the total number of vessels with vascular invasion was 80.0% (50.0–95.4%). P=0.003) or in cases with BVI only (P=0.04). Furthermore, the median number of tumour cells/embolus was significantly higher in lymph vessel emboli than in blood vessel emboli, both in cases with LVI/BVI only (P=0.03) or in cases with both LVI or BVI (P=0.02).Furthermore, LVI was more extensive than BVI and lymphatic emboli were larger than blood vessel emboli. In seven cases, more than 100 vessels with LVI were found, whereas the maximum number of vessels with BVI was 16. In cases with either LVI or BVI, the median numbers of lymph and blood vessels with vascular invasion were, respectively, six (2–20.5) and three (1–5.75)  and with BVI . Nevertheless, the presence of LVI and BVI was missed intratumourally in 71.0 and 67.9% of immunohistochemically positive cases, respectively. Peritumourally, only 22.4 and 25% of cases with, respectively, LVI and BVI were missed on HE. Furthermore, not only the presence of LVI and BVI was underestimated on HE, but also the extent. On HE, vessels with BVI were more frequently missed than vessels with LVI: the median percentage of vessels with BVI and LVI seen on HE was, respectively, 0.0 (0.0–31.3) and 26.8 (0.0–54.0) (P=0.009). Overdiagnosis of ‘vascular’ invasion on one or more HE-stained slides was seen in 23 (24.2%) patients.On HE-stained sections it is impossible to differentiate between blood and lymph vessels. Therefore, the presence of ‘vascular’ invasion, including both BVI and LVI, was assessed. When only the results of this assessment were taken into account, 54 (56.8%) patients had vascular invasion . Both intra- and peritumourally, vascular invasion assessed on HE was associated with LVI (P=0.01) ‘vascular’ invasion on HE was associated with the presence of LN metastases. The presence of peritumoural LVI was strongly associated with the presence of LN metastases (P=0.002). For intratumoural LVI, a trend to a positive association was found (P=0.07). Neither the presence of intra-, nor the presence of peritumoural BVI was associated with LN involvement  was the most important determinant of axillary involvement. The presence of peritumoural  BC patients vascular invasion in BC is LVI and that lymph vessel tumour emboli are larger than blood vessel tumour emboli. This suggests that LVI and BVI are not just different routes of BC metastasis, but that both pathways are characterised by a different biology.