Intelligence Community: Actions Needed to Improve Planning and Oversight of the Centers for Academic Excellence Program

A trusted, diverse workforce with the right expertise is critical to ensuring the IC achieves its mission of delivering distinctive, timely insights with clarity, objectivity, and independence. ODNI established the IC CAE program in 2005 to educate highly qualified students of diverse backgrounds and encourage them to pursue careers in the IC. ODNI and DIA have provided 29 colleges a total of 46 IC CAE grants through fiscal year 2018, totaling approximately $69 million through fiscal year 2021.

This report evaluates the extent to which (1) DIA has planned and overseen the IC CAE program since 2011 and (2) selected IC elements are participating in the IC CAE program and have clearly defined roles.

GAO reviewed IC CAE documentation related to DIA program planning and oversight from 2011 through 2019 and applied key practices of sound planning to evaluate DIA's management of the program. GAO interviewed selected IC elements and IC CAE college officials and reviewed related documentation to assess program planning and implementation.

The Defense Intelligence Agency (DIA) has not sufficiently planned and overseen the Intelligence Community (IC) Centers for Academic Excellence (CAE) program—intended to create an increased pool of culturally and ethnically diverse job applicants for the IC—after the program transitioned from the Office of the Director of National Intelligence (ODNI) to DIA in 2011. Specifically, DIA has not applied most of GAO's key practices of sound planning in overseeing the program (see table), thus challenging decision makers' ability to determine the program's return on investment.

Specifically, while DIA has developed some short-term goals and plans for the program, DIA has not established results-oriented program goals or an overall strategy that details the agency resources and processes required to achieve the program's mission. Similarly, DIA collected some data for the program and required colleges to provide reports on significant program accomplishments, but these data are not complete or reliable and have not been used to comprehensively evaluate the program's success. As oversight responsibility for the IC CAE program transitions back to ODNI in fiscal year 2020, ODNI will not be able to determine the extent to which the program has been successful in achieving its mission without establishing and documenting goals with targets and milestones; developing strategies to achieve those goals; and defining, collecting, and reporting comprehensive performance measures.

Selected IC elements are participating in the IC CAE program to varying degrees, but DIA has not established a process for monitoring and assessing IC elements' participation or clearly defining IC elements' role in the program. The IC CAE program is a collaborative effort that allows IC elements to participate in college events, such as IC CAE recruitment events. However, not all IC elements participate in the program. As IC CAE program manager, DIA has engaged with IC elements in a variety of ways, but this engagement has not resulted in consistent participation among the IC elements. Moreover, program documentation has not clearly defined IC elements' roles and responsibilities for participation. Without a process for monitoring and assessing IC elements' participation and clearly defining roles and responsibilities, ODNI will neither be able to identify reasons for the lack of IC element engagement nor ensure that IC elements are taking advantage of the IC CAE program and its goal of creating a diverse pool of applicants for the IC.

GAO is making seven recommendations to the Director of National Intelligence, including that ODNI establish and document results-oriented goals and strategies for the IC CAE program; define, collect, and report comprehensive performance measures; and clearly define the roles and responsibilities of the IC elements for participation in the program. ODNI concurred with the recommendations but did not identify steps it plans to take to implement them.

A trusted, diverse workforce with the right expertise is critical to ensuring  the Intelligence Community (IC) achieves its mission of delivering  distinctive, timely insights with clarity, objectivity, and independence. The  IC established the Centers for Academic Excellence (CAE) program in  2005 to serve the mission-critical objectives of educating highly qualified  students of diverse backgrounds and encouraging them to pursue careers  throughout the IC. The program is intended to develop and expand  opportunities in the IC through grants provided to institutions of higher  education (hereafter referred to as colleges) chosen on a competitive  basis. Specifically, the program aims to increase the pool of eligible  applicants for the IC in highly desired skills and competencies, targeting  women, minorities, and individuals with diverse ethnic and cultural  backgrounds, skills, language proficiency, and expertise.

A total of 29 colleges have received 46 IC CAE grants through fiscal year  2018 and the total amount of projected grant funding from fiscal year  2005 through fiscal year 2021 is approximately $69 million. The Office of  the Director of National Intelligence (ODNI) led the IC CAE program from  2005 through 2011. In 2011, the Defense Intelligence Agency (DIA) was  appointed the program executive, while ODNI maintained budgetary  oversight of the program. According to ODNI officials, they are planning to  transition the IC CAE program back to ODNI in fiscal year 2020 as one of  multiple actions related to a recent review of DIA’s roles and mission.

You asked us to review issues related to the IC CAE program. This report  examines the extent to which (1) DIA has planned and overseen the IC  CAE program since 2011, and (2) selected IC elements are participating  in the IC CAE program and have clearly defined roles. Appendix I  provides a history of ODNI’s management of the IC CAE Program from  2005 to 2011.

For objective one, we reviewed ODNI and DIA documentation indicating  how the agencies have planned and overseen the IC CAE program  during DIA’s management of the program from 2011 to 2019. Specifically,  we reviewed annual performance reports the agencies have produced  about the program, internal guidance for IC CAE program managers, and  reports produced by IC CAE colleges. We selected a non-generalizable  sample of seven colleges participating in the IC CAE program. We  developed this sample to achieve a mix of active and legacy colleges,  including both new and established IC CAE programs, consortiums with  smaller colleges, and colleges in different regions of the country serving  different minority populations. We collected interim and final reports  prepared by these colleges and interviewed college officials to gain their  perspectives on how the program is performing. The colleges we selected  were Chicago State University, Florida International University, Florida  Memorial University, Rutgers University, The State University of New  Jersey, California State University (San Bernardino), California State  University (Long Beach), and Pennsylvania State University. We  evaluated the program’s management against key practices we have  identified for sound strategic management planning. Furthermore, we interviewed officials at ODNI and DIA to understand their roles in defining  program objectives and performance measures.

For objective two, we reviewed ODNI, DIA, and IC CAE colleges’ reports  to track and identify IC element participation in the IC CAE program  during DIA’s management of the program from 2011 to 2019. We  selected a non-generalizable sample of eight of the 17 IC elements to  discuss the elements’ participation in the program and their perspectives  on the program’s performance. To develop this sample, we reviewed the  size of the IC element and its participation in the program based on ODNI  and DIA documentation. Our sample includes the “Big 6” IC elements,  which represent the largest of the 17 IC elements, and we randomly  selected two smaller IC elements—the Department of State’s Bureau of  Intelligence and Research and the Department of Energy’s Office of  Intelligence and Counterintelligence. We interviewed each IC element in  our sample and collected documentation on their hiring strategies and  diversity goals. We evaluated coordination between the IC CAE program  executive and the selected IC elements against Standards for Internal  Control in the Federal Government and selected leading practices we  have identified for interagency collaboration.

We conducted this performance audit from August 2018 to May 2019 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

The Director of National Intelligence serves as head of the IC and acts as  the principal adviser to the President and National Security Council on  intelligence matters related to national security. The IC is comprised of 17  executive branch agencies and organizations, generally referred to as IC  elements. These IC elements include two independent agencies, eight  elements within the Department of Defense, and seven elements across  five other executive departments. Table 1 provides a list of the 17 IC  elements.

In its first National Intelligence Strategy, issued in 2005, ODNI highlighted  the importance of a diverse talent pool to address the complex challenges  the IC faced. In its most recent strategy released in 2019, ODNI  reaffirmed and emphasized the IC’s commitment to developing and  retaining a diverse workforce to address enduring and emerging mission  requirements. The 2019 National Intelligence Strategy defines diversity  as a collection of individual attributes that include, but are not limited to  national origin, language, race, color, mental or physical disability,  ethnicity, sex, age, religion, sexual orientation, gender identity or  expression, socioeconomic status, veteran status, and family structure.

The Intelligence Authorization Act for Fiscal Year 2004 directed the  Director of Central Intelligence to develop a pilot project to test and  evaluate alternative innovative methods to promote equality of  employment opportunities in the IC for women, minorities, and individuals  with diverse ethnic and cultural backgrounds, skills, language proficiency,  and expertise. The first pilot was initiated at Trinity Washington  University in Washington, D.C., with a 1 year contract totaling $250,000.  The college developed and designed curricular components to align with  IC mission skills sets and competencies and competitively selected  students to participate in the college’s IC CAE Scholars Program. In the  first year, the program sponsored nine students who were selected as IC  CAE scholars.

After the initial pilot year at Trinity Washington University, the pilot  program was expanded to three additional colleges—Tennessee State  University in Nashville, Tennessee; Florida International University in  Miami, Florida; and Clark Atlanta University in Atlanta, Georgia. In 2005  ODNI, on behalf of the IC, established the IC CAE program. ODNI  reported that by 2007, 65 scholars participated in the program from these  four CAE colleges. By 2008, ODNI had expanded the pilot to six  additional colleges. Overall, the 10 participating colleges increased the  student population to 338 IC CAE scholars. During the 2008 to 2009  academic year, ODNI established a continuity strategy with the initial 10  IC CAE pilot colleges and the program continued to expand its academic  outreach to additional colleges. In 2009, a total of 17 colleges were  participating in the program and these colleges had arrangements with  academic consortia that increased the total outreach to 31 colleges.  During ODNI’s management of the program from 2005 through 2011,  ODNI established general goals and oversaw the program’s  implementation by defining and collecting performance measures on a  range of IC CAE activities and working with a contractor to summarize  this information in annual reports. We describe ODNI’s management of  the program in more detail in appendix I.

The Intelligence Authorization Act for Fiscal Year 2010 codified the IC  CAE program to authorize the Director of National Intelligence to carry out  grant programs to enhance the recruitment and retention of an ethnically  and culturally diverse IC workforce with capabilities critical to the national  security interests of the United States. In 2010, ODNI launched an  Intelligence Community Efficiency Studies Initiative that included an  examination of the size, structure, and functions of the ODNI. One  recommendation was to consolidate and streamline the education and  training programs in the IC by transferring the functions and  responsibilities of the IC CAE program from ODNI to DIA. DIA began  managing the program on October 1, 2011. The memorandum of  understanding between ODNI and DIA implementing the decision of the  transfer established that while DIA would manage the IC CAE program,  ODNI would continue to provide periodic strategic guidance and regular  budgetary oversight for program. Figure 1 shows various IC CAE program  milestones, such as grant announcements and program transition dates,  among other details.

According to ODNI and DIA officials, program management and oversight  of the IC CAE program is currently transitioning from DIA back to ODNI,  following a DIA roles and mission review in 2018. According to ODNI and  DIA officials, officials are working to complete the transition in fiscal year  2020 to enable ODNI to assume responsibility for the program. According  to ODNI officials, as of April 2019, the transition plans were still in  progress and ODNI was still in the planning stage of the transition. For  example, officials noted they were drafting an implementation plan for the  transition as well as a transfer memorandum to document the transfer.  According to DIA officials, during this process, ODNI and DIA officials  were also holding weekly coordination meetings and sharing program  documents, such as college reports collected by DIA, and program  guidance. ODNI officials also stated that in February 2019, they hired a  contractor to conduct a study of the program prior to the final transition  date. According to ODNI officials, the study, along with their interactions  with DIA, will help ODNI determine how to manage the program, identify  any challenges or successes of the program, and consolidate the data  collected on the program to date. Officials expect the study to be  completed by October 2019.

The IC CAE Senior Advisory Board consists of representatives of the IC  elements and key organizations that may include representatives from the  National Intelligence University, a U.S. Combatant Command (rotating  basis), and the Office of the Undersecretary of Defense for Intelligence.  The board, which meets quarterly, was created to provide policy and  guidance for the IC CAE program and ensure that participating IC  elements are included in discussions of policy matters. As outlined in the  board’s official charter and business rules, board members are  responsible for attending board meetings, voting on issues before the  board, evaluating colleges for grant funding, acting as points of contact  for the program, and promoting the program as leverage to affect future  IC missions. According to DIA officials, the board advises the IC CAE  program manager on standards for the IC CAE program relating to  college selection, strategies to foster collaboration, and other issues as  needed.

The IC CAE program awards grants to colleges on a competitive basis.  IC CAE grants help colleges establish new intelligence-related programs  and support existing programs at selected colleges. The grants can be  issued for up to 5 years. From fiscal years 2004 through 2018, a total of  29 colleges have received 46 IC CAE grants. Of these 29 colleges, 13  have formed a consortium with one or more colleges to enhance  collaboration with resources from other colleges in the same geographic  area. The IC considers colleges with active grants as active IC CAE  colleges, and those colleges that sustain the program after grant funding  ends are called legacy colleges. Figure 2 shows the location of IC CAE  colleges and which colleges led an academic consortium. See appendix II  for additional details on the years that grants were awarded, grant funding  amount, and a list of consortium colleges.

Since 2011, DIA has issued grants for the IC CAE program through a  process initiated by an announcement published online by the DIA grants  officer. Grant announcements vary by year, but generally include  guidelines for colleges to follow in completing their grant proposal. For  example, the 2014 grant announcement listed eight program components  a college’s proposal would be evaluated on, to include study abroad  opportunities and annual colloquium or speaker series on intelligence and  national security issues, along with other requirements such as cost  program management and sustainment plans. Following submission, a  grants officer reviews colleges’ grant proposals for technical and financial  sufficiency. The IC CAE program office then reviews grant proposals for  program sufficiency. From there, the IC CAE Senior Advisory Board’s  Source Selection Board reviews applications deemed sufficient and  makes a recommendation on which should be funded and at what funding  level. The DIA CAE program office then forwards the selected proposals  to a grant officer who notifies the college of the award.

The grant announcements we reviewed may add specific program  components as an area of focus for a specific year. For example, the  2019 grant announcement added a program component that required  colleges submitting a proposal for a grant to offer courses or programs in  three or more listed science, technology, engineering, and mathematics  topics of interest to the IC. Examples of some other program components  included in grant announcements since 2014 include the following:  IC Curriculum. A key objective of the program is to strengthen  academic programs in intelligence or national security in minority- serving, historically rural and under-resourced population colleges.  Specifically, colleges shall explain how they plan to creatively expand,  upgrade, enrich, or integrate undergraduate and graduate course  offerings to better prepare students to perform work in intelligence or  national security.

Foreign Language. Colleges should demonstrate a capability to offer  language study programs or courses in one or more specified  languages of interest to the IC.

Facilitate Student Participation in Academic Programs. IC CAE  students shall be involved in the program and aware of the numerous  benefits. Colleges are required to facilitate student participation in on- campus programs and activities such as workshops, seminars, and  other off-campus activities such as national security or intelligence  conferences, seminars, or workshops.

Annual Colloquium. IC CAE colleges are required to hold annual  colloquium or speaker series on intelligence or national security  issues. These events should invite rural and under-resourced regional  colleges and universities, government speakers, and industry partners  with a primary goal of maximizing relationships and outreach. The  colloquium should be at least 1 day in length, or a speaker series may  include shorter presentations scheduled over weeks or months, which  equates in the number of hours to a daylong colloquium.

Program Management and Sustainment Plans. IC CAE colleges  are required to have both program management and sustainment  plans. The program management plan must detail the responsibilities  of personnel to attain explicitly stated, measureable, and achievable  program objectives. The sustainment plan must detail what the  college will do during the grant period to build sustainability of the IC  CAE program at that institution after the funding expires.

The IC CAE program is especially interested in colleges with diverse  populations of talent and in geographic diversity—specifically, Historically  Black Colleges and Universities, Hispanic-Serving Institutions, Tribal  Colleges and Universities, Asian American and Pacific Islander-Serving  Institutions, and majority serving institutions with significant populations of  minorities or women. The IC CAE program is also interested in majority  serving institutions with significant populations of minorities and women  that possess credentials in disciplines and specializations that meet IC  core mission requirements. Figure 3 shows the minority designation of  the 29 colleges receiving grants and figure 4 shows the minority  designation of the 43 consortium colleges. See appendix II for a list of  schools and their minority designations.

As part of the IC CAE program, DIA also administers other programs that  provide intelligence-related learning experiences to IC CAE students and  to increase advanced capabilities in national defense. For example:  IC CAE Professional Development Summit. These annual summits  allow the IC to interact with the principal investigators—the individuals  responsible for the IC CAE program at their respective colleges—to  provide them with relevant and up-to-date information to support the  creation and teaching of IC-centric curricula. According to DIA, the  summit is intended to foster collaboration with the IC and college  representatives by providing DIA with a platform to meet the needs of  the IC. According to DIA, IC CAE Senior Advisory Board members are  an integral part of the summit and provide context and perspective  from the agencies they represent.

National Security and Analysis Intelligence Summer Seminar.  This 2-week seminar is designed to provide IC CAE students with  knowledge about the intelligence career field in general, and analytic  tradecraft in particular. The seminar is intended to provide students  from across the IC CAE colleges an opportunity to engage directly  with intelligence professionals in both seminar learning and scenario- based simulation training, focusing on threats to the U.S. homeland by  extremist terrorists. According to DIA officials, the seminar is only  open to a limited number of IC CAE students from active and legacy  colleges. For example, two sessions were held during 2017 and a  total of 80 students were competitively selected by their respective  colleges to attend. According to ODNI officials, the summer seminar  also holds a career fair and provides mentoring opportunities for the  participating students so that those interested in an IC career have an  opportunity to interact with recruiters.

IC CAE Summer Internship. In the summer of 2017, the IC CAE  program held its first IC CAE summer internship program. According  to DIA officials, rather than establish a new IC CAE internship  program, DIA leveraged the IC elements’ existing internship programs  and tracked IC CAE student participation in these programs. The IC  CAE internship offers IC CAE students additional opportunities, such  as an opening and closing ceremony for the internship, an IC career  fair at the National Security and Analysis Intelligence Summer  Seminar event, and IC mentors upon request. DIA identified a total of  141 IC interns from colleges that had an IC CAE program in 2017 and  2018. However, according to ODNI officials, not all IC interns  identified participated in their school’s IC CAE program. The  internship opportunities among the IC elements vary. For example,  according to FBI officials, their internship program is a primary  pipeline for entry-level positions and, in 2017, they had 1,200 interns  with 300 hired into entry-level positions. According to DIA data, the  FBI identified 31 IC CAE scholars in its 2017 internship program and  21 scholars in 2018. According to Department of State’s Bureau of  Intelligence and Research officials, their office has approximately 15  to 20 summer interns each year. According to Department of State  officials, two of their interns were IC CAE scholars since the program  began in 2017.

DIA Has Implemented the IC CAE Program since 2011 by Issuing Grants to Colleges but Has Not Sufficiently Planned or Overseen the Program

While DIA has continued to implement the IC CAE program by issuing  grants to colleges, DIA has not sufficiently planned or overseen the  program since the transition from ODNI in 2011. Specifically, we found  that DIA did not fully implement five of the six key practices of sound  planning that we have identified in our prior work. While DIA continued  the program’s mission to increase the pool of diverse applicants for the  IC, it lacked results-oriented goals, an overall strategy for the program, an  evaluation of external factors, performance measures, and a plan to  assess the program’s performance in order to determine the  appropriateness of the goals and effectiveness of implemented strategies.  Our assessment of the extent to which DIA incorporated these key  practices of sound strategic management planning into the IC CAE  program is reflected in table 2.

Mission Statement: DIA Maintained the Original Mission for the IC CAE Program

DIA annual reports for the IC CAE program and IC CAE grant  announcements emphasize that the overall mission of the program is to  increase the pool of diverse applicants for the IC. DIA’s annual reports  describe the program’s mission as developing national security and  intelligence education programs in order to increase the pool of culturally,  geographically, and ethnically diverse, multidisciplinary job applicants  who possess highly desired skills and competencies in areas of critical  need to the IC. This mission statement is also contained in IC CAE grant  funding opportunity announcements for 2014, 2017, 2018, and 2019,  which also refer to broader IC human capital and diversity guidance. For  example, one goal from the IC’s Equal Employment Opportunity and  Diversity Enterprise Strategy (2015-2020) is to recruit from groups with  lower than expected participation rates and diverse candidates who will  meet the IC’s current and future mission requirements.

Results-Oriented Goals and Strategies: DIA Did Not Develop Results- Oriented Goals and Strategies for the IC CAE Program

Since 2011, DIA has not established results-oriented goals for the IC CAE  program or an overall strategy that details the agency resources and  processes that are required to achieve the program’s mission. First, DIA  failed to document specific policy, programmatic, or management goals  for the IC CAE program. DIA developed a business plan for the program  in 2011; however, this plan describes short-term goals for program  management, outreach, and education and most of these goals were  intended to be complete by mid-2012. DIA’s documentation does not  indicate whether these goals were achieved or whether DIA continued to  use the goals to guide the program after 2012.

Current DIA internal guidance states that the IC CAE program office  carries out the program’s mission by providing grants to colleges to  support the establishment of intelligence-centric curricula. However, this  guidance fails to provide results-oriented goals that are defined in  measurable terms to guide the program. For example:

DIA has not described the number of potential IC employees it  expects to be able to educate or make aware of IC careers by  supporting intelligence programs at IC CAE colleges. This could  include specific goals for targeting underrepresented populations  within the IC, such as women and minorities. According to several IC  element officials, IC elements use the percentage of women and  minorities in the U.S. civilian labor force as a target for their own  diversity recruitment efforts. However, DIA has not developed any  results-oriented goals that include specific targets or milestones for  recruiting potential IC employees who have participated in the IC CAE  program.

In addition, DIA has not developed specific goals for the program that  identify how to prioritize among program requirements contained in IC  CAE grant announcements . Specifically, it is not clear from IC CAE  program documentation how gender and ethnic diversity is prioritized  relative to other IC needs, such as the IC’s long-standing need for  technical and language skills. For example, IC CAE grant  announcements state a general goal of increasing the pool of qualified  women and racial and ethnic minorities to the IC. At the same time, IC  CAE grants have supported training in science, technology,  engineering, and math, and critical languages, but DIA has not  established specific targets or milestones that would allow it to track  the program’s development of a diverse pool of applicants with the  skills that the IC requires.

Second, while DIA has developed some plans and continues to award  grants for the IC CAE program, we found that DIA has not documented  an overall strategy that details the agency resources and processes  required to achieve the program’s mission. In 2016, DIA officials stated  they began developing a document outlining the general structure of the  IC CAE program, but as of May 2019, the document has not been issued.  DIA has also documented its standard operating procedures for  monitoring colleges’ implementation of grants in part to ensure that all  programmatic goals are met, but it is not a strategic document that  describes processes for achieving the program’s mission or goals.

Further, DIA continues to award IC CAE grants to colleges based on  program components or criteria that have changed over time, but these  changes are not clearly linked to an overall program strategy. For  example, in 2014, DIA added the diversity of a college’s student  population as one of the criteria it used to select grant proposals.  Colleges with a minority-serving designation or with a student population  that is more than 75 percent ethnically and culturally diverse are given an  excellent rating, while colleges with a student population that is less than  25 percent diverse are given a poor rating. In 2017, DIA then added  criteria requiring colleges to be part of a consortium in a manner that  promotes diversity. These two diversity criteria have been given more  weight than all other criteria since 2017, while previous announcements  gave greater weight to the development of national security curricula. This  change in approach may align with the program’s overall mission to  increase diversity in the IC, but DIA has not outlined an overall strategy  that explains how such changes to the grant selection criteria would  achieve a results-oriented goal like increasing the number of minority  applicants to the IC.

Two interconnected sound planning practices are to establish results- oriented goals and strategies to achieve those goals. These goals  should be documented in measurable terms that are focused on results  so that the agency can determine how it will achieve its mission. Once  goals are established, strategies explain how these goals would be  achieved.

Since assuming responsibility for the program in 2011, DIA officials stated  that their focus for managing the IC CAE program has been tactical,  focusing on tasks like awarding, executing, and monitoring grants to IC  CAE colleges, rather than strategic planning. In addition, DIA officials  highlighted staff turnover as a challenge to managing the program and  stated DIA has had five IC CAE program directors in its 8 years of  program management.

DIA officials stated that DIA has received little guidance about the goals  of the IC CAE program from ODNI, and they instead rely on the IC CAE  Senior Advisory Board to define goals and strategies that reflect the  needs of IC elements. DIA officials stated that their only source of  guidance from ODNI for the IC CAE program was the 2011 memorandum  of understanding between DIA and ODNI, which DIA officials  characterized as being high level and lacking specificity. DIA officials also  said that they do not have the authority to create a strategic recruitment  plan or set recruiting targets for the IC. The board only meets quarterly to  advise the IC CAE program office on standards and strategies and board  members occasionally review grant proposals. The IC CAE program  managers are responsible for the program, and therefore, defining and  documenting its goals and strategies. As the IC CAE program transitions  back to ODNI, ODNI will not be able to determine whether the program is  meeting the diversity goals of the 2019 National Intelligence Strategy  without results-oriented goals for the program and a documented strategy  showing how those goals are to be achieved.

External Factors: DIA Has Identified Some External Factors Affecting the Program, but Has Not Developed a Process to Evaluate Them

DIA has identified external factors that could affect the IC CAE program’s  success, such as program branding and the ability of colleges to sustain  the program after the grant period ends, but has not developed a process  to fully evaluate them.

One example of an external factor that could affect the IC CAE program’s  success is the fact that not all students are aware of their participation in  an IC CAE program. Colleges participating in the IC CAE program have  not always featured participation in the program prominently, based on  our analysis of selected websites, which are often managed by an  academic department or institute. This can limit the visibility of the  program and the IC’s support of it for both current and potential students.  Since at least 2014, DIA has required colleges to demonstrate how they  plan to promote their program as an IC CAE program to ensure that  students, faculty, and administrators are aware of it. Colleges are also  required to feature up-to-date program information on the college’s  website. However, in November 2018, a DIA official noted that some  colleges continue to use the IC CAE brand without oversight and  accountability to provide intelligence-related courses.

According to officials from selected IC CAE colleges and IC elements,  students graduating from these programs are not always aware that they  have participated in an IC CAE program. One college official stated that  the certificates or degrees do not necessarily indicate that the student  graduated from an IC CAE program. Another college official stated the  college needs to directly inform students who apply to the program that  they are participating in an IC CAE program. NGA and NSA officials  stated that some employees at their agencies first became aware there  was an IC CAE program at their college after being informed directly by  their respective agency. While DIA requires that colleges develop  marketing plans, it does not have a process to evaluate external factors  such as the long-term effect of colleges’ efforts to advertise their  programs’ connection with the IC CAE program. Without adequately  advertising IC CAE programs, IC CAE colleges may not be able to recruit  a strong pool of qualified students with the skills that the IC requires.

Another example of external factors that could affect program success is  the ability of colleges to sustain their IC CAE program. The intent of the  IC CAE program has been to enable colleges to continue the program  beyond the end of the grant period and maintain a continuous talent pool  for the IC. However, DIA has not fully evaluated the challenges colleges  may encounter if they are not able to secure continuous funding for the IC  CAE program. When DIA awards grants, colleges are awarded a base  year of funding and renewable up to 4 additional option years. It may take  time for a college to develop intelligence-related courses and have  students graduate from the IC CAE program. Colleges then need to apply  for another grant in order to continue to receive federal funding following  expiration of any additional option years.

Since 2011, colleges have been required to demonstrate a plan to sustain  their programs after the initial grant period ends. However, according to  some IC CAE college officials, it is nonetheless difficult to continue the  program and secure external funding once the grant is over. Some  college officials have also said that the loss of grant funding can result in  colleges discontinuing key aspects of the IC CAE program and can limit  consortium college participation in activities. We have also observed that  some colleges may have suspended their programs entirely. Specifically:

Colleges may be able to sustain some, but not all components of their  program once grant funding ends. For example, one college has  sustained an IC CAE program since 2005 even though the college did  not receive grant funding from 2008 through 2012. According to  college officials, loss of grant funding resulted in the college  suspending professional development activities. The program  received additional IC CAE grants in 2012 and in 2017 and college  officials stated they hold professional development workshops and  one-on-one mentoring sessions between students and  representatives from IC elements.

Without grant funding, consortium colleges may not have funding for  student travel to IC CAE events. Consortium colleges face a specific  challenge since many of the IC CAE events are hosted by the lead IC  CAE college. We spoke with faculty at two consortium colleges who  said that grant funding from the program helps reduce the cost of their  students’ travel to off-campus IC CAE events, such as annual  colloquiums at the lead college that are attended by subject matter  experts from IC elements. The distance students may need to travel  can be especially challenging for colleges that are not located near  the lead college, including one community college that is 400 miles  away from the lead consortium college according to an IC CAE  college official.

DIA identifies some programs as legacy colleges, but some colleges  have not updated their IC CAE program websites. For example, we  reviewed the IC CAE program websites for two colleges that had  received a grant from DIA after 2012, but the colleges had stopped  updating their websites in 2014 and 2016.

DIA has identified sustainment of the IC CAE program following  termination of grant funding at colleges as a significant challenge. At a  recent meeting with the IC CAE Program’s Senior Advisory Board, the  head of DIA’s program office stated that the sustainment of IC CAE  programs after federal funding ends tended to be a systematic failure,  especially for many smaller colleges that may lack the resources of larger  colleges, and that there have been no consequences for failure. While  DIA acknowledges this problem, it does not have a process to  systematically evaluate this issue or consider alternative approaches for  colleges that may need additional support to maintain relevant curricula or  professional development activities. For example, DIA has not evaluated  whether some colleges’ difficulty with sustaining their IC CAE program  may invalidate underlying assumptions about how the program is  structured, including whether awarding grants to colleges to develop and  maintain an intelligence-focused curriculum is the most effective means of  establishing long-term relationships with those colleges and fostering a  diverse talent pool for the IC.

A key practice of sound planning is to fully evaluate key factors external  to the organization that are beyond its control. IC CAE colleges decide  how to brand the program as well as how to allocate resources in order to  sustain their IC CAE program. These decisions could significantly affect  the achievement of the IC CAE program’s mission and goals.

Both ODNI and DIA officials are aware of some external factors that could  affect the success of the IC CAE program, such as branding and program  sustainment. As of March 2019, ODNI officials have stated that they are  developing plans to address branding and sustainment as the program  transitions to ODNI. DIA drafted a plan for post-grant requirements for  colleges in order to maintain their IC CAE designation, though this draft  plan does not address the sustainment challenges that may make it  difficult for those colleges to follow these additional requirements.  However, DIA internal guidance and the most recent Senior Advisory  Board charter do not outline a process to identify and continuously  evaluate external factors that could affect program performance. As the  new program manager, ODNI may be unable to assess whether factors  like program branding or sustainment might affect the IC CAE program’s  implementation and potential for success without a process in place to  evaluate the effect of these and other potential external factors.

Metrics to Gauge Progress: DIA Has Not Defined, Collected, or Reported Comprehensive Performance Measures

DIA lacks comprehensive performance measures for the IC CAE program  that would allow DIA to measure program success. Specifically, DIA has  not (1) clearly and consistently defined performance measures to be  reported and collected, (2) collected on or reported complete information  on the program, and (3) determined whether data collected may be  incomplete or unreliable due to reporting challenges.

Performance Measures Are Not Clearly and Consistently Defined

DIA has not clearly and consistently defined the performance measures  that need to be reported by the colleges in order to determine the IC CAE  program’s success. DIA required colleges to provide reports on significant  accomplishments related to the objectives in their grant proposals.  However, we reviewed final grant reports that colleges submitted to DIA  from 2014 to 2018 that revealed differences in how colleges reported  measures. For example:

Two colleges reported that a total of 664 students received an IC CAE  certificate, 99 completed an internship, and 128 received a conditional  offer of employment between 2012 and 2017. However, the report  did not indicate whether these offers of employment were from IC  elements or the number actually hired.

A legacy college reported that 49 students received a conditional offer  of employment or were hired by an IC element, but it did not indicate  the total number of program participants.

The final report from a legacy college that had an IC CAE program  from 2013 to 2015 reported the total number of internships, but it did  not report conditional offers of employment or total program  participation.

In 2017, DIA revised the reporting template for colleges to require  progress on the goals and objectives in the approved grant proposal.  However, the information colleges provided varies because DIA’s  performance measures are not clearly stated so that colleges can report  them consistently, and they are not scoped to evaluate specific program  outcomes. For example, IC CAE programs are required to report their  progress in developing critical language studies, but there is no minimum  requirement on the type of information that a college should report in the  updated template. Comparing the reporting template for two colleges from  2018, one college’s narrative provided a high level overview of its foreign  language options at the college and reported that IC CAE scholars will be  encouraged to participate in the language courses, whereas another  college’s narrative provided details on the number of students  participating in the foreign language program and details on stipends  provided to students who studied abroad.

DIA’s updated reporting template also required IC CAE colleges to report  the aggregated totals of IC CAE participants, conditional offers of  employment, internships, and hires into the IC. However, some colleges  track different types of information for these metrics. For example, the  way colleges count student participants in the IC CAE program varies.  Some colleges only track students enrolled in the IC CAE certificate or  degree program, while other colleges report much larger totals of  participants, including those who are not enrolled in an IC CAE certificate  or degree program but may participate in some IC CAE events.

In addition, DIA’s updated reporting template did not clearly describe the  hiring data that colleges are required to report. For example, colleges are  required to report the total number of conditional offers that IC CAE  scholars receive, but it does not specify whether this number is for all  employers or just IC elements. Furthermore, it is not clear whether  students that received a conditional job offer in one semester are being  reported again as a hire in the following semester. Without clearly defined  performance measures, decision makers may not be able to clearly  identify the accomplishments of the program among the various  participating colleges.

DIA Has Not Collected or Reported Complete Information on the IC CAE Program’s Performance

DIA is responsible for reporting on the IC CAE program’s performance to  ODNI, but DIA has not collected complete performance measures that  cover the entire program and has not reported a complete summary of  the performance measures it has collected. Since 2011, the DIA  program office has collected some information from IC CAE colleges in  order to monitor compliance with the colleges’ grant proposals. This  information was reported by IC CAE colleges in their interim and final  reports that include narrative descriptions of IC CAE program activities  and descriptive data about program participants.

However, DIA has not collected complete information that captured  relevant performance measures for the IC CAE program. For example,  between 2011 and 2016, DIA officials stated colleges provided DIA a  spreadsheet of information on IC CAE program activities, including  descriptions of IC CAE courses and events, study abroad program  participation, IC element interaction, and information about individual IC  CAE scholars. However, the data provided by the colleges varied. For  example, based on a review of spreadsheets that DIA provided from the  fall of 2014, some colleges provided details on IC CAE sponsored events,  IC element interaction, and student employment, while other colleges did  not provide any information in these areas. We also found that colleges  summarized this information in their final grant.

DIA’s annual reports to ODNI from 2012 to 2017 reported little of the  information that DIA collected over this time period. The annual reports  described financial data and provided some description of select college  activities, but they did not summarize information related to any of the  program’s core requirements such as curriculum development, critical  language study, or professional development. For example, DIA has not  collected or reported data on the number of IC CAE scholars who have  studied a critical language from 2012 to 2017. The reports also did not  include the total number of IC internships, conditional job offers, or hires  after 2012.

Moreover, college officials stated they do not report on performance  measures after the grant period ends, which may limit DIA’s ability to  provide comprehensive data for both active grant colleges and legacy  colleges each year. DIA officials stated that legacy IC CAE colleges that  have sustained the program but no longer receive a federal grant are not  obligated to provide reports to DIA. According to DIA officials, DIA is  currently developing a plan that would require colleges to report  information in order to maintain their IC CAE designation after the grant  period ends. For example, a college official from a legacy program that  first received a grant in 2006 stated that the college no longer shares  information with DIA because DIA had not requested it do so after the  grant ended. The official noted that the college is no longer receiving  support to facilitate IC recruitment of its students.

Data Collected May Be Incomplete or Unreliable Due to Reporting Challenges

DIA officials stated they have relied on colleges rather than the IC  elements themselves to report data on IC CAE scholars. DIA informs  colleges through its reporting template that data on internships,  conditional job offers, and hires into the IC are definitive evidence of the  success and sustainability of a college’s IC CAE program. However, due  to challenges with collecting these data, the information being provided to  DIA by the colleges may be incomplete and unreliable.

While DIA has not reported on the total number of IC CAE scholars that  have been hired from 2012 to 2017, it has collected some information  from IC CAE colleges. For example, three colleges from our sample  reported that a total of 23 IC CAE scholars were hired by the IC between  the beginning of the fall semester of 2017 and the end of the fall semester  in 2018. However, according to officials at these colleges, it is difficult to  provide complete data on students’ employment as they no longer have  direct contact with students after they graduate and some IC elements  discourage applicants from discussing their employment offers with  others. As a result, the information the colleges report to DIA may be  incomplete because they are not able to track all the students who have  graduated from the IC CAE program. ODNI also reported similar  challenges when it managed the program from 2005 through 2011. ODNI  reported a total of 61 IC CAE scholars were hired into the IC between  2005 and 2011 based on IC CAE college data, but noted that the hiring  data from IC elements was higher than the total reported by colleges.

Further, IC elements have noted that there are security risks associated  with tracking the number of IC CAE scholars that receive a conditional  offer of employment or have been hired into the IC. At the February 2019  IC CAE professional development summit, for example, Senior Advisory  Board members from the CIA and the FBI advised IC CAE colleges that  storing or sharing information about potential IC applicants on unsecured  college systems is a security risk. Some IC element officials have  suggested that the best way to track applicants would be to obtain a list of  IC CAE scholars from the colleges and match the names against IC  element applicants. However, according to officials, the IC elements  would need an individual’s full legal name and college, and some IC CAE  college officials raised privacy concerns with sharing student information.  An IC CAE college official stated that even during the grant period, the  college only provided DIA aggregated totals on student data because of  privacy concerns.

DIA and ODNI have collected some data on the number of applicants  from IC CAE colleges and new hires from the IC elements, but they have  only recently done so in a systematic manner. Officials from DIA’s IC CAE  program office said they cannot force IC elements to report employment  information and that the burden is on the IC elements to track and report  that data. According to ODNI officials, in response to a provision in the  Intelligence Authorization Act for Fiscal Year 2017, ODNI sent out a  request to IC elements for data on hiring and demographic information  that included questions about the number of IC CAE graduates hired by  the IC. As of April 2019, officials stated that they have collected hiring and  demographic information from six of the largest IC elements that includes  data about the number of IC CAE graduates hired by the IC. The officials  said they expect this to be a large enough sample to report in June 2019.  However, according to ODNI officials, ODNI has not yet determined how  it will define performance measures for the IC CAE program or how it will  continue to collect and report these performance measures.

A key practice of sound planning requires the development of a set of  performance measures that will be applied to gauge progress toward  attainment of the plan’s goals. We have also established that key  attributes of successful performance measures, which include measures  that cover core program activities, are that they are clearly defined and  consistent and can be reliably produced. Furthermore, Standards for  Internal Control in the Federal Government state that management  should use relevant data from reliable sources; process this data into  high-quality information that is complete, accurate, and valid; and  communicate high-quality information to all levels of the department.

Comprehensive performance measures would allow DIA to gauge the  success of the IC CAE program in developing a pool of diverse talent with  skills needed in the IC, but DIA has not defined performance measures in  program guidance and documentation. In its 2012 annual report, DIA  stated that it intended to redesign ODNI’s data collection tool in order to  simplify reporting. However, DIA did not report data collected with this tool  and stopped collecting these data altogether in 2016 after informing IC  CAE colleges that the collection effort required a lengthy approval  process from the Office of Management and Budget. DIA officials  continued to require colleges to report performance measures after 2016  through a reporting template. In April 2019 DIA officials stated that they  intended to make improvements to this template given that the way  colleges have tracked student participation has varied. However, DIA did  not clearly and consistently define performance measures for all aspects  of the program, process them via a data system or spreadsheet, or report  them to ODNI. As the new IC CAE program manager, ODNI will not be  able to gauge the success of the IC CAE program in achieving its mission  without defining, collecting, and reporting on comprehensive performance  measures.

Program Evaluation: DIA Has Not Comprehensively Assessed the Program’s Performance

Since 2012, DIA has not conducted a comprehensive assessment of the  IC CAE program. According to a 2013 amendment to the memorandum of  understanding with ODNI, DIA was responsible for providing ODNI with  an annual review of the program’s performance and including possible  outcomes, such as specific benefits to the IC. ODNI was responsible for  evaluating this information to ensure the appropriate and efficient  expenditure of IC resources and performance improvement.

However, DIA’s annual reports to ODNI from 2012 to 2017 did not  comprehensively assess the program’s performance or the extent to  which the program had achieved its mission. These reports only provide a  few details about IC CAE program activities and summarize grant  expenditures. For example, the 2016 annual report for the IC CAE  program provided information on the number of grants awarded, a list of  IC CAE colleges participating in the program, funding and execution data,  and a sample of IC CAE program events from three colleges. However,  the report did not provide complete details on the status of the program at  each IC CAE college, such as a summary of the performance metrics it  had collected from all of the colleges with an active grant. DIA officials  said that they only included the information in annual reports that ODNI  requested in the memorandum of understanding and lacked resources to  provide a comprehensive assessment. However, the memorandum of  understanding requires DIA to provide an annual review of the IC CAE  program’s performance to possibly include outcomes such as the number  of students who completed IC CAE coursework and specific benefits to  the IC.

As ODNI officials work with DIA to transition the IC CAE program back to  ODNI, ODNI officials began working with the MITRE Corporation in  February 2019 to evaluate the IC CAE program. ODNI officials said they  will rely on MITRE’s findings and their own interactions with IC CAE  colleges to determine how to manage the program. Officials stated they  expect the evaluation to be complete by October 2019. However, ODNI  has not yet developed a plan to conduct continuous and comprehensive  assessments of the IC CAE program.

A key practice of sound strategic planning is the use of assessments,  through objective measurement and systematic analysis. For example,  an evaluation plan can assist an agency in determining the  appropriateness of a program’s goals, the effectiveness of implemented  strategies, and the potential need for corrective action.

The memorandum of understanding between DIA and ODNI in 2011 and  amended in 2013, designated performance reporting as a DIA  responsibility, but DIA did not identify performance assessment as a  responsibility in program guidance. The IC CAE program office’s standard  operating procedures provide that the grant officer’s representative  monitors an IC CAE college’s compliance with its grant assistance  agreement and collects performance and financial data reports. However,  there is no mention of a systematic, outcomes-based assessment of  these reports or the program as a whole. Without such assessments, the  IC will not be able to determine whether the IC CAE program is effectively  increasing the pool of diverse applicants. Congress will also be unable to  determine the return on investment in this long-standing program.

Selected IC Elements Participate in the IC CAE Program to Varying Degrees, but DIA Has Not Assessed Program Participation and Roles Are Not Clearly Defined

Selected IC Elements Participate in the IC CAE Program to Varying Degrees, but DIA Has Not Assessed IC Element Participation in the Program

IC elements participate in the IC CAE program in a variety of ways,  including by attending IC CAE college workshops and recruitment events  and participating in the annual IC CAE program meeting, among other  events. Table 3 shows the varying levels of participation in the IC CAE  program among the eight selected IC elements, as reported by IC  element officials.

IC elements’ participation in the IC CAE program varies according to the  specific organizational needs of each IC element. Some IC elements do  not participate actively in the program because they do not directly hire  employees into their intelligence office or because they conduct only  limited hiring. For example, according to officials from the Department of  Energy’s Office of Intelligence and Counterintelligence, the office is small  and hiring is therefore limited. Further, officials stated the office often  hires specialized personnel with advanced degrees and would not hire IC  CAE scholars from undergraduate programs. Similarly, State Department  officials from the Bureau of Intelligence and Research stated that they do  not participate in events since they do not have direct hiring authority.  Further, these officials stated that the State Department’s participation in  IC CAE events is also constrained by limited personnel and financial  resources.

Other IC elements, such as the CIA and the NSA, have developed  separate relationships with colleges and programs to address their  specific hiring needs.

CIA. The CIA has reduced its involvement with the IC CAE program to  better align its needs according to CIA officials. In 2009, CIA selected  senior officers to serve as advisors to 16 IC CAE colleges. The CIA  advisors were directed to make a minimum of two visits per year and  conducted a wide range of activities to include presenting at colleges  events, counseling IC CAE scholars, and discussing CIA and IC  career opportunities. However, about a third of the advisors were  pulled back because, according to CIA officials, the IC CAE colleges  were not meeting hiring expectations. Since 2014, CIA has focused its  efforts on only six of the IC CAE colleges based on the return on  investment from these colleges and alignment with CIA hiring needs.  In addition, according to CIA officials, CIA has designated five  universities as signature colleges to recruit skilled applicants from a  range of cultures and backgrounds. According to CIA officials, the  signature college program targets large, diverse colleges where the  CIA has received a significant number of applications. Its criteria for  selection of signature colleges include high diversity, the size of the  college, and potential for developing a deep relationship. Two of the  five CIA signature colleges are also in the IC CAE program and are  currently receiving or have received grant funding.

NSA. According to NSA officials, NSA has been involved in the IC  CAE program since its inception, and its involvement includes  participating in a variety of events such as colloquium, summer  seminars, and recruitment events. In addition, NSA has also  sponsored two types of Centers of Academic Excellence, one for  cyber defense and one for cyber operations. The goal of these  programs is to develop technical skills by promoting higher education  and research in cyber defense and producing professionals with cyber  defense expertise. In addition, the programs also aim to broaden the  pool of skilled workers capable of supporting a cyber-secure nation.  The programs involve awarding a designation as a Center of  Academic Excellence in Cyber Defense or Cyber Operations to U.S.  universities based on criteria. No funding is provided to the U.S.  universities. According to NSA officials, these programs are  independent of the IC CAE program and have different goals from the  IC CAE program. Officials stated NSA’s CAE programs are focused  specifically on increasing the pipeline of cyber talent.

Further, some IC elements’ recruitment strategies incorporate the IC CAE  program as part of their strategy, but it is not the only aspect of the  elements’ approach to recruiting. For example, according to NGA’s  Campus Recruitment Strategy, the agency targets high-quality colleges  that provide access to diverse applicants in high-quality, mission-aligned  degree programs across a broader geographic reach. The strategy has  31 designated colleges that were selected based on a variety of criteria,  including demographic diversity and academic programs that align with  the agency’s mission areas. According to NGA officials, they continue to  recruit from at least seven IC CAE colleges; however, being an IC CAE  college was not part of the primary selection criteria for colleges in NGA’s  campus recruitment strategy.

As program manager, DIA has relied on the IC CAE Senior Advisory  Board and its charter as a means to engage IC elements in the program.  However, not all IC elements participate on the Senior Advisory Board or  in the IC CAE program. For example, in the  November 2017 board  meeting, only 9 of the 17 elements attended the meeting and a quorum  was not established. Without a quorum, votes held during a meeting are  not valid and actions cannot be approved. Moreover, during board  meetings, members have raised concerns about limited attendance, citing  concerns that only about half of the members regularly attend.  According to some IC element officials, they do not attend IC CAE  program events, including the Senior Advisory Board meetings, because  the program does not meet their IC element’s organizational needs. For  example, as discussed above, some IC elements have developed  separate relationships with colleges not in the IC CAE program. Further,  as discussed above, some IC elements have developed separate  relationships with colleges and programs to address their specific hiring  needs. As a result, some IC element officials have stated they have  intentionally reduced their recruitment at some IC CAE colleges.

Since not all IC elements participate in the IC CAE program or attend the  board meetings, DIA has had to conduct other outreach to engage IC  elements. According to DIA officials, since 2017 the IC CAE program  office has conducted additional ad hoc outreach to engage with IC  elements. For example, DIA officials have stated the IC CAE program  office has utilized ODNI forums, such as the IC Recruitment Council and  IC Chief Human Capital Office Council to engage with IC elements on the  IC CAE program. However, DIA officials also stated that not all IC  elements attend these ODNI council meetings because different offices  within the IC elements are responsible for attending the meetings. Some  IC elements are set up differently with regard to which office within the IC  element participates in the IC Recruitment Council, so the IC element  representatives to the IC CAE Board can differ from those who attend the  IC Recruitment Council. While these ad hoc outreach efforts are likely a  positive step to improving coordination, there remains a lack of  engagement by all IC elements.

Standards for Internal Control in the Federal Government state that  management should establish and operate monitoring activities, to  include a determination of when to revise the program baseline to  address program needs. Further, the standards state that management  should evaluate and document the results of ongoing monitoring and  separate evaluations to identify issues. As program manager, DIA has not  established a process for monitoring and assessing IC elements’  participation in the IC CAE program, and the board’s charter does not  describe such a process. As result, DIA does not fully understand the  reasons for the lack of engagement on the part of IC elements. IC  elements that do not attend board meetings are not engaged in the  discussions and decisions being made about the program. Similarly, IC  elements that do not participate actively in the program have limited  contact and interaction with IC CAE colleges, which has hampered the  effectiveness of the IC CAE program. Without a process for monitoring  and assessing IC elements’ participation in the IC CAE program, ODNI  will not be able to tailor the program to meet the needs of the IC and  address the overall program goal of creating a diverse pool of applicants  for the IC. Assessing and addressing IC elements’ reasons for not  participating in the program would increase ODNI’s understanding of the  factors that inhibit participation and inform an approach to mitigating  these factors and achieving program goals.

IC Elements’ Roles in the IC CAE Program Are Not Clearly Defined

The IC CAE program is a collaborative effort that allows IC elements to  participate at college events, such as colloquia, speaker series, and  campus recruitment events. The IC CAE Senior Advisory Board was  created to provide policy and guidance for the IC CAE program and  ensure that participating IC elements are included in decisions related to  policy matters. The board’s charter states the Senior Advisory Board  members are responsible for attending board meetings, voting on issues  before the board, acting as points of contact for the program, and  promoting the program.

However, the charter does not define the expected or required level of  participation of IC elements at IC CAE colleges. The IC CAE program  manager, DIA, has communicated the schedule of IC CAE college events  during Senior Advisory Board meetings and also asked for IC elements to  participate in various events. Through the IC CAE grant process, IC CAE  colleges are required to host a variety of events to educate IC CAE  colleges about the IC. Based on the IC CAE grant announcements, these  events are predicated on IC element participation. Specifically,  recruitment fairs at colleges are facilitated by IC elements and IC element  officials are speakers at colloquia events, with a primary goal of  maximizing relationships and outreach. However, some colleges have  experienced challenges with engaging with IC elements to attend these  events. For example:

An official from a legacy IC CAE college noted that it has been difficult  to get IC elements to attend college events or recruit from the college.  The official stated that IC element participation has been ad hoc and  based on personal relationships with the IC elements rather than  assistance from the IC CAE program office. For example, the official  noted that at recent events the college was only able to attract 8 IC  elements to a recruiting event compared to the 20 representatives  across 12 IC elements who attended the events in the past.

An official from an active IC CAE college also noted that some IC  elements are not well informed about the IC CAE program. For  example, the official noted that the college would like more IC  elements to attend IC CAE college events. However, the official stated  that the responsibility of developing relationships with IC elements has  been placed on the college. According to the official, the IC elements  should be more aware of which colleges have IC CAE programs and  should be the first stop for IC element recruitment. The official also  stated IC CAE colleges would like the IC element to drive the  relationships with colleges.

Our leading collaboration practices include (1) having participating  agencies clarify roles and responsibilities and (2) ensuring that  participating agencies document how they are collaborating in a written  agreement and develop ways to continuously update and monitor these  agreements. Roles and responsibilities can be defined through laws,  policies, memorandums of understanding, or other requirements.

The IC has defined the mission for the IC CAE program, but the current  program manager, DIA, has not clarified IC element roles and  responsibilities for program participation and the Senior Advisory Board  charter does not clarify what is expected of the IC elements regarding  participation at IC CAE events. According to DIA officials currently  managing the program, the Senior Advisory Board charter is the key to  getting IC element participation in the program and overall program  success. An update to the Senior Advisory Board charter could include all  relevant participants and define roles and responsibilities. Without clearly  defined roles and responsibilities, the IC elements are not taking full  advantage of what the IC CAE program has to offer, including  participation in events and college engagement. Thus, the IC CAE  colleges will not be able to fully execute their IC CAE programs and the  program may not be able to meet its goal of creating a pool of diverse  applicants for the IC.

In 2005, ODNI established the IC CAE program with a goal of creating an  increased pool of culturally and ethnically diverse, multi-disciplinary job  applicants for the IC. However, the current program manager, DIA, has  not sufficiently planned and overseen the program and the IC is unable to  determine whether the program has been successful in meeting its goal  to create an increased pool of culturally and ethnically diverse job  applicants for the IC. Specifically, DIA has not developed results-oriented  goals or documented an overall strategy for the program, evaluated  external factors that could significantly affect the program’s success,  defined and collected comprehensive metrics, or conducted an  assessment of the program’s performance. As ODNI takes over the  program, it needs to address these sound planning practices in order to  determine whether the program is being implemented successfully and to  help ensure the IC has a trusted, diverse workforce with the right  expertise. Further, without sufficient planning and oversight, decision  makers will also be unable to determine the return on investment in this  long-standing program.

In addition, ODNI also needs to improve IC element participation in the  program. The IC CAE program is a collaborative effort that encourages  participation among all IC elements. However, DIA has not established a  process to monitor and assess IC element participation in the program or  clearly defined IC elements roles and responsibilities for the IC CAE  program. A process for monitoring and assessing IC element participation  and addressing IC elements’ reasons for not participating in the program  will increase understanding of the factors that inhibit participation and  inform ODNI’s approach to mitigating these factors and achieving its goal  for the program. Further, without clearly defined roles for IC element  participation in the program, IC CAE colleges may not be most effectively  executing their IC CAE programs and the program overall may not be  able to meet its goals.

We are making the following seven recommendations to the Director of  National Intelligence as the IC CAE program transitions to ODNI:  The Director of National Intelligence should establish and document  results-oriented goals that include specific targets or milestones for the IC  CAE program. (Recommendation 1)

The Director of National Intelligence should establish and document  strategies to achieve the results-oriented goals that are established for  the IC CAE program. (Recommendation 2)

The Director of National Intelligence should develop and document a  process to identify and continuously evaluate external factors that could  affect the program’s ability to achieve identified goals. This should  include, but not be limited to, a consideration of program branding and  post-grant sustainment. (Recommendation 3)

The Director of National Intelligence should define and document  comprehensive performance measures for the IC CAE program, collect  and evaluate the completeness and reliability of information it receives  from grant recipients and IC elements, and report this information on a  regular basis. (Recommendation 4)

The Director of National Intelligence should establish a requirement for  and develop a plan to periodically evaluate the IC CAE program’s  performance through objective measurement and systematic analysis.  (Recommendation 5)

The Director of National Intelligence should develop a process for  assessing why some IC elements are not participating in the IC CAE  program and address these reasons in order to ensure the program is  structured to meet the needs of IC elements. (Recommendation 6)

The Director of National Intelligence should clearly define IC elements’  roles and responsibilities for participation in the IC CAE program to better  facilitate interagency collaboration in support of the program.  (Recommendation 7)

We provided a draft of this report to ODNI for review and comment. In  written comments, ODNI concurred with all seven of our  recommendations but did not identify the steps it plans to take to address  the recommendations as the IC CAE program transitions to ODNI.  ODNI’s comments are reprinted in their entirety in appendix III. ODNI also  provided technical comments, which we incorporated as appropriate.

We also provided a draft of this report to the CIA, Department of Defense,  DIA, FBI, NGA, NRO, NSA, the Department of State’s Bureau of  Intelligence and Research, and the Department of Energy’s Office of  Intelligence and Counterintelligence for review and comment. These  agencies concurred without providing comments on the draft report. NGA  provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees. We are also sending copies to the Secretaries of Defense,  Energy, and State; the Directors of National Intelligence, DIA, CIA, NGA,  NRO, and NSA; and the Attorney General. In addition, this report will be  available at no charge on our website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please  contact Brian M. Mazanec at (202) 512-5130 or mazanecb@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  key contributions to this report are listed in appendix IV.

Appendix I: History of the Intelligence Community Centers for Academic Excellence Program from 2005 to 2011

The Office of the Director of National Intelligence (ODNI) was the  Intelligence Community (IC) Centers for Academic Excellence (CAE)  program manager from 2005 through 2011. Internal documents and  grant announcements from that period state that the program’s mission  was to increase the pool of eligible applicants in core skills areas,  specifically targeting women, racial and ethnic minorities and individuals  with varied cultural backgrounds, regional expertise, and language  proficiency. ODNI outlined four goals in its 2008 guidance for the  program, including a focus on developing relationships with colleges,  providing resources and grants to competitively selected colleges,  providing technical assistance in the design and implementation of  colleges’ IC CAE programs, and documenting results to improve the  efficacy of the IC CAE program. Each of these goals included supporting  objectives. For example, the goal of providing support, resources, and  grants to competitively selected colleges included four supporting  objectives, such as instituting long-term practices to increase  relationships with minority-serving institutions and providing access to IC  internships, co-ops, and graduate fellowships. These goals and objectives  were aligned with the program’s overall mission, but they were not  defined in measurable terms that would allow future assessments of  whether they were being achieved. For example, ODNI did not establish  targets for the goals or supporting objectives listed above that would have  allowed it to determine how successful it had been at supporting long- term programs at minority-serving institutions or providing access to IC  employment opportunities.

In addition, ODNI defined a strategy to support its program goals, and the  strategy included the following four elements: outreach to high schools;  operations at colleges, including curriculum development; infrastructure at  the colleges to support these operations such as faculty and  administrators; and relationships between IC CAE programs and IC  elements. These elements of ODNI’s strategy described specific  operational requirements for the program. For example, IC CAE grant  announcements in 2006, 2009, and 2011 supported a wide range of  academic activities that prioritized the development of curricula in national  security studies, science and technology programs, study abroad  programs, courses in critical languages, and pre-collegiate outreach  through activities like summer camps to raise awareness and interest in  IC careers.

ODNI also defined assessment and evaluation as an overarching part of  the program’s strategy, as shown in figure 5. ODNI worked with a  contractor to conduct annual performance evaluations through 2012. The  contractor developed an evaluation methodology and reviewed colleges’  interim reports, collected and verified performance data, and developed  findings and recommendations. For example, the contractor  recommended that IC CAE colleges broaden their critical language  offerings and increase the number of IC CAE Scholars enrolled in foreign  languages courses in each of the annual reports from 2007 to 2010.

ODNI defined performance measures and reported data on activities,  including the number of IC CAE courses and events, demographic  information, and employment outcomes. Specifically, IC CAE colleges  were required to report these data quarterly, and the contractor compiled  the data annually into its program reviews. Table 4 shows selected  performance measures outlined in ODNI’s final report that summarized  information collected from 2004 through 2011.

Appendix II: List of Intelligence Community Centers for Academic Excellence Grants to Colleges and Minority Designation

Table 5 and table 6 show the 46 grants managed by Office of the Director  of National Intelligence (ODNI) and the Defense Intelligence Agency  (DIA). The total amount of grant funding projected to be obligated from  fiscal year 2005 and fiscal year 2021 is $69,053,618, not including a  $250,000 contract in September 2004 to initiate a pilot Intelligence  Community (IC) Centers for Academic Excellence (CAE) program at  Trinity Washington University. Tables 7 and 8 list the IC CAE colleges by  designation of eligibility for Department of Education funding as a minority  serving institution under various statutory grant programs including  programs authorized by the Higher Education Act of 1965, as amended.  Eligibility for grant funding under these statutory programs as determined  by the Department of Education in 2018 does not designate or certify any  college as a particular type of institution, for example, as a Hispanic  Serving Institution.

The colleges listed in tables 5 and 6 are listed in the order that they  received a grant by fiscal year and some IC CAE colleges received  multiple grants. Grants fund a base year and up to 4 additional option  years. The consortium colleges below are listed alongside the IC CAE  college that received a grant. ODNI and DIA awarded IC CAE grants to  colleges following an announcement for proposals in fiscal years 2006,  2009, 2011, 2014, 2017, 2018, and 2019.

Appendix III: Comments from the Office of the Director of National Intelligence

In addition to the individual named above, Kristy Williams, Assistant  Director; Jason Bair; Tracy Barnes; John Bumgarner; Meeta Engle; Gina  Hoover; Amie Lesser; Benjamin Licht; Ned Malone; Parke Nicholson;  Alice Paszel; Sarah Veale; and Lillian Yob made key contributions to this  report.

The Centers for Academic Excellence is a federal grant program designed to encourage highly-qualified college students with diverse backgrounds to pursue careers at U.S. Intelligence Community entities like the CIA. From 2005-2018, 29 colleges received 46 grants totaling about $69 million.

The Intelligence Community doesn't know if the program is achieving its goal of increasing diversity in its applicant pool. In addition, Intelligence Community entities are unclear about their responsibilities in the program and need to improve their participation in the program.

We made 7 recommendations to address these and other issues.