text:
I. INTRODUCTION Background The Equal Employment Opportunity Commission (EEOC) established a National Contact Center (NCC) to accomplish three objectives: upgrade its customer service based on 21st Century technology, improve human capital effectiveness, and deliver accurate and consistent service to its customers. In February 2003, the National Academy of Public Administration (NAPA) submitted a report recommending, among other things, that the EEOC establish a national call center. In June 2003, an internal EEOC work group completed a review of how the EEOC handles calls and issued a report that recommended implementation of a contact center - the Assessment of a National Contact Center Solution for EEOC (2003 Assessment Report). 2003 Assessment Report Conclusions and Proposed Solution. The 2003 Assessment Report concluded that implementing a contact center would be instrumental in resolving concerns about system capacity to effectively handle unsolicited calls and in ensuring consistent and accurate customer service across all EEOC offices. A contact center was seen as an option that would benefit all EEOC offices rather than hiring staff, which would benefit only a few offices. In addition, a contact center addresses requirements in the President's Management Agenda and the E-Government Act of 2002. NCC Contract. In September 2004, the EEOC contracted with NCS Pearson, Inc. (Pearson) to develop, implement, and operate the NCC as a two-year pilot project, with an option to extend the term for three additional years.[4] Contractually, the Commission can obligate funds only one year at a time. The EEOC can exercise this option by giving Pearson 60 days' advance written notice of its intention to renew the contract, and then written notice of renewal within 30 days prior to expiration. The total term of the contract, including the pilot and option, is not to exceed 60 months. The NCC implemented a program to provide the following new services: Capability for the public to communicate on first contact in more than 150 languages, increasing meaningful access to people with limited English proficiency Twelve hours of operation - 8 a.m. to 8 p.m. Eastern Standard Time (EST), longer than business hours for EEOC field offices A live person should answer calls within 30 seconds FAQs on the EEOC's website An Interactive Voice Response (IVR) system accessible to all callers 24 hours per day, 365 days per year Text Telephone (TTY) service attended by trained staff available 12 hours a day Email access that does not require knowing the name or email address of a specific EEOC employee Contract to Evaluate the NCC. In November 2003, the EEOC Commission unanimously approved implementation of a two-year pilot phase of the NCC, to include an independent audit of the NCC prior to committing to a multi-year contract. After discussions between the Inspector General and other senior executives, the EEOC transferred to the Office of Inspector General (OIG) the funding to evaluate the NCC. OIG then conducted research, formed a project team, obtained input from Agency officials and selected stakeholders, developed a statement of work, and held a competition to award the contract. In September 2005, the OIG contracted with Job Performance Systems, Inc. (JPS) to evaluate the impact of the NCC on the EEOC's staff, operations, and customers. JPS and its subcontractor Convergys conducted its evaluation from October 2005 through April 2006. The purpose of the evaluation was to provide EEOC management with information to aid in a decision on whether to exercise the option to renew the Pearson contract and inform stakeholders on how the NCC has affected the EEOC and its customers. Since the purpose of the NCC is to improve efficiency in field offices and enhance customer service, the JPS evaluation focused upon the following topics: NCC implementation and operations NCC impact on efficiency and effectiveness of EEOC operations NCC impact on field staff NCC impact on EEOC customers The EEOC expected the NCC to have a significant effect on customers and field offices, with a minimal impact on Headquarters; therefore, the JPS Team placed more emphasis on the NCC and its impact on field offices than the impact on Headquarters activities. Context for the Evaluation The 2003 Assessment Report recommended that there be at least a two-year pilot phase of operations. The first year (September 2004-2005) was to be dedicated to start-up operations, including the following: Developing accurate baseline data on the volume and nature of calls Developing and refining scripts based on actual requests for information and their frequency Developing and refining standard operating procedures and business rules Developing effective working relationships between the NCC and EEOC offices in the field The second year (September 2005-2006) was to be the time to assess vendor performance. To meet the required notices to renew the Pearson contract in September 2006, the evaluation started seven months after officially launching the NCC (March 21, 2005). This meant that the JPS Team evaluated the impact of the NCC on EEOC Headquarters and field operations while the NCC was still in its first year of actual operations and the new processes were still being implemented and refined.[5] Thus, to some degree, the evaluation was of a new program while it was still in the development phase and operations were not yet stabilized. To allow time to stabilize operations before a high call volume, the EEOC initially decided to have only the calls to the two toll-free numbers directed to the NCC rather than all unsolicited calls to field offices. Then from November 2005 through March 2006 the Office of Field Programs (OFP) gradually authorized all offices to redirect their unsolicited calls to the NCC. The NCC commenced initial pilot operations with seven field offices in February 2005. On March 21, 2005, the NCC started accepting all calls on the EEOC's general inquiry lines, which includes two national toll-free lines serving voice and TTY calls. The EEOC has not yet phased in the option to have the NCC respond to requests for all EEOC publications. Throughout the NCC pilot, OFP has made operational changes. In December 2005, they increased the time employees in the field have to respond to NCC requests from two to four days. They have also shifted responsibility for hot line calls (callers at risk of losing their rights to file a charge) from all district offices to four offices (three hours per day for each office) in different time zones across the country. In addition, the EEOC has revised scripts to include new information, and made them easier for the Customer Service Representatives (CSRs) to understand and use. After receiving results of the customer satisfaction survey, the EEOC added a closing line to emails and web inquiries advising customers to contact the NCC with any additional questions and providing the TTY number. Further, the EEOC is planning a nationwide outreach campaign in July 2006 to encourage use of the toll-free telephone numbers. The EEOC has not yet required that the NCC fulfill requests for publications or answer calls for any offices in Headquarters. In addition, the EEOC and NCC have been pilot testing the EEOC Assessment System (EAS). Once tested, the EAS will become a web-based e-government application to help on-line users determine whether the EEOC is the appropriate agency to provide assistance. The NCC is using the EAS and the related EEOC Assessment Questionnaire (EASQ) to help filter out non- jurisdictional inquiries and, when appropriate, electronically submit basic information about the inquiry to the proper EEOC office.[6] During the NCC pilot, changes have been made to the EASQ, most notably providing look-up tables based on post office data to derive the county code and thus ensure questionnaires are directed to the proper EEOC office. The amount of time it takes to launch a program at a new contact center varies by the type and size of the program. The EEOC contact center operation is considered small compared to industry standards. Similar programs may take from six to nine weeks to launch and standardize; however, this pilot is more complex because of the work required to streamline processes between the NCC and the EEOC offices. Since all EEOC field offices follow different processes, it is difficult to estimate the time it would take to streamline such activities. II. METHODOLOGY A. EEOC HEADQUARTERS AND FIELD LOCATIONS The scope of the JPS Team's work included evaluating the impact of the NCC on operations at Headquarters and in the field; however, the EEOC never anticipated that the NCC would have a significant, direct impact on Headquarters. Therefore, the team devoted most of its resources to determining the impact of the NCC on offices in the field. 1. The Search for Baseline Data and a Control Group Traditional evaluations often rely upon a combination of quantitative and qualitative data. One strategy to assess the NCC's impact is to compare changes in variables of interest before and after program implementation. The JPS Team searched for pre-NCC baseline data to evaluate empirically the impact of the NCC on EEOC operations; however, there were only limited quantitative data available, partially because offices follow different intake procedures, which made it impossible to isolate the impact of variables related to NCC activities. This led the JPS Team to draw many findings and conclusions from subject matter experts. The team gathered this information through interviews, focus groups, and survey data. Where possible, the team compared data collected from different sources to ensure consistent findings. The JPS Team was unable to identify any pre-NCC data on costs related to handling inquiries, so it was not possible to compare pre- to post-NCC costs. However, the team did provide an analysis of the current estimated impact on an Investigator Full-Time Equivalent (FTE), and projected possible future impact on Investigator FTE based upon certain assumptions. Telephone Calls. To investigate whether the JPS Team could compare telephone call data for offices pre- and post-NCC implementation, the team explored the possibility that field offices may have been using their telephone systems to automatically track calls to their offices. There is presently no systematic process in place to collect data on the number of calls made to EEOC offices. As a result, there is no automated way to track the number of calls to all or many individual offices pre- and/or post-NCC. The JPS Team did identify that two offices have tracked telephone calls on their public number since October 2004 (pre-NCC). The team evaluated the impact of the NCC on the telephone calls to those offices. The JPS Team investigated the possibility of replicating the telephone survey conducted as part of the 2003 Assessment Report in order to compare pre- and post-NCC data.[7] As part of this effort, the team inquired with members of various offices at Headquarters and determined that the field office-level electronic and raw data used to develop the conclusions and recommendations in the 2003 Assessment Report no longer exist. These data included volume and types of telephone calls as well as staff time required to answer calls to offices in the field. The team considered repeating the data collection effort for randomly selected offices to determine what changes have transpired.[8] This strategy would have been efficient and practical to compare data across a sample of offices given the available time and resources, but the EEOC and JPS Team could not find the raw telephone data collected in 2003. The team made searches at Headquarters, at field sites, and with the data entry vendor. With no office-level baseline data, it was impractical to replicate this study for selected offices.[9] Lacking historical data across offices, the team included some items in the electronic survey sent to all employees located in the field, asking them to estimate the average number of calls they received pre- and post-NCC implementation. The drawback is that these data are only as reliable as the survey respondents' memory. Productivity Measures. The JPS Team also investigated evaluating traditional performance metrics such as productivity, merit factor, and cause rate, but these data were not good measures to use in the evaluation because of confounds[10] over the past several years that have affected many offices. For example, many offices have experienced attrition (staff have left and not been replaced) and/or undertaken changes in intake procedures. The JPS Team specifically evaluated merit factor pre- and post-NCC, as it has long been an important EEOC measure of positive resolutions, cause finding, withdrawals, mediations, and conciliations. To use merit factor as a relevant measure, one must assume that all of the following occur: the NCC frees up Investigator time; Investigators spend that time conducting more thorough and higher quality investigations; and those investigations lead to increased benefits. OFP indicated that merit factor is relevant given the current environment, in which Investigators are managing a heavy workload. The team evaluated the trend of discrete quarterly merit factor for all 51 offices from October 1, 2003 through December 31, 2005.[11] For the same period, the team also evaluated the merit factor for only those offices indicating that the NCC had enabled them to redirect Investigator resources to spending more time on investigations. The JPS Team could not identify any change in the merit factor other than a trend toward a very small increase across time. It is possible that, because the analysis included only nine months of NCC operations, there has not been sufficient time to realize an impact on merit factor. Integrated Mission System (IMS) Data. The JPS Team initially considered a comparison between inquiries generated from the NCC and those from all other sources reported in the IMS. The team considered this as a means to measure the NCC's impact on increasing EEOC efficiency, but the team later identified major data input errors on one of the critical variables, finding a wide range of differences between the percent of EASQs received from the NCC and the number entered into the IMS. Percentages of EASQs entered into the IMS ranged from 1 percent (2 entries in the IMS with an EAS/NCC source coding compared to 249 EASQ sent by the NCC) to 754% (1666 entries in the IMS with an EAS/NCC source coding compared to 221 actual EASQs sent by the NCC). The team found that different offices enter EASQ information into the IMS at different times during the intake process; some immediately enter the information upon receiving it from the NCC, while others enter it after they mail their office intake questionnaire and receive it from the PCP. These differences introduce confounding factors that will affect any analysis comparing offices. Close examination of one office showed that 19.5 percent (28 of 143) of the EASQs sent by the NCC were entered into the IMS in error (2 were duplicate entries and the remaining 26 indicated that the EASQ originated from a non-NCC source). This finding indicated that any further analyses of these data would be unreliable and therefore invalid, so the team stopped at this point. Due to the coding errors identified above, the JPS Team did not use any IMS data related to the coding of inquiries and charges, but the team did use IMS data to evaluate accuracy and completeness of EASQs sent to EEOC offices. Treatment/Control Groups. The JPS Team also explored whether it could identify some offices as control groups. This was only an appropriate strategy if other important confounding factors could be held constant so that the treatment (in this case implementation of the NCC, or even implementation of redirecting telephone calls to the NCC) and control groups were similar. This procedure would help to isolate and quantify the treatment effect; therefore, it would be important to put "like" offices in the treatment and control groups and try to hold other confounding factors (such as turnover) constant across the treatment and control groups. Using treatment and control groups proved impossible due to the manner in which the EEOC rolled the NCC out. The EEOC initially set up seven offices to pilot the NCC for one month, and then commenced implementation across all remaining offices on March 21, 2005. This one- month initial pilot did not give sufficient time to stabilize operations and measure the NCC's impact. Then, beginning November 29, 2005, the EEOC gradually authorized (but did not require) offices to change their telephone voice message to redirect all first time callers to the NCC.[12] This initiative started with eight local offices and has since gradually expanded to include all offices. In both instances, there was no effort to create treatment and control groups and, as a result, there are no comparative groups to evaluate impact. 2. Document Review The JPS Team began this project by reviewing background documents related to the EEOC's decision to implement the NCC. This review included the initial study conducted by NAPA that recommended a contact center. The team also reviewed the 2003 Assessment Report and the transcripts of the Commissioners' meeting reviewing those findings. The team also reviewed the Pearson Contract as well as documents related to the field-repositioning plan and other documents related to operations. A list of key documents is provided in Appendix B. 3. Data Analysis For three offices (two district and one area office), the JPS Team compared information in the EASQs with data in the IMS for the period April 1, 2005 through March 31, 2006 to evaluate accuracy and completeness of the EASQs. The team also analyzed NCC-related GroupWise emails for one district office for the period January 1, 2006 through March 31, 2006. Headquarters Interviews The JPS Team interviewed the Chair, 3 Commissioners,[13] 17 additional officials and staff in Headquarters, and the President of the Union. The team conducted nearly all interviews in person; a few were conducted by telephone. The team performed most interviews during the weeks of October 31, November 7, and 14, 2005, with the intention of collecting employees' impressions of the NCC's impact on Headquarters and field operations. The team also wanted to identify areas to pursue when conducting visits to field offices. The interviews generally took from 60 to 90 minutes to complete. The team began most interviews by describing the team's project responsibilities and asking the interviewees to provide a brief description of their job. Then, depending upon their role, the team asked a series of prepared questions. Most of the questions related to their expectations of the NCC and its impact. The Interview Protocol is attached as Appendix C. In addition to these formal interviews, the team met several times throughout the evaluation, on a less formal basis, with certain employees in the OFP to obtain information on field operations, relevant statistics, and other issues related to the Pearson Contract and operations. 4. Field Site Visits The JPS Team visited seven offices in the field, including five district offices (New York, Charlotte, St. Louis, Dallas, and Los Angeles) and two area offices (Raleigh and Cincinnati). The visits took place during the weeks of December 5 and 12, 2005. The criteria the team used to select offices included geographic and size diversity; offices that the team understood to be positive, neutral, or negative toward the NCC; and offices with different levels of performance. The team also desired an office that provided a representative on the technical evaluation panel that generated the 2003 Assessment Report. Due to the number of criteria and budget and time limitations, the selected seven offices generally addressed the criteria, but not all offices met all criteria; for example, the visits included district and area offices, but no local offices, mostly because they have only a few professionals, limiting the team's ability to collect meaningful data. A description of the procedures followed during these visits is provided in Appendix D. Surveys The JPS Team conducted two surveys; one was sent to Office Directors, and the other, an electronic survey, was sent to everyone in the field. The team developed these questions based upon document reviews, interviews at Headquarters, and visits to the seven field sites as well as the NCC (described next). Descriptions of the Office Director Survey and the Electronic Survey are provided in Appendix D. The Director Survey is provided in Appendix E and the Electronic Survey to all field staff and responses are provided in Appendix F.[14] B. NCC EVALUATION 1. Document Review and Headquarters Interviews JPS team members began this effort by reviewing many of the documents described earlier in this report.[15] 2. NCC Site Management Interviews The JPS Team conducted interviews with the NCC Site Director and management team prior to its visit to the NCC. The purpose of these interviews was to gain an initial understanding of NCC operations, clarify the manner in which the NCC and EEOC communicated, and identify important areas to assess during the site visit. The interviews took from 60 to 75 minutes. The team began by describing its project responsibilities, and then asked respondents to provide a brief description of their day-to-day responsibilities. The team followed up with questions regarding the NCC's organization and operations. The Interview Protocol is attached as Appendix G. 3. Initial Data Gathering Prior to the team's visit, it sent the NCC Site Director and the EEOC Project Manager a data request document pertaining to typical call center operations (e.g., workforce management, call handling techniques, call metrics, agent performance management, and customer satisfaction) stats. The team followed up with a phone call to the Site Director and the contract administrator to explain the details of the requirement, and received data in the form of monthly reports that the NCC sends to the EEOC. The team conducted some basic trending and correlation analyses on call volume, call duration, and primary reasons for calls using Microsoft Excel and SAS.[16] 4. Remote Call Monitoring The JPS Team obtained a remote calling number from the NCC. Remote calling numbers are established so that a third party (other than the caller and CSR) can dial into the call and listen to the conversation. A team of four people dialed into the remote calling number and monitored several calls to gain a high-level understanding of the call flow and gather call statistics. The team used a proprietary call observation tool to conduct the call monitoring exercise. This tool enabled the team to capture information such as call start and end time, amount of time spent in caller identification, amount of time spent in problem identification and resolution, call reason, call disposition, whether the call was transferred, number of holds, caller questions, and CSR answers. The team dialed into the remote monitoring number during different times of the day and different days of the week in order to capture a sufficient cross section of calls. The team monitored 411 calls over a four-week period from October 26, 2005 to November 17, 2005. The team compared its results with the EEOC reports and found the numbers from both sources were comparable. The information the team collected during this exercise therefore served as the primary source of data for most of the team's NCC assessment analyses. 5. NCC Site Visit The team prepared an agenda for focus groups and interviews. It also prepared interview guides and a focus group protocol. The team sent the agenda and interview/focus group protocol to the site director a week in advance of the site visit. The site director responded with the names of the personnel who would participate in the focus groups and interviews. The team made a few changes based on knowledge acquired during the remote call monitoring sessions. The Interview Guide is attached as Appendix G, and the Focus Group Protocol, as Appendix H. The JPS Team spent the week of November 7, 2005 at the NCC, assessing its operations. The team asked NCC employees in the interviews and focus groups about their job description, their understanding of NCC objectives, metrics the EEOC uses to measure the effectiveness of the NCC, and potential improvement opportunities. When meeting with management personnel, the team inquired about management and coaching techniques. The team also met with the reporting, technical, and training subject matter experts who explained their current processes, issues, and opportunities for improvements. The JPS Team conducted interviews with the Site Leader, Team Leaders, and the Quality Manager. The team also led focus groups with the CSRs and conducted side-by-side observations of CSR English calls, TTY calls, faxes, and emails. In addition, the team reviewed monthly reports, scripts, and training manuals. The team also reviewed the NCC processes for call handling, reporting, training, and call monitoring. Because the NCC is in the pilot phase and in transition, many things have changed since the beginning of the evaluation the NCC. The team based most of its evaluation on the detailed analysis and the situation during its site visit in November and December 2005. 6. Data Analysis While visiting the NCC, the JPS Team collected scripts used in answering the calls, data on call flows, training quizzes and answer keys, call monitoring forms, and EASQs. The team analyzed this information, along with other data collected during all of the above listed activities, with quantitative analysis techniques. The team used qualitative data collected during the interviews and focus groups to develop hypotheses. The team used the quantitative data collected during the remote call monitoring sessions and from the NCC reports to quantify the impacts. 7. Customer Satisfaction Data Analysis The NCC had no data on customer satisfaction when the team began its work; therefore, the team attempted to collect customer satisfaction information as a part of its call monitoring exercise. The team tried to capture CSR empathy, professionalism, and subject knowledge, as well as overall customer satisfaction with each call; however, because a small group of individuals monitored most of the calls, there was no variation, and, therefore, the team could not use the data in its analysis. During the course of the project, the EEOC contracted with CFI Group to collect customer satisfaction data. The JPS Team conducted an initial benchmarking analysis comparing their initial customer satisfaction index to the best practices with Federal agencies and in general to service industries in the private sector. The team also compared the results between channels and reviewed the detail analysis to understand other aspects of satisfaction. III. FINDINGS AND CONCLUSIONS A. THE NCC IMPLEMENTATION The JPS Team compared the 2003 Assessment Report with the NCC implementation as well as the metrics required in the Pearson Contract with the metrics that are submitted to the EEOC. 1. 2003 Assessment Report recommendations implemented. The EEOC has successfully implemented several of the recommendations in the 2003 Assessment Report. The Report included a number of recommendations pertaining to establishing an NCC. As shown in Table 2 on the following page, the NCC and EEOC have followed each of the recommendations. Table 2. Implementation of the 2003 Assessment Report Recommendations 2003 Recommendations Findings of Current Evaluation 1. Establish an NCC to handle all unsolicited public To establish the NCC, the EEOC signed a inquiries received by all field offices. At this time, contract with NCS Pearson, Inc., dated do not include Office of Federal Operations or September 20, 2004 that was amended on Field Management Programs in the NCC. September 28, 2005. The contract includes the consolidation of two toll-free phone lines (voice and TTY) for general inquiries and centralized email access via the Internet. 2. Competitively outsource the NCC to get the best The contract was competitively bid: full and value for start-up and operations; use the Statement open procurement, best value award. of Objectives procurement process to expedite implementation. 3. Start operations with a pilot phase for at least two Initial contract is for two years with an option years. Collect baseline data on performance to renew for three years (annually or for all metrics and costs during the first 12 months and three years). The NCC is serving all 51 offices assess vendor performance. The pilot should be in the field. national in scope. 4. NCC services should cover the spectrum of basic The NCC handles calls and other inquiries, Frequently Asked Questions (FAQs), correspondence about basic inquiries, information on laws and procedures, the EEOC including: EEOC Overview, contact Report series, office hours, locations, staff information for EEOC offices as well as for directories, and case information, with caller other Federal agencies, hours of operation, how authentication. The NCC should respond to to file charges, eligibility criteria, and Charge inquiries from Potential Charging Parties (PCPs) status. The NCC also updates the FAQs on the and assist in completing on-line charge website and handles TTY calls, emails, faxes, questionnaires, if appropriate. Other services to and other formats of correspondence. They phase in later include handling email, facsimile, take Spanish and English calls and use a tele- postal mail, and fulfilling requests for publications interpreter for other language calls. The NCC and printed materials. The NCC should not handle does not handle filing of charges. filing charges. 5. Modify EEOC Order 150.005 to allow for Effective January 14, 2005, a paragraph was monitoring/recording of calls of contact center added to EEOC Order 150.005 to permit employees for quality assurance purposes. monitoring and recording calls between NCC employees and the public for purposes of quality assurance. 6. Develop internal and external marketing strategies National Contact Center Outreach Plan, to communicate the compelling case for a national Version 2.0, dated February 2, 2005. For each contact center to the EEOC's staff, customers and external and internal action item there are target stakeholders to solidify support for this initiative. completion dates, responsible groups, and an indication of any funds needed. 2. Pearson is meeting most contractual metrics. The JPS Team compared performance targets to actual performance from September 2005 through April 2006. As Table 3 shows, most of the contractual metrics and objectives set forth in the Pearson Contract are being measured. Customer Satisfaction was originally intended to be measured by a third party and monitored on a regular basis by the NCC Quality Manager. However, due to the long approval processes required by the Office of Management and Budget (OMB), the EEOC determined that the results would not be available in time for the decision on whether to renew the Pearson Contract. As a result, the EEOC contracted with CFI Group to conduct a one-time customer satisfaction survey. The survey was completed in the month of February, 2006, for all of the contact types (eg. Fax, telephone calls, written correspondence) handled by the NCC. NCC is meeting most of the performance metrics specified in the contract, except for Average Speed of Answer, which they met five of eight months (September 2005 through April 2006). Due to technology issues related to recent increased volume, the wait times have been high since the middle of April 2006 (up to 25 minutes before the CSR comes on the line). As of June 19, 2006, this problem was resolved. Table 3. Comparison of Contractual Performance Metrics and Reporting Status Performance Metric in Pearson Expected Target in Extent the NCC Met or Contract Pearson Contract Exceeded the Target Call Monitoring Scores 90% - 95% of monitored 100% calls Accuracy of Information Provided 95% - 97% 100% (Measured through Call Monitoring) Accuracy of Information Captured 95% - 97% 100% (Measured through Call Monitoring) Customer Satisfaction 70% - 75% satisfied or Customer Satisfaction Index is extremely satisfied 77* Service Level - Average Speed of 70% to 80% in 30 seconds 62.5% Answer or less Average Speed to Respond to Email 70% - 80% in one business 100% day Average Speed to Respond to Fax 70% - 80% in two business 100% days Average Speed to Respond to 70% - 80% in two business 100% Written Correspondence days Blocked Calls 1% - 3% 100% * Gathered for the week of February 7-13 by an independent contractor. The index is not a percentage. See Section IIIG8. B. COMPARISON OF NCC ESTIMATES VERSUS ACTUAL STATISTICS 1. The actual call volume and savings in staff hours are less than projected. The JPS Team reviewed and compared several estimates of NCC activity contained in the 2003 Assessment Report with 2005 actual statistics. Based upon NCC reports and call monitoring analyses, call volume is lower than 2003 estimates, which has resulted in less EEOC field staff time saved compared to projections. During the first year of pilot operations (April 2005 - March 2006), the NCC received 269,693 calls. The 2003 Assessment Report estimated all unsolicited calls to offices in the field (toll-free and local). According to comments from OFP, prior to implementation, the decision was made to route only toll-free calls to the NCC, with unsolicited calls to each office in the field handled locally. Therefore, the baseline 86,824 calls identified in the 2003 Assessment Report (1.2 million calls annually) will be higher than the anticipated number of calls to the NCC until the EEOC decides to require that offices direct all unsolicited calls to the NCC. The present call volume to the NCC is also low because call volume to the EEOC's toll-free numbers has decreased 17.5 percent (when comparing March 2003 to March 2006). This is consistent with reduced annual charge receipts since fiscal year 2003 (see Figure 2). The NCC handles a variety of different types of communication, including calls handled by CSRs, calls handled by IVR, and other forms of written communication. Figure 1 below shows the call volume (English, Spanish, and TTY calls handled by CSRs) by month as well as the trend of calls. As the figure shows, there is not a significant variation month over month (except from December 2005 to January 2006 and February to March 2006) or a clear trend showing an increase in the number of calls received by the NCC. Figure 1. English, Spanish and TTY Calls Handled by the CSRs, April 2005-April 2006 In addition to calls handled by a live person, Table 4 shows the number of calls handled by the IVR during the first 13 months of operations. There is no particular pattern evident in the number of calls terminating at the IVR. Table 4. IVR Calls Handled by the NCC Month IVR Calls April-05 8,739 May-05 8,342 June-05 7,981 July-05 6,708 August-05* 8,300 September-05 7,812 October-05 8,705 November-05 7,207 December-05 6,014 January-06 8,363 February-06 7,885 March-06 8,564 April-06 8,028 *Estimate due to NCC equipment malfunction The NCC also handles other forms of communication, including emails, written correspondence, web inquiries, and faxes. As shown in Table 5, the NCC receives and processes significantly more emails than other forms of communication. Table 5: Other Forms of Correspondence Handled by CSRs Month Email Written Web Fax Total Other Correspondence Inquiries Correspondence April-05 1,436 52 209 36 1,733 May-05 1,436 46 264 45 1,791 June-05 1,417 67 267 72 1,823 July-05 1,296 51 265 64 1,676 August-05 1,549 75 292 48 1,964 September- 1,551 62 348 63 2,024 05 October-05 1,476 68 293 74 1,911 November-05 1,254 75 274 73 1,676 December-05 1,247 58 221 62 1,588 January-06 1,540 71 257 84 1,952 February-06 1,379 44 359 69 1,851 March-06 1,363 62 865 64 2,354 April-06 1,313 51 719 76 2,159 Since the 2003 Assessment Report was based on actual calls answered by EEOC employees and not calls handled by message machines, the JPS Team's evaluation will consider only actual calls handled by the CSRs so that the numbers are comparable. The evaluation does not use calls handled by the IVR to compare estimates with actual results, as they could include misdialed numbers; further, if a call can be answered by an IVR at the NCC, it can also be handled by an IVR at any EEOC office. It is difficult to determine whether callers obtain useful information from the IVR without an in-depth analysis of the IVR. Calls handled by the IVR are not equivalent to calls handled by EEOC employees, because no matter where they are handled, they would not require an EEOC employee. During the first full year of operations (April 2005-March 2006), the CSRs handled 269,693 calls. During the second full year of operations, the JPS Team projects that the NCC will receive between 273,804 (annualized based on the average of the 13 past months) and 319,020 (annualized based on the March 2006, the month with the highest call volume) total calls. The higher projection must increase four times to reach the volume anticipated in the 2003 Assessment Report. Based on the estimated annualized volume of 273,804 calls per annum and the 51 percent resolution rate achieved by the NCC,[17] the team estimates that the NCC saves the EEOC 13,964 staff hours. Table 6. Comparison of 2003 Survey Data to 2005 Evaluation Findings Measure Estimates from 2003 Current Report Findings Assessment Report Call Volume Projection of 1.2 million 269,693 calls handled by CSRs calls Based upon 86,824 calls received by the field and Headquarters in 20 working days Contact Channels EEOC Field EEOC Field Offices Offices EEOC Headquarters EEOC NCC Headquarters o English Calls o Spanish Calls o Language Support o TTY o Fax o Email o Written Correspondence o Web Inquiries Call Duration* 69% - Up to 3 Average handle time for calls is six minutes minutes. Calls vary between 2 and 15 16% - Between 3 minutes, depending upon the call type. and 5 minutes 9% Between 5 and 10 minutes 4% Between 10 and 20 minutes 2% Above 20 minutes A significant percent of the calls were less than 3 minutes. Time freed up for EEOC Estimated savings of Estimated savings of 13,964 hours per year employees to engage in 43,224 hours for GS 5, 7, based on data collected from the NCC. investigations, outreach, 9, 11, 12, and13 and support activities with implementation of the NCC. *Call duration for contact center operations are typically reported in averages The JPS Team compared the NCC estimates listed in the Pearson Contract with actual statistics. The Pearson Contract contains estimates for the first six months of NCC operations. As shown in Table 7, the NCC has received substantially more emails than expected and significantly fewer calls and faxes. Table 7. Six Month Estimates in Pearson Contract Compared to Actual Statistics, April through September 2005 Estimates in Pearson Contract Actual English IVR - 719,554 English IVR - 47,882 (the numbers for August 2005 are estimates as the actual numbers were not English CSR- 306,332 available due to malfunction of the IVR) Fax - 4,500 English CSR - 130,822 TTY English - 3,245 Fax - 327 English Email - 2,381 TTY English - 1,139 English Email - 8,616 C. NCC IMPACT ON HEADQUARTERS OPERATIONS AND STAFF The JPS Team reviewed written and telephone communication, website activity, charge statistics, and outreach efforts at Headquarters. The team found little impact attributed to the NCC. The fiscal year data presented below was provided by the EEOC. 1. Written and telephone communications to Headquarters decreased. There may be a relationship between reduced communications and implementation of the NCC; however, there are no data indicating the decrease is attributed to the NCC. Controlled Correspondence Congressional Correspondence and Correspondence to the Chair decreased over the past three years. The reduction in Congressional Correspondence (generally pertaining to status of Federal Sector complaints) typically relates to inquiries about case status and may be the result of the normal fluctuation of appeals in the Federal sector. It is also possible that the 29.5 percent reduction in correspondence from 2004 to 2005 reflects better service by offices in the field as well as the Office of Federal Operations due to the NCC answering calls, thus enabling employees to manage their workload more effectively, resulting in fewer complaints. Table 8. Congressional Correspondence to Headquarters, 2003- 2005 Fiscal Year Congressional Correspondence Percent Reduction 2003 1,407 - 2004 1,403 .28 2005* 988 29.50 *NCC implemented 3/21/05 As shown in Table 9 below, Correspondence to the Chair increased in relation to operations in the field during the first six months of NCC operations, and then decreased. When comparing equivalent periods pre- and post-NCC implementation (October through March), there is an 8 percent decrease in correspondence related to Headquarters operations and a 16 percent decrease related to field operations. The combination of better customer service, increased outreach, establishment of the NCC, and increased information on the EEOC's website (including the FAQs developed by the NCC) may have resolved concerns that in previous years would have resulted in written correspondence to the Chair. Table 9. Correspondence to the Chair, October 2004-March 2006 Percent Percent Percent increase increase increase (decrease) (decrease) (decrease) Headquarters from previous Field from previous from previous Fiscal Year Operations period Operations period Total period st 1 Half 2005 129 - 150 - 279 - (October- March) 2nd Half 2005 206 59.7 180 20.0 386 38.4 (April- September)* 1st Half 2006 119 (42.2) 126 (30.0) 245 (36.5) (October- March) *NCC implemented March 21, 2005 Telephone calls to the main EEOC public number Table 10 on the next page shows available data for telephone calls to the public EEOC telephone number (202/663-4900). Over the past four calendar years, there has been a steady reduction in total calls. Data for the 2004 and 2005 calendar years are incomplete, but four months are comparable. There is a similar reduction in total calls pre- (February 2005) and post- (May and June 2005) NCC implementation. There is a larger percentage reduction in total November calls. When considering just the calls related to the field offices, there was a larger reduction in calls pre-NCC (February) than post-NCC (May). It is possible that some of the call reduction can be attributed to the NCC. Table 10. Comparison of Four Years of Telephone Contacts to EEOC Headquarters Calls to Headquarters Regarding Total Headquarters Calls Field Offices Month 2002 2003 2004 2005 Percent Reduction 2004 2005 Percent Reduction 2004-2005 January 6,423 8,877 2,897 * - * * - February 7,831 7,246 2,994 1,812 39.5 811 308 62.0 March 9,977 * 3,595 * - 963 * - April 6,834 * * * - * * - May 9,753 * 1,943 1,342 30.9 500 261 47.8 June 8,316 * 2,502 1,616 35.4 * 371 - July 6,838 * * 996 - * 417 - August 11,841 * * * - * * - September 8,738 * * * - * * - October 8,843 * * * - * * - November 9,398 * 2,268 1,124 50.4 * 308 - December 6,287 * * * - * * - *Missing or incomplete Office of Field Programs (OFP) OFP Management has a general sense that telephone complaints from the public to OFP regarding an inability to reach an EEOC employee in the field by telephone have diminished, but there is no tracking system to show numbers or types of complaints. 2. Website activity has increased and it is possible that some of the increase is attributable to the NCC. The total number of visitors to the EEOC website has increased over the past five years. During the first seven months of Fiscal Year 2006, average monthly user sessions have continued to increase to 551,490. According to the Web Site Content Manager, this reflects a typical growth pattern for Federal websites. Table 11. Average Monthly Website Hits, Fiscal Years 2001- 2005 Fiscal Year Average User Sessions/Month Percent Increase 2001 208,534 - 2002 246,756 18 2003 340,706 38 2004 360,000 6 2005 444,533 23 For the last six months of fiscal year 2005, the NCC's IVR provided the NCC web site address, and NCC staff also informed customers about information available on the web site; therefore, some of the increase from 2004 to 2005 may be attributable to the NCC. It is also possible that customers who visited the website when they were unable to get through to field offices are now contacting the NCC and getting their questions answered before visiting the web site. The EEOC developed the FAQ page on its website in conjunction with the formation of the NCC, and the NCC manages the order of the questions on the page (the most frequently asked questions appear first). There was an average of 11,285 visits to the FAQs over the 12 months from April 2005 through March 2006, with a range from 9,234 to 15,440 visits. Nearly 45 percent of the total FAQ web hits were related to intake and charge filing. 3. Post-NCC implementation, the pattern of annual and monthly charge statistics remains consistent with prior years. Charge Statistics - Annual and Monthly Over the past 13 years, the volume of charge statistics has varied. As shown in Figure 2 on the next page, the general change across time appears to be cyclical.[18] Figure 2. Annual Charge Receipts, Fiscal Years 1992-2005 Following is a comparison of the monthly individual charge receipts for four fiscal years (2002- 2005) and October through April of the current year. While there are monthly differences in total receipts, the overall pattern of receipts across each of the 12 months has remained consistent. There have been no significant fluctuations since the inception of the NCC. Figure 3. Monthly Charge Receipts, Fiscal Years 2003-2006 4. Employees perceive a need for increased awareness of the NCC, and efforts are underway to broaden publicity. Managers and employees interviewed at field sites as well as respondents to both the director and employee surveys indicated that they believe the EEOC needs more visibility and that it is not sufficiently publicizing its mission or its toll-free number. They suggest that current outreach events inform human resources groups, but that most employees in small-to-medium sized companies do not know about the mission or work of the EEOC. The national outreach group at Headquarters conducts 5,000 to 6,000 outreach events per year, as well as 500 to 600 media events that reach hundreds of thousands of people. As a result, many employers know about the EEOC; further, in an effort to reach more employee groups, the EEOC is continuing to expand their advisory councils and partner with other agencies. EEOC/NCC Awareness. Specific to the NCC, a few survey respondents commented that many potential EEOC customers do not know about the toll-free number, even though it has been in existence since 1991. Interviewees suggest developing an advertising campaign similar to the Freedom to Compete and Youth at Work initiatives. Survey respondents recommended providing pamphlets about the NCC to each individual coming into the offices, having every office redirect callers receiving voicemail to the NCC's 800 number, and contacting civil rights organizations (e.g., National Hispanic Civil Rights) to provide awareness of the NCC's services in their communities. The EEOC anticipates launching a major new advertising campaign in July 2006 to link the NCC and the 800 number. The plan currently includes a theme and logos to increase awareness of the 800 number. This effort will include developing and distributing brochures, handouts, and promotional giveaways. There will also be press releases and stories for field offices to send to local print media contacts including ethnic and minority publications. In addition, there are plans for radio and Internet Public Service Announcements. This campaign is intended to create awareness among target audiences of the EEOC's readiness to help, position the 800 number as the preferred method to access the EEOC, and increase inbound calls to the NCC. The EEOC ordered promotional materials for this campaign on June 15, 2006, and plans to distribute them to field offices a few weeks thereafter. Telephone Listings. Headquarters is currently responsible for entering and updating all telephone listings in local telephone books. Listings are submitted to GSA for entry into the Blue Page (Federal) section of local telephone directories. The EEOC provides GSA with both the toll-free and local numbers. Therefore, the toll-free number should be listed in all local directories, but a review of local telephone directories by field personnel revealed that the NCC toll-free number is listed in the government listings section of printed telephone directories for 36 cities across the country, but not listed in another 27 cities. In addition, the on-line blue pages directory (http://www.usbluepages.gov/index.html) lists the local telephone number for EEOC offices under "discrimination" but does not list the NCC toll- free number. It also does not list the EEOC under the category "Agencies and Services." The EEOC has recently made a decision to publish the numbers under "discrimination" and Equal Employment Opportunity Commission and to work with GSA to ensure the toll-free numbers are published in local directories. The EEOC has made a policy decision to continue to publish local telephone numbers in the blue pages. This policy limits the number of calls going to the NCC and the potential number of customers served by the NCC, particularly since not all offices redirect first-time callers to the NCC (discussed later). D. NCC IMPACT ON EFFICIENCY AND EFFECTIVENESS OF FIELD OPERATIONS There have been some reductions in unsolicited calls to the field offices. The CSRs are not resolving as many calls as they could. 1. Field employees have mixed perceptions of the impact of the NCC on call volume to offices. Comparing call volume pre- and post-NCC, employees in focus groups indicated a perceived decrease in calls in an average week, ranging from a 0 to 60 percent. Respondents to the director survey who reported reduced call volume thought their employees were better able to answer and return phone calls on a timely basis. This has relieved pressure on people answering phones, and now fewer people must answer "cold calls" (initial calls from the public). A few respondents to the director survey indicated they have not experienced a drop in call volume. As suggested by some employees in focus groups, one reason may be that the NCC is picking up calls that have been previously dropped or lost due to telephone technology, thus increasing the workload. The electronic survey asked employees who answer and/or return unsolicited calls to their office public number to estimate the number of calls they received in an average week before and after NCC implementation. The data showed a clear break in the distribution between 400 and 600 total calls. Cases reporting more than 400 calls were considered outliers and removed.[19] The results show that employees in Pay Grade 4 perceived a 76.9 percent reduction in calls and that employees in Pay Grade 8 perceived a 44.7 percent reduction. Respondents in Pay Grades 5, 7, and 12 perceived an approximate 25 percent reduction. The mean increase in calls reported in Pay Grade 15 is largely attributable to one person who indicated 150 more calls post-NCC versus pre-NCC.[20] Table 12. Mean Change in Calls to Office Public Telephone Numbers in the Field, Pre/Post- NCC Pay Mean Mean Calls Mean Call Percent Number of Grade Calls Pre- Post-NCC (Reduction) Increase (Decrease) Employees NCC per week Increase Responding 4 32.5 7.5 (25.0) (76.9) 2 5 79.6 58.8 (20.8) (26.1) 39 6 23.3 14.7 (8.6) (36.9) 16 7 87.8 66.3 (21.5) (24.5) 43 8 63.3 35.0 (28.3) (44.7) 6 9 44.0 34.3 (9.8) (22.3) 24 11 16.5 15.9 (0.5) (3.0) 15 12 43.0 31.6 (11.5) (26.7) 251 13 33.6 30.3 (3.3) (9.8) 81 14 16.7 14.0 (2.6) (15.6) 73 15 17.5 26.7 9.3 53.1 11 SES 15.0 10.7 (4.3) (28.7) 3 Comments from some survey respondents indicated that they perceive no reduction in calls. It is likely that some offices are receiving more calls, perhaps due to increased outreach, and some offices receive fewer calls. It is also possible that some employees have the same number or more calls because of other factors, such as attrition. Support staff in one field office mentioned that PCPs have said that the NCC is telling them to call offices directly, which may be another reason why employees do not perceive reduced telephone calls to their public number. In part, this is occurring because of NCC business rules requiring that the CSRs advise some callers to contact the EEOC office directly. Callers directed to the field offices are those who specifically request to contact the office, have complex or unusual questions for which the NCC has no approved scripts, or insist on speaking with the office even though the EEOC does not have jurisdictional coverage or time limits have expired. Another possible reason that calls have not decreased more significantly may be related to NCC operations. The NCC does not measure or monitor call resolution, and the CSRs take very little ownership of the call. During the JPS Team's call monitoring and side-by-side observations, it observed that some CSRs subtly push the callers to contact the EEOC offices directly when they should take time to answer their questions and/or fill out the EASQ. The NCC's focus on Average Handle Time (AHT) may be pushing the CSRs to get off the phone to meet their AHT goal. This combination of business rules and CSR practices may account for reports from many employees in the field that the NCC has had little positive impact on them. The JPS Team was able to identify the impact of the NCC on the call volume for the Dallas and Charlotte District Offices because they tracked the number of telephone calls received on their public line pre- and post-NCC.[21] As shown in Table 13, the impact of the NCC on telephone calls differed for each office. The Charlotte Office realized increased telephone calls and, therefore, a net increase in contacts (telephone calls and NCC forms) after implementation of the NCC. During the team's site visit, the Charlotte District Director indicated they may be receiving increased calls due to outreach efforts. Commencing in January 2006, callers to the Charlotte Office received a recorded message to contact the NCC if they were calling during specific times before or after regular business hours (8:00 a.m. to 8:30 a.m. and 5:30 p.m. to 8:00 p.m. EST). The Dallas Office experienced a net decrease in calls to their public number, for an average monthly savings of 489 contacts for the months of October through December 2005 and 302 contacts for the months of January and February 2006. The relatively larger number of NCC forms in January/February 2006 over the previous quarter is likely due to two factors: in January, the Dallas office started redirecting overflow calls during the day to the NCC, and also received some duplicate GroupWise emails during this period. Table 13. Comparison of Charlotte and Dallas District Office Telephone and NCC Forms Telephone Calls Net Increase / Average Average Average Average No. (Decrease) in Office / Calls Pre- Calls Post- Increase EAS/GroupWise average contacts Months NCC NCC (Decrease) sent to Office to EEOC Office Charlotte May- 495 878 383 59 442 September (2004 and 2005) January- 537 839 302 174 476 February (2005 and 2006) Dallas October- 1,798 1,135 (663) 174 (489) December (2004 and 2005) January- 1,754 936 (818) 516 (302) February (2005 and 2006) The JPS Team also compared the first two months of NCC operations with two recent comparable months.[22] As shown in Table 14, both offices are realizing decreased telephone calls. This may be because both offices are redirecting overflow calls on their intake lines during the day. The Charlotte office experienced a net decrease in contacts from the public, whereas the Dallas Office experienced a net increase. EASQs sent to the Dallas Office for March and April 2006 increased by 50 percent. The increase in contacts at the Dallas Office may be related to their decision in January to redirect those callers who reach voice mail during the day to the NCC. Table 14. Comparison of Dallas District Office Telephone Calls and NCC Forms Telephone Calls Average Increase Net Increase Average Average Net Increase (Decrease) in (Decrease) in March and March and (Decrease) EAS/GroupWise per Number of Office April 2005 April 2006 Month Contacts Charlotte 963 712 (251) 188 (63) Dallas 1,390 1,064 (326) 457 131 2. The NCC receives low call volume, and the NCC is not resolving as many calls as it could. Therefore, the NCC has had minimal impact on the call volume received by EEOC offices. Call Volume According to the NCC reports sent to the EEOC, over the first 13 months of operations, the NCC handled an average of 22,695 calls and 1,879 other forms of correspondence per month. As discussed earlier, this is substantially lower than the estimates in the 2003 reports. One factor affecting total calls to the NCC is changes in EEOC policies during the timeframe of this analysis. From April through most of November 2005, customers called either the toll-free number (answered by the NCC) or the local number (answered by the EEOC offices). From November 2005 through March 2006, the EEOC gradually authorized, but did not require, offices to put a message on their voicemail redirecting first-time callers to the NCC. In May 2006, the JPS Team called each office during and after business hours to determine the extent to which offices presently redirect callers to the NCC. As Table 15 shows, 28 of 51 offices redirect callers to the NCC after normal business hours. During business hours, the team reached 19 live people and 32 message machines. One of the EEOC Receptionists redirected the team to the NCC. The Receptionist for that office answers general questions such as location and office hours and refers all other "cold calls" to the NCC. According to the Office Director, customers have not objected to calling the NCC after talking with the Receptionist. Of the 32 message machines the team accessed during business hours, 19 redirected callers to the NCC. Twelve of those offices also redirect callers after business hours. Table15. Calls to Offices During and After Business Hours During Business Hours Office Response After Hours Message Machine Person No answer after three tries* 2 - - Redirect Callers to NCC 28 19 1 Do Not Redirect Callers to NCC 21 13 18 Total Number of Offices 51 32 19 *The first time the JPS team called offices after hours, it received no answer or an indication of a full message box for five offices. As Table 16 shows, there has not been a significant increase in either call volume or other forms of correspondence for the month of April 2006 when compared to the other months, and, particularly, April 2005. The increases shown in January through April 2006 may reflect the normal increase in inquiries during the second fiscal quarter (see Figure 3); therefore, the team does not expect a significant change in volumes unless the EEOC develops a policy requiring all field offices to redirect their unsolicited calls to the NCC. Table 16: Contacts Handled by NCC CSR Contacts Total Calls Total Other Month/Year Handled Correspondence Total April-05 25,490 1,733 27,223 May-05 23,561 1,791 25,352 June-05 23,574 1,823 25,397 July-05 21,561 1,676 23,237 August-05 24,899 1,964 26,863 September- 20,043 2,024 22,067 05 October-05 21,119 1,911 23,030 November- 19,368 1,676 21,044 05 December-05 18,046 1,588 19,634 January-06 23,388 1,952 25,340 February-06 22,059 1,851 23,910 March-06 26,585 2,354 28,939 April-06 25,336 2,159 27,495 Call Resolution The JPS Team captured information indicating the extent that CSRs resolve calls during its remote call monitoring. The team recorded a call as "resolved" when, upon hanging up with the CSR, the caller would not have to contact or be contacted by an EEOC office; for example, non- jurisdictional calls, PCPs requesting general information, and CPs inquiring about charge status were considered resolved at the NCC. The team's analysis shows that the NCC is successful in resolving 51 percent of its calls. These calls relate mostly to general inquiries or information about other Federal agencies. The other 49 percent of the calls to the NCC are either referred (the caller is asked to contact the local EEOC office) or forwarded via an EASQ to the EEOC office. Table 17: Call Types by Resolution at NCC for Calls Handled by NCC Percent Percent Forwarded / Handled by Call Type Description Referred to EEOC NCC Eligibility Call relating to general overviews, 43 57 Criteria inquiring to understand their rights, etc. Charge Status Caller has already filed a charge and 59 41 wants to know the status File a Charge Caller is interested in filing a charge 70 30 Non EEOC Caller is seeking information about other 0 100 Related Federal agencies EEOC Office Caller is seeking information about an 20 80 Information EEOC office Other Other types, e.g. caller wants a 42 58 publication, an employer calling, complaints about the EEOC, etc. Total All Calls 49 51 As shown in Table 18, the 51 percent of calls that are resolved at the NCC equate to approximately 13,964 staff hours per year, which is equivalent to 6.7 Full Time Equivalent (FTE) Investigators.[23] Table 18. Average Hours Saved per Month Projected Current Value (Based on Average Best Case (Based on Call Volume Call Volume for first 13 months of for March 2006, claimed as the Metric NCC Operations) busiest month for NCC) Projected annual 273,804 319,020 calls (1) Percent of calls 51% 51% resolved by CSRs Annual call volume 139,638 162,700 handled by the CSRs (2) Call duration in 6 6 minutes Staff hours saved 13,964 16,270 per year Staff hours saved 1,164 1,356 per month FTE 6.71 7.82 (1) Actual total calls for April 2005 through March 2006 was 269,693 (2) Excludes IVR calls This hourly savings per month is consistent with employee comments that they perceive little savings resulting from the NCC taking calls to their office public number. E. NCC IMPACT ON FIELD STAFF The NCC is beginning to create additional time for field staff, but some factors are limiting further improvement. Both organizations are duplicating work, and the NCC is creating more work by sending EASQs and GroupWise emails that are often inaccurate and incomplete. The NCC also inconsistently forwards "hot line" callers (those at risk of losing their right to file under the Statute of Limitations (SOL)). Comments about the NCC (by EEOC management and employees) were generally similar across all visited offices. 1. The NCC has created some additional time for field staff to perform other duties. Redirecting Office Resources Fourteen respondents to the director survey indicated that the NCC has helped address some of their office staffing resource challenges (Investigators, 6 offices; Support Staff, 8 offices), as shown in Table 19,. Six respondents indicated that the NCC has saved Investigators time because they no longer have to answer unsolicited calls on their office public number or return calls that are not in the EEOC's jurisdiction. Eight respondents said that the NCC has improved the efficiency and effectiveness of their support staff, and some Directors have been able to redirect their support staff to help Investigators. Eleven of these fourteen offices are redirecting callers to the NCC after business hours. The JPS Team found no relationship between these 14 offices and Merit Factor or Non-ADR Resolutions per available Investigator. Two respondents to the director survey indicated no ability to redirect resources, which they attributed to an increased number of inquiries since launching the NCC. Five respondents reported that the NCC has increased their workload, particularly at the supervisory level, largely because supervisors review and track the NCC forms to ensure that staff handle them appropriately, which causes them to get behind in other duties. All remaining offices reported no ability to redirect resources. Reasons cited included increased paper and follow-up work related to EASQ and GroupWise emails, and attrition over the past few years (people are still behind in their duties). Table 19. Impact of the NCC on EEOC Offices Number of Offices District Field Area Local Total Impact on Offices Office Office Office Office Offices Some positive impact for Investigators due to 2 - 1 3 6 no longer answering unsolicited calls (savings of 1-3 days per month) Some positive impact for support staff - 1 2 5 8 (Receptionist, Investigator Support Assistant); instead of answering phones, they are more efficient at their job or are handling NCC paper work Resource allocation the same, partially due to 1 - - 1 2 perceived increased number of inquiries (possible reasons include NCC) There is more work because they are handling 1 2 2 - 5 more calls and NCC paper work requires more Supervisor / Investigator time No change. Everyone is doing what they did 3 - - - 3 before and/or phone time savings have shifted to NCC paper work Do not know or same amount of work so have 8 6 10 3 27 not redirected any resources The net staff savings for the 6 offices that realized the greatest benefit is equivalent to 5.2 FTE Investigators. The savings for the eight offices that realized benefits in support staff are difficult to quantify, because the improvement ranged from enabling Receptionists, Office Automation Assistants, and Investigator Support Assistants to do their own job better to helping support intake and/or help with NCC related forms. The team assumes that the support staff savings equate to approximately 1.5 FTE Investigators, yielding the same 6.7 FTE Investigators as described earlier. One District Director commented that it is difficult to assess the NCC's overall impact because of possible confounding factors. As reported, this is partially because offices have no baseline data on number of calls or volume of email and regular mail. As a result, there are no objective data describing the NCC's impact on the workload in the field. In this district, offices were unable to handle their call volume, which resulted in many unanswered calls or people calling the office again. The NCC is picking up the slack on the unanswered calls, but since those calls were not absorbing office resources, the NCC has not significantly freed up staff members to perform additional duties or reduce assigned workload. Investigator Work During field site visits, Investigators indicated they still have the same duties as they did before NCC implementation, assertions that have been confirmed by Enforcement Managers who reviewed the specific duties and Attorneys who said the Investigators have no more time since NCC implementation. Attorneys who spoke with the JPS team indicated that the quality of cases has improved over the past five years, but there are no data suggesting that this is attributable to the NCC.[24] Some survey respondents commented that the NCC has not reduced their workload . Employees in focus groups and survey respondents indicated that one reason may be that the NCC is providing only limited support by taking phone messages and relaying information unless the call is blatantly non-jurisdictional. Survey respondents commented that the NCC functions like a glorified answering machine. Comments by Investigators and other staff that the NCC has not created additional time for investigations may suggest a lack of communication about expectations related to the NCC. A main purpose of the NCC was to relieve Investigators from answering unsolicited calls, which was expected to increase time available to conduct investigations - not to relieve staff of intake duties. As shown in Table 19 above, six respondents to the director survey reported that Investigators are no longer answering and/or returning unsolicited calls. However, even with the support of the NCC, one office is still receiving around 100-115 pieces of direct mail inquiries a week, 45-50 EAS/GroupWise emails from the NCC, 60-80 messages on their phone intake line a week, and an untracked number of calls on their public telephone number. If an average intake inquiry takes 45 minutes, they estimate that Investigators would spend 25% of their time on intake; and management estimates that it actually takes over 30%; therefore, Investigator workload remains high. The JPS Team has no empirical data with which to judge changes in workload. At the sites the team visited, Investigators indicated that they are no less busy now than prior to the NCC. One of the Regional Attorneys noted that the Investigators need more people and relief on their workload, and that there is presently hostility because Investigators perceive that the NCC is receiving a high amount of money when money is tight. Other Staffing Attorneys reported that they performed their own clerical work because some of their support staff are helping with intake (answering phones and other duties), which they anticipated the NCC would handle. Some offices report no noticeable change in workload for Office Automation Assistant and Investigator Support Assistants and the Secretaries/Receptionists indicate new duties as result of the NCC, such as downloading the EASQs and GroupWise emails and tracking disposition. 2. The NCC is creating inefficiencies by duplicating/creating more work for offices in the field and/or performing insufficient screening. Duplicating Work The NCC is collecting and forwarding basic information on the EASQ to the EEOC offices. Intake employees are repeating some of the process, which is causing a duplication of work. Office supervisors and survey respondents reported that they have to start the intake process over and obtain basic information from PCPs who call or come to their office through the NCC. One office has started screening all NCC contacts for all EEOC jurisdictional issues because it does not appear as though the calls are screened by the NCC. One respondent to the director survey wrote: "The NCC has actually increased the work for [this] office by taking duplicate calls, capturing and forwarding incorrect information about the individual's contact information and providing incorrect information to callers." The NCC also creates duplicate work when it does not sufficiently crosscheck the IMS, which occurs for the EASQs as well as the GroupWise emails. One director explained that often the "Notes" section in the IMS may already contain information stating that the office has left a message for the PCP when the office receives a GroupWise email that no one has returned the PCP's calls. As one respondent to the director survey wrote: "It seems like a duplication of effort when we have made contact with a PCP [to] then get information from the NCC that we need to call the person (who has already been called)." As described above, some survey respondents and employees in focus groups in several offices reported that the NCC is not checking to determine whether an individual has an existing charge or looking up case status. Thirty-nine percent of survey respondents involved in intake reported that many (more than twenty percent) of the EASQs that they review are for PCPs who already have an inquiry in the IMS and therefore should not have been forwarded as an EASQ. This duplicates work and frustrates the Investigators when they invest time discussing the potential charge and then, upon entering the inquiry in the IMS, discover that another Investigator had already entered the information a week earlier. Of the 238 survey respondents involved in intake, 46 percent reported that many of the GroupWise emails that they reviewed were for people inquiring about charge status, which could possibly have been resolved at the NCC. The 2003 Assessment Report indicated calls related to case status as one of the types of calls that the NCC could handle; however, to ensure CP privacy, NCC business rules require that the CP provide the EEOC charge number and one other piece of identifying information. This means that CSRs should forward some calls inquiring about case status to the EEOC office for handling, but because CSRs think that they must have a charge number to look up a case in the IMS, they do not use the IMS to identify whether callers have previously filed an inquiry with the EEOC. Creating More Work The NCC may be creating more work for EEOC employees under the current processes. During JPS Team site visits, office managers and supervisors reported that the NCC has created another avenue for customers to complain if issues are not handled as quickly as they think they should or if they do not agree with the determination. This results in offices having to reopen previously closed cases. Twenty-seven percent of survey respondents involved in intake reported that many (more than twenty percent) of the GroupWise emails that they reviewed were complaints about the processing or outcome of a closed case. Employees commented that callers appear to perceive the NCC as another avenue to contact the EEOC, often creating more work. One survey respondent wrote: "A CP that is not satisfied with a Predetermination interview or anything else the field office does, simply calls the NCC who now generates a customer complaint [that the field is already aware of] but now must [stop other work and] respond to within 48hrs." In addition, by not fully disclosing their role in the intake process, the NCC may be creating confusion about the difference between the NCC and EEOC. For example, one comment in the responses to our office director survey explained that a CP would often call the Investigator, leave a voice mail message, and then dial the NCC. In a similar vein, 42 percent of survey respondents indicated that many PCPs they talked with had the impression that they filed a charge with the NCC. One office is trying to eliminate this confusion by requesting that staff discuss the NCC's role during charge counseling to assure the CP that the NCC is part of the EEOC. Table 20. Field Staff Opinion about whether Potential Charging Parties Thought they Filed a Charge with the NCC Potential Charging Parties who thought they Filed a Charge Percent of Survey with the NCC Respondents None 33 Relatively few (1-20%) 25 Relatively many (>20%) 42 Focus group participants indicated that some PCPs do not want to answer questions from Investigators because they already gave most of the same information to the NCC and ask why they had to talk with the NCC. This perception by PCPs can affect Investigators and the PCP because it takes Investigators extra time to backtrack and explain the limited role of the NCC. Survey respondents wrote that CPs indicate frustration when they have to retell their "story" to another person. NCC Screening The field survey respondents indicated that the NCC is screening out many non-jurisdictional cases. Table 21 shows that the NCC is effectively screening most of the categories listed. For example, 62 percent of respondents (involved in intake) said that no EASQs inquiring only about age discrimination were from people less than 40 years old, and were therefore appropriately screened; however, screening related to the EEOC bases of discrimination shows that 38 percent of respondents indicated that many of the EASQs they received should have been screened out. This number may be somewhat high compared to the other categories if survey respondents indicate that the basis recorded on the EASQ was incorrect (one office indicated that it looks like the CSRs are randomly picking a basis). In support of these survey responses, the JPS Team observed that CSRs read the list and let the caller pick the issue, often without asking why the caller thought the basis was discriminatory (for example, if age discrimination, the CSR often does not ask the person's age/date of birth). Table 21. Effectiveness of NCC Screening for Non-Jurisdictional Calls Percent of Survey Respondents Believe A few EASQs Many EASQs All EASQs were should have should have Categories the NCC is appropriately been screened been screened Screening screened out (1-20%) out (>20%) Not job related 43 36 21 Federal complaints 67 27 6 Not national origin, race, 27 35 38 sex/gender/pregnancy, religion, age, color, disability or retaliation Company with 14 or fewer 43 41 16 employees Age only and in companies 63 26 11 with 19 or fewer employees Age only and less than 40 62 27 11 years old A few survey respondents also commented that it would be helpful if the NCC would screen out or identify repeat callers and frequent filers. 3. EASQs and GroupWise emails provide limited useful information. The EASQ was designed to provide only basic information to submit an inquiry to the EEOC, including who, what, where, when, and why a PCP believes an action was discriminatory. Its purpose is to provide intake staff sufficient information to prepare for the formal interview, not to relieve field staff of intake responsibilities. During the pilot phase of operations, the number of EASQs sent by the NCC to the EEOC offices has continued to increase. One year after implementation, the number of EASQs has more than doubled since the NCC's initial month of full operations. Table 22. Number of EASQs sent to EEOC Offices No. EASQs sent to EEOC Month/Year Offices April-05 1,979 May-05 1,676 June-05 1,923 July-05 1,748 August-05 1,950 September-05 1,578 October-05 1,612 November-05 1,788 December-05 1,843 January-06 2,668 February-06 2,472 March-06 3,821 April-06 4,276 Inaccurate or Incomplete EASQs The EEOC Project Manager encourages EEOC staff to report any NCC related problems to him or his staff. He indicated that he often works with field management staff to handle complaints regarding accuracy of EASQs. OFP reported that EEOC staff forward misdirected EASQs, misspellings, and any other problems to Headquarters and the EEOC Project Manager discusses them with NCC managers. These NCC Managers counsel CSRs who have made errors. Survey respondents commented in February that NCC forms often have errors. Investigators from one office agreed that EASQs are "threadbare" (inaccurate and/or incomplete) 85 percent to 95 percent of the time. One Office Director wrote: "More often than not, there are major errors in the information that has been provided by the NCC. Just two examples: 1) Shondra Weager is really Sondra Yeager. 2) PCP's Religion is Musselman. There are so many more examples. Addresses, telephone #s, and general information are often incorrect. We spend numerous hours receiving, responding, correcting and then re-entering information from EAS to IMS." During the JPS Team's focus groups, intake staff from several different offices reported that the NCC usually provides the name correctly, but that sometimes the phone number is wrong and that they often receive little useful information on the form. Inaccurate or incomplete EASQs may limit the extent of their usefulness. Survey respondents indicated that they use varying types of information on the EASQs, which may be related to inaccurate and incomplete EASQs and/or different intake methods in respective offices. Table 23. Extent Intake Staff Uses Information on the EASQ Information used on the EASQ Percent of Survey Respondents User information 73 Employer information 57 Complaint information 61 Never use the EASQ 24 As shown in Table 24, survey respondents reported that many of the EASQs are inaccurate, thus requiring more work by the EEOC office to fix the error. Twenty-four to forty percent of survey respondents involved in intake reported inaccuracies on many of the EASQs they reviewed over the previous three months. Table 24. Inaccurate EASQs Percent of Survey Respondents Believe All EASQs A few EASQs Many EASQs were were not were not Types of EASQ Information accurate accurate (1-20%) accurate (>20%) User Information Name 38 36 26 Address 35 35 30 Zip code 40 36 24 Phone number 38 38 24 Employer Information Full name 25 35 40 Full address 24 36 40 Complaint Information Description of action against person 38 37 25 in the Complaint Information section was not employment related Issue in the Complaint Information 29 41 30 section was not one of the bases covered by the EEOC Focus group participants in four of the seven offices that the JPS Team visited indicated they receive many incomplete EASQs. Survey respondents reported that they receive incomplete PCP information, employment information, and details about the basis of discrimination. As shown in Table 25 below, 25 percent to 47 percent of the survey respondents involved in intake reported that many of the EASQs that they have reviewed over the previous three months are incomplete. Table 25. Incomplete EASQs Percent of Survey Respondents Believe Types of EASQ All EASQs were A few EASQs were not Many EASQs were not information complete complete (1-20%) complete (>20%) User Information Name 51 24 25 Address 36 30 34 Zip code 37 32 31 Phone number 40 31 29 Employer Information Full name 23 33 44 Full address 20 33 47 The JPS Team independently audited 12 months (April 1, 2005-March 31, 2006) of EASQs sent to 3 offices to determine the accuracy and completeness of the user and employer information. The team was only able to compare those EASQs for which there was a matching record in the IMS. On average, 37 percent of the EASQs had errors in the PCP and/or Respondent fields (range 24 to 59 percent), and 20 percent of the forms had errors in just the user information fields (range 16.4 to 25.7 percent). There is no indication that accuracy has improved over time. In addition to user and employer information, focus group participants indicated that the complaint information is often incomplete, which has led office intake staff and supervisors to suggest that CSRs need to ask a few simple questions to provide some descriptive information on the form. Employees indicated that the issue in the allegation is often not clear on the forms and there is less information than staff expected. For example, if the form indicates that the basis is race, it does not indicate why the PCP thought it was discriminatory. As shown in Table 26, survey respondents indicated that they would particularly like to receive more information relevant to the basis for discrimination. Table 26. More Complaint Information Desired Percent of Survey Complaint Information Respondents* More details regarding what happened to the customer and why they 67 believe it to be discriminatory Specific behavior/conduct prompting the complaint 58 If the basis for discrimination is race, then provide the person's race 71 If the basis for discrimination is age, then provide the date of birth 73 If the basis for discrimination is disability, then provide the disability 70 If the basis for discrimination is retaliation, as on what basis they 72 were retaliated *Respondents checked all that apply Focus group participants suggested that if the CSRs asked just a few questions, the Investigators could make a decision as to whether to send a questionnaire, thereby weeding out cases that will not lead to a charge. One survey respondent wrote: "If a complaint is about national origin, do not go into race, but discuss what the issue is!!" As reported in one site visit, as the system is presently working, the EASQs often provide less information than a typical mail receipt and require additional time for Investigators to resolve the inaccuracies. Validating the above EEOC staff observations, while monitoring calls at the NCC, the JPS Team observed that the CSRs do little to inquire about and understand the nature of the charge. For example, if the basis that the caller selected for discrimination is age, the CSRs do not ask for age or date of birth to make sure the callers qualifies under that category. They leave it completely up to the PCP to determine whether his/her situation is discriminatory. Additional Information Needed As shown in in Table 27, survey respondents indicated they would like additional information on the EASQs. Table 27. New Information Desired in EASQ Types of New EASQ Information Desired Percent of Survey Respondents* Date of most recent incident of discrimination 77 Convenient time to contact the customer 56 Number of employees in company 71 Who the customer is using for comparison 61 *Respondents checked all that apply In addition, survey respondents wrote that they would like the following new information on the forms: PCP's Date of Birth and Social Security Number Language requirements if not English Staff in one office stated that the EASQs are of minimal value since they do not contain sufficient information to begin an intake interview (e.g., the alleged harm, when it occurred, PCP's date of birth, bases/issues involved, and why the PCP believes it is discrimination). GroupWise Email Forms The CSRs do not provide sufficient information on the nature of the complaint in the email. The GroupWise email forms only state that the PCP/CP cannot reach the office. The offices need more information, including who and when the person called, the reason they were calling, what happened when they called, and whether they called the Investigator directly. Because of this lack of information, EEOC supervisors do not know if there is a problem that needs to be corrected and what actions/procedures to correct. As shown in Table 28, more than 60 percent of the survey respondents involved in intake reported that the GroupWise emails they receive from the NCC are inaccurate and/or incomplete. Table 28. Accuracy and Completeness of GroupWise Emails Percent of Survey Respondents Believe A few GroupWise Many GroupWise GroupWise All GroupWise Emails Emails have problems Emails have problems Email were accurate/complete (1-20%) (>20%) Accuracy 8 24 68 Completeness 12 25 63 The JPS Team analyzed NCC-related GroupWise emails to one district office for January- March, 2006.[25] The team was unable to assess accuracy or completeness because many of the emails related either to information not in the IMS or to EASQs that were not yet updated in the IMS. The team did, however, identify that approximately 40 percent of the emails were from PCPs who were calling the NCC before allowing offices the allotted four to six days to process the EASQ. Receiving these emails and having to track down information adds no value to the end-to-end process and creates extra work for the office staff. Table 29. Content Analysis of GroupWise Emails GroupWise Email Subject Number Percent Transmit letter/fax/email 13 6.4 Complaints about office/Investigators 13 6.4 Language needs 5 2.4 Convey message to office (e.g., to set up appointment, change address, request 11 5.4 forms) Questions the NCC cannot answer (ADA, training, employer rights/questions, 19 9.3 attorney questions, insist on filing charge) Cannot call long distance 12 5.9 PCP/CP uses the NCC to leave message for Investigator instead of calling 13 6.4 directly Complaint — tried but cannot reach Investigator (usually do not explain what 10 4.9 happened e.g. how many times tried, whether left a message) EAS sent to wrong office (2 emails from NCC, 4 directly from other offices) 6 2.9 Case status 21 10.3 Second call before office response time elapsed (within 1-4 days of initial 81 39.7 contact to NCC) Total GroupWise Emails 204 100 4. "Hot line" transfers of callers (when the date of harm is within the 60-day window of the SOL expiring) are handled inconsistently. Procedures for callers at risk of losing their rights to file a charge (because they are close to the end of the SOL) differ from procedures for callers not at risk. When a PCP indicates a date of harm that is within a 60-day window of the SOL expiring (30 days on either side of the projected deadline for filing a charge), the CSR transfers the caller directly to the office designated to handle these "hot line" calls. Effective January 1, 2006 two EEOC offices were designated to receive hot line transfers. The EEOC added two offices to help with these calls on April 17, 2006. The four offices are in three different time zones across the United States and each is supposed to handle calls within a designated three-hour period, starting at 8 a.m. and ending at 8 p.m. EST; however, managers in two offices indicated they receive hot line calls all day long. To ensure only callers within the 60-day window are "hot" transferred, the EAS has a built-in calculator that automatically notifies the CSR when a caller should be "hot line" transferred to the appropriate EEOC office. When the EAS is not used, the CSRs use a desktop calculator to determine if the caller should be "hot" transferred to the designated EEOC office. Nevertheless, one office stated that 30 percent of the calls they received from January through April were outside the 60-day window, and 5 percent to 10 percent of the calls were not truly hot line calls (PCPs were inquiring about the status of a case that had a date of harm close to the expiration window). Managers in one EEOC office said that transferring these calls outside the designated window can create unrealistic expectations on the part of the caller and create extra work for the office because employees must stop everything to address an issue that is not truly urgent. There are presently no written procedures for how EEOC offices should handle hot line calls, and procedures vary across the offices. For example, Headquarters requires that a non- bargaining employee answer the initial call from the NCC. Three of the offices have a dedicated cell phone for the hot line calls, but in one office, the calls come in all day long on their regular office number and are answered by whoever picks up their main number. If their two extensions are busy, the call rolls to the next available person in the office. Some managers believe they are personally responsible for handling everything related to the call, which is very time consuming. Others take down basic contact information and pass the information to a bargaining unit employee to follow up. 5. Not all calls forwarded or referred to EEOC offices are related to filing charges. EEOC Workload Call volumes, types of calls, and call resolution (calls handled entirely by NCC) have an impact on the EEOC's workload. As discussed previously, current call volumes to the NCC are lower than anticipated and are not having a major positive impact on EEOC workloads compared to EEOC expectations. Forty nine percent of calls received by the NCC are sent (directly or indirectly referred or forwarded) to EEOC offices. Of these calls, 61 percent consist of people who wish to file a charge, and the remaining 39 percent are related to general inquiries, EEOC overviews, or questions about charge status Figure 4. Types of Calls Forwarded to EEOC Offices Numbers in the chart do not total 100 due to rounding. Streamlining processes and integrating technologies can help reduce the number of calls (38 percent of all calls) forwarded to EEOC offices that are not related to filing charges. Of the total calls forwarded to the EEOC related to filing charges, 38 percent are forwarded to EEOC offices by an EASQ and the remaining 62 percent of callers are asked to contact the EEOC office directly. When CSRs ask callers to contact the EEOC office directly, they send no information to the offices; therefore, the time spent by the EEOC in handling these calls is a duplication of work. These calls are handled by the EEOC as if they were first-time callers and all of the information collected by the NCC is collected again. The CSRs' practice of telling people to contact the EEOC office may create problems for PCPs who visit the office and desire to file a Charge. For example, particularly in the smaller offices and depending upon the staffing and the workload, Investigators may not be available when the person shows up, wasting the PCP's time. Overall, there remains an opportunity to reduce the number of calls sent to EEOC offices even if the volume remains constant. As stated previously, the NCC currently sends 49 percent of calls to EEOC offices. Following are areas where the NCC can further reduce these calls: Thirty nine percent (of the 49 percent) of calls that NCC sends to EEOC are issues not related to filing a charge.[26] Streamlining processes and integrating technologies would substantially reduce these calls. Another 38 percent (of the 49 percent) of the calls sent to EEOC offices relate to callers who inquired about filing a charge and made the first contact with NCC but were asked to contact the EEOC directly. The CSRs enter data regarding every call into RightNow software, a Customer Relationship Management (CRM) technology that documents all information related to calls. If the EEOC were to use the NCC's RightNow system, they would be able to access the preliminary information about the contact and thus reduce this duplication of work. The NCC is not resolving as many calls as they could, and therefore is having minimal impact on EEOC workload (1,164 hours saved per month, as shown in Table 18). F. INTEGRATION OF THE NCC AND EEOC Operationally, there is minimal direct communication between the NCC and EEOC field offices, as their technologies are incompatible. The NCC has provided some recent support to EEOC offices in identifying trends. EEOC staff experiences with the NCC, coupled with minimal change management practices, have resulted in unrealized expectations. 1. There is minimal direct communication/knowledge sharing between EEOC offices and the NCC, which inhibits integration of the organizations and processes. Supervisors in the field indicate that they receive little or no useful information generated by the NCC, NCC processes are not transparent, and there is no process for the Investigators to feel vested in the NCC. Prior to February 2006, the only regular written communication to the field about the NCC had been a monthly report to directors (this monthly report was also sent to all staff in the field in February 2006) describing the following statistics: Total calls handled, average handle time, IVR calls, and other types of transactions. The largest number of customer contacts, the number and type of foreign language calls, peak calling time, and number of hot line calls. Number of EASQs sent and to which offices, the number of customer concerns/complaints forwarded to offices, and the number of people whom the NCC provided with information about contacting the appropriate EEOC office. Number of visits to the FAQ page on the EEOC's website and most frequently visited topics. The EEOC circulated the inaugural issue of the EEOC National Contact Center Newsletter to all EEOC employees in February 2006. The newsletter included a section describing results to date, positive comments from a few field offices, training activities at the NCC, personal stories about two people who contacted the NCC, and a picture and short biographical description of two CSRs. Communication exists between the NCC Project Manager at Headquarters and Office Directors, Deputy Directors, Enforcement Managers, Intake Supervisors, and a few others. One District Director said that the EEOC Project Manager often contacts them with requests for feedback. The EEOC Project Manager regularly briefs District Directors and other field personnel when they have occasion to visit Headquarters. From time to time, OFP responds to inquiries from office managers and supervisors regarding the NCC. There is no formal process for EEOC employees to ask questions directly of CSRs or give feedback on how the NCC can better serve the offices. When a recent concern arose, one Office Director notified the district office of an urgent problem caused by the NCC but two weeks later had not receive feedback on how or whether the issue was ever addressed. To date there has been minimal direct communication between one EEOC office and the NCC when the office sent an email request for more information about incident reports, even though there was no established process for this direct communication. In another office, a manager would like to communicate directly with the NCC to ask more questions about information in EASQ/GroupWise emails. Investigators would also like to communicate directly with CSRs rather than through the formal system (Headquarters and GroupWise Email). For example, the NCC uses the GroupWise emails simply to convey that a party is returning the call of an Investigator (perhaps avoiding a long distance call), adding a layer of communication and disrupting the normal workflow. One office director and survey respondents indicated that they believe that the NCC adds another layer of bureaucracy for customers. Further complicating the relationship between EEOC offices and the NCC is that the GroupWise emails contain phrases suggesting that the offices are not doing their job properly. Survey respondents indicate that it appears the CSRs do not understand what is happening in the offices. Following is a written comment from one EEOC Office Director: "The emails come through auto-assigned to wording that says the office failed to do something. The standard language is 'fail to respond.' This creates misunderstanding and is frustrating for the offices because usually they are in the process of trying to reach the person and for one reason or another have not yet connected. First, the NCC should cross-reference with the IMS to see what is happening on the case. The language in the form and process should be revisited." There is no direct communication process between the NCC and EEOC offices; most communications are presently routed through the EEOC Project Manager. This inhibits the ability of employees in each organization to understand the activities and requirements in other organization. 2. NCC knowledge about EEOC offices is minimal, and the information flow is sporadic and inaccurate. NCC Staff Understanding of EEOC Objectives and Operations Interviews with NCC team leaders and focus groups with CSRs revealed that there is no common understanding of the NCC's program objectives. One Team Leader stated that the mission of the NCC is to "provide a general overview about the EEOC, information on discrimination, and forward escalations." Another Team Leader stated that it is to "handle 70 percent of the calls that the EEOC was receiving and document the types of calls received for trending purposes." While these two mission statements seem similar, their focus is completely different. The first one focuses on providing general information and forwarding escalations, and therefore does not put a great deal of emphasis on documenting the call so that the EEOC can identify trends. A CSR taking direction from this Team Leader will be more focused on listening to the customer. The second Team Leader focuses on providing information and documenting the types of calls. A CSR taking direction from this Team Leader will be more focused on documenting information than listening to the caller. This presents two very different customer experiences. While these two Team Leaders have different roles, they should have the same views on the mission of the NCC. The issue is that, throughout the day, CSRs may ask questions of one or the other Team Leader and get different instructions on how to handle calls. NCC management staff and CSRs said they had little insight into EEOC operations. They have regular visitors from Headquarters; however, few people from the field have ever visited the NCC. There are no standardized processes across the EEOC field offices. According to the team leaders, having contact with the field operations through either visits or regular phone meetings could improve the effectiveness of operations at the NCC. NCC/EEOC Communication Interviews and focus groups at the NCC have also revealed that the CSRs have little insight into the use and validity of information that they provide to EEOC offices. Several CSRs stated that there is no feedback from the field and that they would like to hear when there are inaccuracies in the information they send so they can make improvements. The only feedback these CSRs mentioned related to spell checks, and so they had the perception that everything else in terms of accuracy was fine, but, as discussed earlier, the JPS Team's findings contradict this impression. 3. Many EEOC and NCC technologies are incompatible, which adds to the workload of EEOC offices. There is no integration of technologies between the NCC and EEOC. The CSRs capture a great deal of relevant information about the transaction into RightNow. If the caller wishes to file an EASQ, the CSR must transfer the information to the EEOC Assessment System (EAS). The NCC presently has "read only" access to the EEOC's IMS. This fact, combined with a lack of integration between IMS and/or RightNow and EAS, results in the requirement that EEOC staff enter information from the EASQ into the IMS. When callers want to get a message to offices or have a complaint, the NCC puts the information into the RightNow software and then separately sends a GroupWise email to EEOC offices. The EEOC offices send information back to the NCC describing disposition of the complaint, which the NCC then records in the RightNow software. Further complicating this issue is that some EEOC offices respond to the NCC with the disposition of each specific issue, and others send back an almost automatic response indicating that the general issue was addressed. In the latter instance, the CSRs never record correct information into RightNow and never have an opportunity to understand or, if the caller contacts the NCC again, explain what is happening at the EEOC office to the customer. The two organizations are maintaining parallel information systems that are not communicating with one another. As a result, inefficiencies exist at NCC and the EEOC. There is duplication of data entry for the NCC (RightNow and EAS), duplication between NCC and the EEOC (EAS and IMS), and the EEOC cannot see the string of information maintained by the NCC in RightNow. This causes the EEOC to have callers who have already spoken to the NCC repeat their entire story, since the EEOC does not have the benefit of information recorded in RightNow, creating a frustrating experience for the customer. The lack of integrated technology between the NCC and EEOC has led to a duplication of work and added to the reasons that the NCC has not reduced the workload on EEOC employees. The EEOC's Office of Information Technology (OIT) plans to develop a link between the EAS and IMS in the near future so that the EEOC's field offices will not have to continue performing double entry of data for inquiries referred by the NCC. Escalations The lack of technology integration also prohibits the NCC from viewing the status of inquiries until they become a charge, leading to emails that escalate issues that EEOC offices are already handling. This causes duplication of work because EEOC employees now have to attend to these emails and provide an explanation back to the NCC, which the NCC then records in the RightNow software. As described earlier, EEOC business rules prohibit the NCC from disclosing information about charge status unless the caller can provide the charge number and one other piece of identifying information such as home address, phone number, or date of birth. During the team's site visit to the NCC, the CSRs indicated that they require a charge number to look up information in the IMS. If the caller does not have a charge number, which occurs often, the CSR must refer him/her to the EEOC offices; therefore, the NCC forwards many calls related to charge status to EEOC offices. Communication with the staff at Headquarters indicated that the NCC has exactly the same access as the EEOC offices in terms of looking up information in the IMS, i.e., they can look up charges several different ways including by CP first and last names, Social Security Number, receiving office, and Investigator name. CSRs have the same "read" access to the IMS as the field offices, but under EEOC business rules, they are not allowed to reveal any information to callers without their charge number and one other piece of identifying information such as home address, phone number, or date of birth. Since CSRs do not have access to any other system the EEOC uses or any way of knowing EEOC information that the EEOC has not recorded in the IMS, they sometimes send escalations that are truly duplication of work. This adds to the EEOC workload. Integration of all of the EEOC's data systems has been a long-time goal, and the EEOC plans for integration of the EAS and IMS for late 2006. The JPS Team is not aware of any discussions to integrate the RightNow software into EEOC systems. 4. The NCC processes are not adequately set up to identify trends. One of the NCC's objectives is to provide the EEOC with trends such as geographic and demographic information about the callers and reasons for their calls. One Office Directors reported that the NCC reports they currently receive are not much help and that they need data relevant to their region. They also indicated concern about any trends due to inaccuracies in forms they receive from the NCC. Assuming inaccuracies are corrected, the trend information the offices would like to receive is provided in Appendix I. Some of the trends directors requested are currently available by querying the IMS or in monthly NCC reporting. Some directors were not aware that they have access to some of the information that they requested. Over the past year, the NCC has identified a new trend: the NCC forwarded pre-employment discrimination to the EEOC as a new type of call for which there was no previous information within either the FAQs or the scripts. This led to the EEOC developing FAQs for its website and scripts for CSRs. Review of the reports generated by the NCC shows that there is no automated method to identify trends. The NCC sends monthly reports to the EEOC that are a snapshot of that month. There is no month-by-month data analysis to identify trends in terms of callers, reasons for calls, or call dispositions. Collecting and reporting demographics were important trends cited by most of the NCC staff. CSRs collect demographic information on all calls; however, none of the reports show these data. The NCC recently began providing some demographic information, but even these reports are not set up to identify trends proactively. Even though the EEOC has identified some trends based on questions raised by CSRs, there is presently no methodology or logical approach to identifying trends. 5. EEOC employees have concerns and unrealized expectations about the NCC Change Management During the JPS Team's site visits, office management reported that there was not enough up-front communication about why the business decision was made to implement the NCC, so initially the role of the NCC seemed undefined. One Office Director commented that the NCC should have field staff concentrate on identifying low priority Category C cases that are usually dismissed after only minimal investigation. Identifying Category C cases before Investigators spend time investigating should increase their efficiency and effectiveness. Survey respondents confirmed that they recollect hearing information about the goal to reduce office calls. One survey respondent wrote: "I can't remember ever having been advised on the major goals of the NCC, other than to reduce the calls that come in to EEOC offices." As shown in Table 30, survey respondents indicated that they generally understand that the purpose of the NCC is to reduce call volume to offices in the field and provide general information to callers. Fewer respondents believe that the NCC is supposed to refer calls not in the EEOC's jurisdiction and less than one third understand that another purpose of the NCC is to gather trend information. Table 30. Survey Respondents who Understand the Role of the NCC NCC Role in Enforcement and Outreach Percent Survey Respondents* Reduce calls requiring field office attention 81 Provide general information about EEOC law, and potential 83 charging party's options Handle calls not in EEOC's jurisdiction 58 Gather trend information on employment issues affecting the public 31 *Respondents checked all that apply Even though employees understand the NCC's purpose, focus group participants and survey respondents indicated that they still do not understand why the EEOC believes that the NCC is a better business decision than allocating the money to hiring staff and upgrading the infrastructure, such as the telephone technology. Field office management staff and survey respondents also indicated that it would be more helpful to have additional staff than to spend the money on the NCC. Unmet Expectations Office managers believe that the concept of the NCC has been "watered down" and would like confirmation that the NCC is only supposed to provide "bare bones" support. The JPS Team's survey and interview data suggest that staff in the field expected more from the NCC than is presently being delivered. The staff may have expected more from the EASQ than it was designed to deliver. Management and staff anticipated that the CSRs would conduct more extensive screening, provide better quality of information in the EASQs/emails, and require less from field staff due to the NCC handling initial screening. They also expected that inquiries generated by the NCC would take less time and that the completed EASQ would be a step above the intake questionnaire (once the questionnaire is completed, PCPs could essentially write their own charge). Since the EEOC never meant for the EASQ to replace the intake questionnaire, some EEOC staff may have misinterpreted the reason for the EASQ. Training CSRs receive two weeks of classroom and on-the-job training on EEOC jurisdiction (including application and interpretation of laws enforced by the EEOC), repositioning, antidiscrimination laws, identification of issues and basis for discrimination, Title VII and other statutes, and Federal sector cases, as well as how to handle irate customers. Survey respondents indicated a belief that CSRs need more training on these issues. In addition, respondents to the director survey wrote that the CSRs need more training on the application and interpretation of laws enforced by the EEOC. On the JPS Team's EEOC site visits, managers and supervisors indicated concern that the CSRs need to understand what they are talking about; anyone can read a script. Managers also would like assurance that CSRs understand what they should and should not do so that they do not jeopardize the EEOC's ability to expand the charge and do not turn customers away because they do not ask the right questions. Survey respondents also suggested refresher training on common issues such as geography; as an example of this need, CSRs have sent information to the wrong offices due to confusion of AR (Arkansas) with Arizona (AZ) and the State of Louisiana (LA) with the City of Los Angeles. G. NCC IMPACT ON EEOC CUSTOMERS With the launch of the NCC, customer experience has improved, particularly in areas related to access. EEOC offices have modified procedures to reflect the NCC activities, driving an increased focus on customer service. There is an opportunity to improve customer satisfaction further by streamlining existing and adding new procedures. The EEOC's Customer Satisfaction Index of 77 is strong. 1. While field office intake procedures vary, they provide timely customer service. EEOC policy is to give offices flexibility to establish their own intake procedures that meet staffing needs and other unique demands of their respective offices. As a result, there is a wide variety of intake methods across the EEOC offices. Walk-in Customers with No Risk of the SOL Expiring Most offices (46) provide initial screening by an Investigator or supervisor, if available, while the PCP is in their office or later the same day, although respondents to the director survey acknowledge that this is not always possible. The remaining five offices do not have sufficient staff to see all walk-in customers, so they provide initial screening by an Investigator Support Assistant, give the PCP a questionnaire to complete, and/or make an appointment for them to return on another day for screening by an Investigator. Telephone Calls to EEOC Offices For people calling the EEOC office to file a charge, 39 offices provide screening by an Investigator Support Assistant or an Investigator. The remaining offices may perform a quick screening by the Receptionist and then send an intake questionnaire and/or have an Investigator return the call within 24-48 hours for more thorough screening. 2. Post-NCC, service to customers has improved in some areas. NCC as a Customer Service Initiative Respondents to the office director survey reported best practices related to the NCC including tacking all NCC forms, making personal telephone calls to NCC contacts and in one instance, ensuring customers are called and information is entered into the IMS within 24 hours after receipt. According to one District Director, the NCC has institutionalized the delivery of customer service, and now the customer service orientation happens every day; rather than being a one-time initiative. For example, before the NCC, Investigators in one office would take up to eight workdays to respond to an inquiry. Now Investigators in that office are required to make two attempts to contact the PCP within 48 hours after receiving the EAS, each at different times of the day (e.g. morning and afternoon). Foreign Language Service The NCC provides multi-lingual availability on the initial call. Spanish speaking customers can immediately reach someone who speaks their language and are given the direct line of the bilingual person in the EEOC offices, adding focus to the outreach and supporting underserved populations. In the future, the NCC's multi-lingual service could potentially help Investigators with translation services. Complaints about Offices Though offices can accept collect calls to ascertain callback information and then return the call, some offices reported that no longer having a toll-free number is a concern, since some customers do not want to pay for or cannot afford to place a long distance telephone call. This issue is relevant to investigations and litigation. Several survey respondents commented that not having access to a toll-free number has created a barrier between the EEOC and CPs because of the long distance expense. One Office Director reported no differences in general complaints pre- and post-NCC, mostly due to antiquated telephone technology. OFP is exploring the feasibility of establishing unpublished toll-free telephone numbers for use by CPs and witnesses involved in cases. OFP is presently evaluating the cost and potential impact on the NCC. 3. While offices have different intake procedures, some say they have modified and improved their procedures because of the NCC. With the NCC as the initial step in the intake process, offices have reviewed their intake procedures and provide back-up systems to front line people, creating an employee sense of urgency in responding to the public. Offices have also improved the method by which they track PCPs by recording receipt and disposition of EASQs and GroupWise emails, either in an electronic (IMS and/or the GroupWise log assignment system) or paper form. 4. Customers have significantly improved access to the EEOC. Launching the NCC has added the following new channels (ways) for customers to contact the EEOC: English Line Spanish Line Tele Interpreter TTY Line Web Inquiry Email Fax Written Correspondence EEOC employees report another benefit of the NCC is that customers have access to a live person on their initial call for more hours of the day rather than having to leave a voice mail and not receiving a return call until the next day. In addition, each PCP immediately hears from an office by phone or mail.[27] CSRs can also listen to callers who want to vent their frustrations. The team identified an opportunity to improve the average speed of answer, as the contractual metric of answering 70% - 80% of the calls in the first 30 seconds was only met 5 out of the past 8 months. The improvement in customer access is limited to those customers contacting the NCC. 5. Customer experience is not consistent when they call the NCC. It depends upon the CSR reached. Customer Experience Due to the lack of integrated procedures, customers may sometimes end up confused about their interaction with the NCC. For example, when customers call the NCC, they do not understand that it is a separate organization; they perceive their communication as 'contacting the EEOC.' If the transaction is not properly documented and not everyone (within the EEOC and NCC) interacting with the customer has access to that information, there may be possible lost customer information or confusion about the charge filing status. As described earlier, callers who explained their situation at length to a CSR have indicated to Investigators that they are confused about having to repeat the same information. Differences in how CSRs view their roles also create a widely varying customer experience. Some CSRs believe their role is to act as a sounding board and listen patiently to the callers about their discrimination stories while others think their role is to take the preliminary information and move on to the next call. Those who feel they have to listen to the customer are patient and attend to all details, focusing on the customer. This provides a very different customer experience from a caller who reaches a CSR who is solely focused on collecting initial information. While monitoring calls, the JPS Team noticed that some CSRs were empathetic and patiently listened to their customers. Other CSRs were focused on capturing and documenting information and not on listening to the caller. The latter experience, as we observed, felt as though the phone call was just a routine process to fill out a form. Customers calling for general and/or EEOC overview information also have varied experiences, depending on their particular questions. If the CSR is able to locate the answers easily, he or she gives the customer the information and the process ends; however, if the CSR is unable to locate the answers easily, he or she asks the customer to call the EEOC office directly, at which point the experience becomes the same as pre-NCC. Figure 5 shows the types of calls received by the NCC and where they ended (EEOC offices or NCC). The NCC could handle all of the calls related to Eligibility Criteria, Charge Status, and Filing a Charge, but, as can be seen in Figure 5, the NCC is handling only a portion of these calls. For example, only 57% of the calls relating to EEOC overview and/or eligibility criteria are completely answered by CSRs; the remaining 43% are asked to contact an EEOC office. Figure 5. Call Types by the Organization that Resolved the Call Customers calling to file a charge also have varied experiences. Some CSRs lead customers to fill out EASQs, while others lead customers to contact the EEOC office. Only 35 percent of the customers who called to file a charge actually completed the EASQ. All they have to do after the call is wait four to six days for a contact from an office. The remaining 65 percent were asked to contact the EEOC office directly. These customers must continue to try to contact an EEOC office and find someone to help them. These customers may believe that they have filed a charge with the NCC. According to the customer satisfaction survey conducted by CFI (Claes Fornell International) Group in February 2006, 34 percent of the respondents indicated they thought they had actually filed a charge because of their communication with the NCC. OFP recently advised the JPS Team that they had changed CSR scripts to include language advising callers that they cannot file a charge over the telephone. The JPS Team listened to call records from the NCC and did not hear a CSR convey this information. This may be because CSRs sometimes do not follow the correct scripts, as reported during the live call monitoring (discussed later). Customers who do not complete the EASQ have to contact the EEOC offices directly. Their experience would be no different than it would have been pre-NCC, e.g., calls not answered by a live person, mailboxes may be full, etc. 6. Call handling methods are usually consistent, but not soft skills.[28] CSRs are given call flows for public and private sector calls, and they consistently follow them; however, the soft skills side of handling calls is not consistent (e.g. courteously answering questions while maintaining call control, asking the right questions depending upon the customer tone, and steering the conversation). For support professionals, soft skills refer to anything that falls outside of the traditional product and development skill set. Soft skills are the ability to communicate effectively, present ideas, solve problems, and provide excellent customer care. Some CSRs are patient, listen to the customer for a long time without controlling the call, while others are impatient, and cut off their callers. In both cases, the CSRs have not mastered how to control calls. The NCC has pushed for the handle time (length of the call) to be under six minutes, but provides little coaching and/or training on how to control the call. There is minimal focus on improving the CSRs' soft skills. 7. Information about general overviews and inquiries is fairly consistent and accurate; however, other specific information is not consistent or accurate. The NCC uses standard operating procedures to update the script content, which ensures information accuracy, but there is no focus on ease of use by the CSRs. CSRs provide customers with consistent, accurate information regarding charges and other general inquiries, but specific information relating to infrequently asked questions is inconsistent. Most CSRs perceive that they have 800 scripts to reference, but, in reality, they have a subset depending upon the type of call that they are handling. According to the Site Director, there are 400 Spanish Language scripts and FAQs and 400 English Language scripts and FAQs. Unfortunately, in this case, perception is reality. Since CSRs perceive that there are 800 scripts, they do not take time to understand and use all of them; instead, they select a few scripts that they use most often and refer only to those for all answers. The result is that, when a customer has an infrequently asked question, some CSRs try to fit one of the answers into their most frequently used script, while others refer the customer to field offices. CSRs and one of the Team Leaders stated that the key word searches are not efficient; for example, during a side-by- side observation, one CSR said that when she typed 'Right to Sue,' nothing appeared. 8. The Customer Satisfaction Index is very strong. The EEOC recently contracted with CFI Group to collect customer satisfaction data on NCC calls. According to the results, the overall Customer Satisfaction Index is 77. The JPS Team conducted a benchmarking analysis by comparing this index to the American Customer Satisfaction Index (ACSI) scores for the Federal government. The overall Federal government score ranges from the high 60's to low 70's. A few low 80's exist, but they are rare, making the EEOC's score of 77 relatively strong. Service industries in general tend to score in the 70's, unless they are premier companies (e.g. Nordstrom's or LL Bean) that base their overall reputation on a superior service model. For comparison purposes, the overall ACSI score across numerous service industries stands at 73.5 through the fourth quarter of 2005, putting the EEOC above both the national average and the average for the Federal government. As shown in Figure 6, the EEOC's Customer Satisfaction Index of 77 is above the satisfaction indices for the total Federal government and the overall ASCI Index. Calls coming into the NCC are more comparable to calls coming into Federal Trade Commission (FTC) and Federal Citizen Information Center (FCIC) than CMS-Medicare, and the EEOC's score is less than those two indices. Figure 6. 2006 ASCI Index and Federal Benchmarks[29] The satisfaction index for other forms of correspondence is significantly lower than the satisfaction index for calls. Figure 7. EEOC Customer Satisfaction Index In summary, the overall Customer Satisfaction Index for the EEOC is 77, higher than some Federal agencies. The satisfaction index varies by channel,[30] with the web channels having significantly lower scores. An overall score of 77 is good, but, since there are no baseline data, there is no way to know whether this is an improvement over pre-NCC. In addition, these data reflect only the customer experience with the NCC, a narrow measure of the overall EEOC customer experience. It does not reflect satisfaction with the EEOC in general or with everyone in the end-to-end process. Examining customer experience at the NCC evaluates only the initial step in the customer experience, and does not reflect, for example, any dissatisfaction from customers who misunderstand the NCC's role. As described earlier, Investigators indicated that PCPs have thought that they filed a charge with the NCC and experienced frustration at having to repeat their information to Investigators. 9. Some of the contact center operational procedures at NCC are not optimal. This is not having a significant impact on the customer experience at the current volumes; however, with increased call volumes, customer experience can be affected. As stated earlier, there have been changes to operating procedures at the NCC and EEOC offices since the beginning of this project. Most of the assessment of the NCC is based on the findings identified in November and December 2005. Work Environment and CSR turnover rate (attrition) The work environment for CSRs at the NCC is very good. All the CSRs have their own workstations. The floor is well lit and spacious. The team leaders are readily available to the CSRs. The current CSR rate of turnover at the NCC is 36%. Industry averages vary by the type of industry served by the call center (e.g., technical support, customer service, and sales) and the nature of the call. According to a Pearson Study, the CSR turnover rate in Government call centers averages 29 percent. The NCC has access to state of the art technology in terms of scheduling, forecasting, and timekeeping, but they do not use these technologies efficiently. Table 31 provides an overview of each of the relevant technologies, with in-depth descriptions below. Table 31. NCC Processes and Technology Process Technology Scheduling Blue Pumpkin/Aspect Time Keeping Deltek Call Management Avaya - CMS Contact Management RightNow General Reporting No particular tool. NCC uses reporting from the source systems Agent Performance No particular tool. NCC uses reporting from the source Management systems Call Monitoring NICE Recording Software[31] Scheduling—Blue Pumpkin/Aspect The NCC used Blue Pumpkin for scheduling purposes until recently, and are in the process of converting to Aspect. The NCC does not use all the capabilities of these technologies. Information is manually recorded on Excel spreadsheets and then entered into the system. With the current size of the NCC, this is not a large concern, but this is not an efficient way to schedule CSRs and it will pose many problems if the program expands. Timekeeping - Deltek The NCC uses Deltek for timekeeping purposes, but the current process requires a significant amount of manual entry and is recorded in three different places: an Excel spreadsheet, the timekeeping system, and paper copies. The NCC was unable to provide a valid reason for these manual activities. Aux codes (codes generally used on switches to track CSR time) are not used, resulting in manual tracking of CSR times. The duplication of this process adds inefficiencies to NCC operations. Call and Contact Management/General Reporting - Avaya and RightNow The NCC does not use a special reporting tool. Reports are produced from the source systems: RightNow (CRM), Avaya (Switch), and tele-interpreter reports from the web. Reporting is provided as follows. Switch Reports - Daily, weekly, and monthly reports on call attributes, such as speed of answer and abandoned calls. The EEOC receives these metrics on a monthly basis. Interval reports (call metrics for every half hour) are produced and used to manage the floor in a typical call center, but these reports were not available for this program until recently. CRM Reports from RightNow - These reports show contact by State, channel, action taken, topic, etc. The EEOC receives these reports on a monthly basis. Analysis across months would allow the NCC to identify trends, but no such analysis is currently performed. In addition, the NCC collects demographics but did not report them until recently, when it began sending demographic reports to the EEOC; however, even these are snapshots and do not present any trends. Other Reports - IVR reports and reports on correspondence and emails are available. Agent Performance Management - No tool is used Agent performance management is not standardized. The Site Director said that team leaders and staff members have meetings once a week to discuss monitoring scores, changes, and general questions, but CSRs denied this. CSRs also said they do not have any one-on-one coaching sessions. Their coaching is more like refresher training. There is also no reporting relationship between CSRs and team leaders; CSRs can approach any team leader with any questions. The NCC does not use any agent scorecards or other performance metrics to review CSR performance. The only measurement used is through call monitoring, which the CSRs say they receive once every two months. Call Monitoring Call monitoring is an essential and important process for any call center. Call monitoring ensures consistent quality service to customers by regularly coaching CSRs on findings from call monitoring. At the NCC, the call monitoring process consists of three distinct activities: NCC call monitoring, EEOC live call monitoring, and NCC and EEOC call calibration meetings. NCC Call Monitoring. The NCC uses two tools for call monitoring: CMS Supervisor and NICE The NCC quality manager and supervisors monitor the calls of each CSR four times a month (as required per the contract). They use a specific form to grade each call that they monitor. These grades are then summarized and shared with the CSRs; however, according to the CSRs, information from the monitoring sessions is not consistently shared with them. During focus groups, some CSRs said they get one form per month, while others said they have been there for a few months and only received one. There is no documented process to show that CSRs have been provided with feedback. The quality monitoring forms include performance measure terminology that is not defined on the form. Standard definitions exist, but neither management staff nor CSRs could provide any definitions for the measures during interviews and focus groups. EEOC Call Monitoring. The EEOC conducts live call monitoring. The live call monitoring process started in December 2005 (technology problems precluded starting earlier). The EEOC has a remote call monitoring number. Depending upon the time that the EEOC connects to the call, they could be waiting on the line for the CSR to answer as long as the caller waits, or the caller could be waiting longer. The EEOC monitors 15 to 20 calls per week, and an average of 3 to 5 calls at different times throughout the day. When the EEOC identifies problems, they forward the information to NCC managers and supervisors for feedback to CSRs and appropriate corrective action. The EEOC Project Manager also discusses issues with the NCC content and training manager when it is appropriate to revise scripts or provide additional training. The EEOC has occasionally recommended that CSRs call customers back to correct information. As shown in Table 32, during live call monitoring from January through May 2006, EEOC staff identified errors in technical information (e.g., errors in the law or other substantive matters) in 22.5 percent of the 285 live calls monitored. In addition, during 11.2 percent of the total calls monitored, CSRs created unnecessary barriers for customers. Following are a few examples of such barriers: CSRs have told callers that they cannot hire an attorney until they file a charge with the EEOC CSRs have told callers that certain EEOC offices are not taking calls from the public, which was independently verified as incorrect CSRs require the complete and exact address of the employer to proceed with the phone interview when, by EEOC regulations, the employer's address is not required to file a charge Live call monitoring also identified an issue in the complaint policy. If customers complain three times that a field office did not contact them, the NCC must notify OFP. This has delayed filing charges for more than 34 days. In addition, when verifying repeat callers, CSRs do not follow any "confidentiality" procedures. The CSR may read from the file and ask the caller if the address in the file is correct, rather than asking the caller to provide his/her address. CSRs also enforce a "30 second hold rule," whereby they tell customers they cannot hold more than 30 seconds to allow the caller to retrieve additional information. The rationale is based on the performance metric that CSRs must answer calls within 30 seconds, which is not always the case. Table 32. Live Call Monitoring Results Percent of Live Calls Monitored Errors in Customer Complaints Script not Call more CSRs Technical Service about Field Followed than 6 create Month No. Information (CSR rude, Offices minutes* barriers) (2006) Calls Provided not polite) January 43 27.0 11.6 4.7 7.0 41.0 4.7 February 60 15.0 11.7 6.7 5.0 48.3 13.3 March 75 21.3 22.7 8.0 13.3 36.0 9.3 April 60 33.3 25.0 18.3 25.0 38.3 11.7 May 47 14.9 8.5 10.6 38.3 48.9 14.9 Total 285 22.5 16.8 9.8 17.2 42.1 11.2 *Excludes wait time. April through part of June 2006, a technology problem caused a customer to have to wait to talk with a CSR. The time ranged from 7-25 minutes. The problem has been resolved. EEOC and NCC Call Calibration Meetings. Each week the NCC and EEOC hold calibration sessions to understand quality performance metrics. This information is then loaded to an Access database to produce reports. The JPS Team requested but did not see any examples of the quality monitoring reports, although, according to management, these reports do exist. According to the reporting analyst, there is no good reporting from this quality monitoring. The only metric that is used is the number of times each CSR is monitored. H. SUMMARY 1. Implementation of the NCC has created some benefits, but not to the extent anticipated in the 2003 Assessment Report. As shown in Table 33, the NCC has made some progress in a few areas toward meeting the anticipated benefits set forth in the 2003 Assessment Report. As a state-of-the-art customer service solution, the NCC provides another channel for the public to contact the EEOC and customers have reported above average satisfaction with the NCC. The NCC, however, has not realized many other proposed benefits. As described earlier, this is partially due to the JPS Team conducting the evaluation during the pilot year of operations when new processes were being refined. In addition, initial business rules limited the calls to the NCC to only the two general information toll-free lines. The reduced benefits are also related to operational factors. The standard forms from the NCC are often inaccurate and incomplete, causing extra work for intake staff. There has been some positive impact on Investigator work, but less than anticipated. Table 33. Comparison of Benefits in the 2003 Assessment Report to Current Findings Anticipated Benefits Cited in the 2003 Assessment Report Findings of Current Evaluation 1 Provide a centralized point of contact for the The NCC can handle calls, faxes, emails, TTY EEOC, using multiple modes of calls, and mail correspondence; however, the communication. EEOC offices still take calls directly. 2 Improve the quality and effectiveness of A recent study reports the EEOC's customer customer service. satisfaction is above average, but without baseline data, there is no way to know if this is an improvement compared to pre-NCC. 3 Answer calls immediately by live bodies. CSRs answer calls to the NCC immediately; however, most of the callers desiring to file a charge are referred to the EEOC offices, relegating them into the same pre-NCC process. 4 Spot trends and emerging issues (i.e., logging, NCC reporting does not show significant tracking, and monitoring of inquiry nature, identification of trends. There were no monthly volume, duration and resolution coupled with trending reports. extensive reporting on demographics). 5 Meet e-government expectations. The EEOC has implemented the NCC; however, the technology across the NCC and EEOC is not integrated, and some of the reporting at the NCC is manual. An additional, albeit unanticipated, benefit has been that the NCC provided the EEOC with the capability to communicate with employees and PCPs/CPs after Hurricane Katrina in the Gulf States and with parties during the transportation strike in New York City in December 2005. The pilot phase of the NCC has not resolved the question as to whether the NCC is an effective customer service solution for the EEOC, though there are some indications that it is beginning to have a positive impact. It is clear that: NCC operations are not as efficient or as effective as they could be, largely because EEOC and NCC systems and processes are not integrated The EEOC has yet to obtain many of the anticipated key benefits The JPS Team's electronic survey, sent to all staff in field offices, provided two opportunities for respondents to write in comments. On the item that asked for suggestions to improve the NCC, approximately 18 percent of the 377 respondents writing in comments stated that the NCC is a waste of money and should be disbanded. At the end of the survey, respondents were given a final opportunity to enter any thoughts related to the NCC. Approximately 24.5 percent of the 267 write-in comments indicated that the NCC is ineffective and has not reduced workload or telephone calls, and that they would rather have more staff in the field.[32] Employees also indicated that if this is not possible, it is important to improve the processes and eliminate the duplication of work. Two of the most commonly suggested methods for achieving this goal included providing training to the CSRs and ensuring that CSRs transmit more accurate and complete information to EEOC offices. During the team's visits to EEOC offices, many managers and Investigators said that they would rather have more staff than the NCC; however, if that is not possible, they expressed a desire to make the NCC work more effectively. Respondents to the director survey suggested that the NCC issue intake questionnaires and take charges. One director wrote the following comment: "To simply refer PCPs to servicing offices for Investigators to speak with them or to send EAS questionnaires that are not detailed enough to be considered minimally sufficient charge information is not as helpful as the NCC could/should be. If a PCP wishes to speak with an Investigator, there should be someone on staff to handle that. If they wish to file a charge, questionnaires should be mailed out with self-addressed envelope to servicing office. There is great potential for the NCC to relieve workload for field offices." During the team's site visit to the NCC, the managers indicated they are aware that they are not as effective as they could be (the volume of telephone calls is significantly less than they anticipated). They also indicated a desire to communicate directly with EEOC offices to learn what they can do to help the offices. The CSRs also have a keen interest in learning more about how they can better serve the EEOC offices. 2. NCC Costs The JPS Team attempted to quantify the NCC's benefits; however, it is very difficult to conduct a cost/benefit analysis, since no initial business case was made for the NCC. The team developed a high-level comparison of costs (associated with the NCC's operations) with time saved at EEOC offices, but the financial element is only one component of the analysis. It is also important to understand the qualitative benefits offered by the NCC, because some of the EEOC's anticipated benefits cannot be quantified. For example, it is very difficult to quantify benefits resulting from customer satisfaction or the ability to identify trends that enable the EEOC to better serve future customers. In addition, there may be indirect savings over the current operations resulting from standardizing processes and making other improvements. The EEOC can redirect these savings to other mission critical activities, such as enhancing the efficiency and effectiveness of filing charges and the quality of investigations, mediation, and litigation, as well as preventing discrimination through outreach and training. The team estimated the amount of time the NCC is saving EEOC staff by combining the resolution rates estimated from its call observation analysis, average talk time from NCC reports, and the number of worked hours. Based on this estimation, the NCC is currently saving the EEOC approximately 13,964 hours per annum, the equivalent of 6.71 FTEs. The value of the current savings is $489,830.[33] Following are the NCC costs for taking calls in English, Language support, and TTY. Table 34. Costs of Calls Handled by CSRs Description Costs English Calls $1,145,300 Language Support $82,109 TTY Calls $3,802 Total Cost for CSR-Handled Calls $1,231,212 IV. RECOMMENDATIONS The JPS Team believes that the NCC has the potential to make a significant contribution to the EEOC; however, as presently operated, it is not effective. The team recommends that the EEOC continue with the NCC, but only if significant changes are made to improve call volume, optimize customer satisfaction and operational efficiencies, measure on-going performance, and ensure readiness for the future. The JPS Team's study found that the NCC is not achieving many of the benefits originally anticipated in the 2003 Assessment Report, partially due to business rules developed after the Report was created, which has led to a call volume that is much less than originally estimated. The NCC has freed up some field staff time, but not to the extent anticipated. Significant issues remain concerning the quality and value of the information that the NCC provides to the field; therefore, the team believes that it would be a mistake for the EEOC to simply renew the contract and continue current operations. One option is for the EEOC to abandon the NCC, as some of the changes that the team recommends will require the EEOC to commit time and resources to achieve a successful outcome. The largest negative result of eliminating the NCC would be its impact on customer service. The largest positive impact of eliminating the NCC would be that of potentially freeing up money for other uses, including upgrading telephone technology in field offices, hiring more professional staff to help with investigations, and hiring more clerical staff to help answer unsolicited telephone calls and perform other intake duties. If the EEOC does not hire additional staff and does not upgrade telephone technology, customer service may deteriorate to conditions at the time of the 2003 Assessment Report. Calls may be dropped, customers may have to leave a voice mail message and wait to have their questions answered, and customers will have fewer hours during the day to talk with an EEOC representative. Further, office efficiency and effectiveness may suffer if professional staff is again required to support clerical staff, diverting them from other mission critical tasks. As a result, office complaints may increase, which will take more time from Office Directors. In addition, the ability to quickly identify and react to trends may be limited. Recommendations to Improve Operations Quickly 1. Increase the NCC's call volume by routing all initial calls through NCC and increasing customer awareness. Route all initial calls through NCC. One of the main reasons that the NCC is not having a more significant impact on the EEOC's workload is low call volume; to increase this volume, the team recommends that the EEOC route all unsolicited calls to the NCC. This would involve making local numbers available only to people who have existing relationships with the EEOC offices. The purpose is to reduce call volume to EEOC offices and lighten the workload. This would also ensure that callers have a similar customer experience, since services provided by the NCC are regularly monitored for quality service. Customers calling about matters related to existing charges and cases would still have the local office telephone number and continue to be able to reach the appropriate person directly. Develop methods to increase awareness of NCC. Increasing public awareness of the NCC would likely increase NCC call volume. The planned advertising campaign will be important to increasing awareness of the toll-free number, and plans to implement this campaign in early July should proceed. The EEOC should update its marketing plan (National Contact Center Outreach Plan updated February 28, 2005) to incorporate planned activities for the current and upcoming fiscal years. The EEOC should also track this document for progress. In the present version, action items have an "end date," but it is not clear whether those action items have been implemented. One item is indicated as "done", while the others simply state dates, and not whether they have been completed. The EEOC should also modify its home page to display clearly the NCC's importance and relevance to customers. The website presently shows the NCC under the banner and under the link for "Contact Us." Showing customers the benefits of contacting the NCC will encourage them to do so. It is important that the EEOC follow through with GSA to ensure that all telephone directories list the toll-free number as the EEOC contact number. The team recommends that the EEOC consider publishing only the toll-free number in the blue pages. Offices or the NCC can give the local number to CPs and other customers who have existing relationships with local offices. If this is not practical due to other policy decisions, it is essential that all technology, including the NCC's RightNow software, be integrated to ensure smooth operations. Table 35 shows some assumptions and projected possible future call volumes and benefits of routing all unsolicited calls through the NCC. The only financial cost to implement this recommendation relates to the advertising campaign to increase awareness of the NCC. Table 35. Potential Savings Due to Increased Call Volume Factors Current Best Case (100%) Most Likely (50%) Worst Case (25%) Annual call volume 273,804 547,608 410,706 342,255 NCC resolution rate* 51.0% 51.0% 51.0% 51.0% Total calls resolved 139,640 279,280 209,460 174,550 Call duration (minutes) 6 6 6 6 Total hours saved per year 13,964 27,928 20,946 17,455 FTE saved per year** 6.71 13.43 10.07 8.39 Value of FTE $73,000 $73,000 $73,000 $73,000 Annual Savings $490,083 $980,166 $735,124 $612,604 Improvement NA 100.0% 50.0% 25.0% *Rounded to the nearest tenth of a percent. ** Rounded to two decimal points 2. Improve call resolution at the NCC by clearly defining the NCC's role. The NCC should make an effort to train the CSRs to more effectively screen and take information from Potential Charging Parties, including asking initial basic questions. The NCC and EEOC should clearly define their respective roles and responsibilities. At a high level, the NCC should provide the primary customer contact until an Investigator is assigned to the case; then the EEOC should be the primary contact. The NCC should also train CSRs to ask effective initial questions so that they can screen out cases that clearly do not have a basis of discrimination. The JPS Team recommends that the NCC devise a pilot program in collaboration with a few offices. The offices should detail an Investigator to the NCC to coach the CSRs in regards to relevant questions that will allow CSRs to screen more effectively and send fewer calls to EEOC offices. The Investigators should then evaluate and be ready to intervene as needed in calls to their office. The EEOC should develop methods to document CSR effectiveness in this additional screening. If CSRs can attain a high, consistent level of performance, then the EEOC should expand the pilot program throughout the organization. Table 36 shows the assumptions and possible improved benefits if the NCC fully implements this recommendation. Training the CSRs is the only cost to implement this recommendation. Table 36. Improvements in EEOC Workload Savings Due to Improved Call Resolution Best Case Most Likely Worst Case Factors Current (15%) (10%) (5%) Annual call volume 273,804 273,804 273,804 273,804 Additional calls resolved due to increased NA 5,298 3,532 1,766 resolution Additional time saved NA 1,325 883 442 Total hours saved per year (13,964 base 13,964 15,289 14,847 14,406 plus additional time saved) FTE saved per year* 6.71 7.35 7.14 6.93 Value of FTE $73,000 $73,000 $73,000 $73,000 Annual Savings $490,083 $536,568 $521,073 $505,577 Improvement NA 9.5% 6.4% 3.3% * Rounded to two decimal points 3. Improve the resolution of non-charge related issues by integrating processes and technology across the EEOC and NCC. Only 66 percent of calls not related to filing a charge are resolved at the NCC. Integrating processes and technology would afford an opportunity to increase the resolution of this call type as well as provide qualitative benefits such as reduce duplication of work and increase efficiency. From a field operations perspective, the two organizations are operating in parallel and nearly autonomously from one another. The NCC and EEOC should use the same or integrated technologies to capture and maintain customer information. This would include integrating the NCC's RightNow technology with the EAS and the IMS, and to the extent possible, the GroupWise email system. Making information transparent across the two organizations will promote communication and enable a more effective customer experience. Customers should not perceive a difference between the NCC and EEOC when inquiring about issues. The EEOC should also develop processes for regular, direct communications between EEOC offices and the NCC. This will provide each group with a better understanding of the other and help sustain an ongoing working relationship to resolve issues at the "local" level before they become problems that require intervention from Headquarters. The NCC and EEOC should set up a process for EEOC offices and the NCC to regularly share knowledge, communicate questions, and provide feedback. This could include periodic meetings, designating CSRs to be responsible for coordination and communication with certain districts/offices, and/or other means of exchanging information, such as participating in each other's staff meetings. The NCC should be treated as an extension of the EEOC offices. One model that can be tested is assigning a single point of contact from the NCC to a group of offices. Integrating processes across the two organizations as well as regularly sharing knowledge and information will help the NCC employees feel more a part of the EEOC and help them resolve more calls because they will better understand field office operations. In typical call centers, when CSRs have an understanding of how they contribute to the sponsor's success, they gain a sense of brand loyalty to that organization rather than to the call center operator. The following table shows the potential benefits of integrating technologies and processes. The benefits shown here are only the portion that could be quantified. Other benefits, such as improved customer satisfaction and improved efficiencies, cannot be reliably quantified at this time. Table 37. Improvements in EEOC Workload Savings due to Improved Resolution in Calls not Related to Filing a Charge. Best Case Most Likely Worst Case Factors Current (15%) (10%) (0%) Annual call volume 273,804 273,804 273,804 273,804 Calls resolved due to increased 139,640 153,604 146,622 139,640 resolution Total hours saved per year 13,964 15,360 14,662 13,964 FTE saved per year* 6.71 7.38 7.05 6.71 Value of FTE $73,000 $73,000 $73,000 $73,000 Annual Savings $490,083 $539,091 $514,587 $490,083 Improvement NA 10.0% 5.0% 0.0% *Rounded to two decimal points Summary - Recommendations 1-3. Each of the above recommendations provides opportunities for the EEOC to improve call volume and operating efficiencies. The team expects the current year call volume will continue if the NCC continues as it is currently operated. The team made some assumptions to estimate the possible improvement in number of staff hours saved for the EEOC if all of the above recommendations are implemented. As shown in Table 38, there are synergistic benefits to implementing recommendations two through four. The team's estimate shows that the NCC can save the EEOC between 23,705 hours per annum (possible worst case) to 56,164 hours per annum (possible best case), or from 11 to 27 FTEs,[34] respectively. This is an improvement over the estimated savings in the 2003 Assessment Report (43,224 hours per annum, which equates to 20.73 FTEs). The quantifiable benefits of implementing all of the above recommendations might increase the likelihood of obtaining the savings estimated in the 2003 Assessment Report. There will also be non-quantifiable benefits, such as customer satisfaction. Table 38. Quantifiable Benefits of Implementing Recommendations One through Four Factors Current Best Case Most Likely Worst Case Annual Call Volume 273,804 547,608 410,706 342,255 NCC Resolution Rate* 51.0% 57.9% 55.3% 52.6% Total Hours Saved per Year 13,964 56,164 34,715 23,705 FTE Saved per Annum** 6.71 27.00 16.69 11.40 Value of FTE $73,000 $73,000 $73,000 $73,000 Annual Savings $490,083 $1,971,142 $1,218,360 $831,970 Improvement NA 302.4% 148.7% 69.8% *Rounded to the nearest tenth of a percent. **Rounded to two decimal points Recommendations to Optimize Customer Satisfaction and Operational Efficiencies After implementing the above recommendations, the following recommendations should be implemented to optimize NCC and EEOC operations. 4. Make significant changes to the operating model in order to realize any additional positive impact on EEOC operations and customer experience/satisfaction. The EEOC should standardize the citizen contact process into one seamless process across the EEOC and NCC. Clear standard operating policies, procedures and methods should be defined and set up for each step of the process. Before implementing this process across all offices, it should be tested with a few different types of offices to identify potential gaps. The JPS Team understands that the idea of standard intake processes across all EEOC offices runs counter to the current philosophy of allowing offices discretion in how they conduct their initial client contact and intake procedures. The EEOC could standardize the initial stages of the intake process and then give offices discretion in how they conduct the rest of the intake process. The goal is to create efficiencies in which the NCC relieves more of the clerical burden from offices than it currently does. One option advanced by survey respondents and managers during team interviews was to have the NCC issue intake questionnaires. This would eliminate some of the clerical work currently performed at the offices and enable managers to redirect resources to other mission critical activities. The NCC could accommodate a standardized EEOC questionnaire or a different one for each office. Longer term, the EEOC may also wish to evaluate the option of the NCC taking charges under close supervision and communication with EEOC offices. The JPS Team understands that taking charges has been identified as an inherently governmental responsibility; therefore, such outsourcing would require careful evaluation, consideration, and compliance with all relevant regulations. In order to evaluate, establish, and implement this and other recommendations, the EEOC and NCC should also establish a combined "EEOC/NCC Steering Committee." This Steering Committee should have representation from all stakeholders, including the field (management and Union employees), Headquarters, and NCC. This group should establish processes to integrate the systems and procedures of the EEOC offices and the NCC, increasing operating efficiencies and improving end-to-end customer satisfaction. This could lead to an expansion of the Intake Services Workgroup if membership is broadened to include all NCC stakeholders. 5. Provide training and feedback to CSRs on soft skills[35] and improve the quality of information sent to EEOC offices. CSRs need training on soft skills, which are very important in driving consistent customer experience and improved customer satisfaction. Examples of soft skills include being able to answer customer questions courteously while maintaining call control, being able to ask the right questions depending on customer tone, and steering the conversation. CSRs also need training to improve the quality and quantity of information they provide to EEOC offices. This training should help CSRs understand the data that EEOC offices require. Providing a view into uses of the data will help CSRs understand the reasons that offices need the information. The NCC should also provide regular feedback and mentoring to maintain high quality calls. This will indirectly influence customer satisfaction, because regular feedback and training will enable CSRs to provide consistent quality service. 6. The NCC should standardize and automate internal processes to achieve better efficiency, consistency, and quality of service. Part of this involves better use of available technologies. The NCC should set up, standardize, and automate their processes so that they are ready to handle increased call volume. The NCC should standardize operations, including workforce management, coaching, and training, to readily identify issues. Process automation and standardization will reduce the number of errors and force all CSRs to follow the same processes. This will make communication and training easier and provide a consistent customer experience. The JPS Team rejects the notion that the NCC should expect CSRs to be proficient users of a large number of scripts in communicating with the public. As this report documents, CSRs are unable to find the right script most of the time; they instead select the handful of scripts that they find easy to utilize and/or understand. The team recommends that the NCC create a more user- friendly engine for its knowledge base and reduce the number of scripts to those few pertaining to opening the call, closing the call, and disclaimer information. Actual product information should be available in the knowledge base. The NCC can implement this recommendation if its roles and responsibilities are clearly defined. . Recommendations to Measure On-going Performance and Optimize Readiness for the Future 7. Develop reports and other processes to identify trends proactively. The NCC should set up reporting processes to identify issues and trends. The NCC can accomplish this by understanding the EEOC's mission, operations in field offices, and the EEOC's need for information, and then linking these elements with information captured at the NCC. Call monitoring sessions should also focus on identifying potential trends and issues, which can then be tracked through reporting. 8. Implement a process to measure customer satisfaction. The NCC should implement an ongoing customer satisfaction measurement process. The NCC and EEOC should also use information obtained through satisfaction surveys to make operating model changes that positively impact customer experience and gain more efficiencies in terms of overall EEOC and NCC effectiveness. The NCC should conduct surveys on an ongoing basis, not as a one-time project. Customer expectations change from time and time, and it is important to measure and monitor them on an ongoing basis. The NCC should utilize the current methodology that the CFI Group uses to collect satisfaction information. Using an IVR option after each call is not effective for collecting this information, as customers have the option to self-select whether they participate. This can bias the results. 9. Develop baseline EEOC metrics to measure performance and monitor periodically. There are presently no reliable baseline operational metrics to evaluate the NCC's impact on the EEOC's efficiency and effectiveness. It is essential to develop baseline metrics in order to evaluate the NCC's future impacts. The team recognizes that metrics specific to measuring the NCC's impact may have broad application across the EEOC, but this team's only purpose is to recommend development of metrics that measure the NCC's impact. The performance metrics upon which the EEOC measures the NCC's success should be based on key desired operational outcomes that will indicate increased efficiency and effectiveness of EEOC field and Headquarters operations. The EEOC's live call monitoring is a good source to evaluate the quality and consistency of technical information provided to customers. The JPS Team recommends that the Steering Committee be responsible for establishing key goals and desired outcomes related to how the NCC affects the EEOC. Once these are identified, the EEOC can develop metrics. If these metrics are developed in terms of desired outcomes, they can also be used to drive operations. Table 39 shows possible key goals, desired outcomes, and metrics. The chart is provided for illustrative purposes only. Table 39. EEOC Metrics to Measure Impact of the NCC Key Goals Desired Outcomes Metric Customer Customers receive accurate Customer satisfaction via post card mailed to Service and timely service. customers one week after the charge is filed or dismissed. Fewer customer complaints. Operations Increase field efficiency and Reduced number of Class C cases. Increased effectiveness. benefits per charge. Business Reduce charge-processing Reduce time from the initial contact to when the Processes time. charge is filed. In addition to the above, the Government Accountability Office (GAO) recommends the following regular oversight practices in its February 2006 report on Federal Contact Centers: Regular knowledge database management - at least annually review information CSRs use to respond to inquiries. Regular contact monitoring - at least weekly review information provided in calls/emails to evaluate how well CSRs handle inquiries by using a scoring sheet and rating CSRs in multiple areas including courtesy, accuracy, completeness of information provided to the customer, and timeliness. Post-contact customer satisfaction survey - ask customers about the level of service received from the contact center and, to a limited extent, opinions about the accuracy of information received. Validation of contractor reports - validate data to ensure accuracy in prepared reports, including operational information such as the center's workload volumes, transaction handling times, and results of the contractor's monitoring. If future measures rely upon source coding in the IMS, the EEOC should review existing coding, correct errors and inconsistencies across offices and establish standardized procedures to ensure that all offices are following the same guidelines in coding future inquiries. In addition, if the EEOC considers telephone calls to offices in the field and Headquarters an important measure of future progress, then it should establish procedures for all offices and Headquarters to capture the important information. 10. Implement Change Management practices. The implementation of a contact center is a significant change for any system. EEOC employees must become engaged in the process of implementing the NCC in order to become vested. The EEOC can accomplish this goal by developing a change management process. This should include defining and making visible the reasons that the NCC is important, articulating the vision of the NCC, removing some of the obstacles encountered to date (implementing the recommendations above), and celebrating and building on short term wins. The JPS Team recommends establishing a core advisory group responsible for managing the change process. Through communication and training, they can ensure that the decision-making process related to the NCC is transparent to EEOC employees. It is important to communicate with employees on a regular basis. This communication could come in the form of a web page on the EEOC's intranet that would also be accessible to CSRs. The EEOC should articulate the business case and urgency for continuing the pilot, as well as the NCC's mission, goals, and objectives. Communication could include information about the upcoming advertising campaign, as well as progress toward implementing recommendations. The EEOC and NCC metrics should be visible and shared with all staff on a regular basis. Other information may include new FAQs and trends developed by the NCC. Offices could also share their successes and best practices. The existing NCC Newsletter should evolve into a means of communicating information in addition to successes. Staff at the NCC and EEOC will need training regarding the integrated processes and their respective roles and responsibilities. They will also need training on communication policies and procedures in order to increase effectiveness. This training will give each organization a better understanding of how to help the other, resulting in synergies that will benefit all (EEOC and NCC staff as well as EEOC customers). [1] The separate effects of the variables cannot be isolated. [2] Investigators and other staff that have left EEOC offices. [3] Soft skills relate to how CSRs handle transactions, e.g. professionalism in the way they handle the transaction, controlling the call flow, taking ownership of the call, and being empathetic with the caller. Hard skills relate to the product, in this case understanding and knowledge of EEOC laws. [4] The contract provides for three one-year options, which may be exercised annually, or the Commission can vote to extend the contract for three years; however, because the EEOC only has one year funding, contractually the option exercises are limited to one year at a time. [5] Field work was conducted from October 2005 through February 2006. [6] The EASQ provides basic information (who, what, when where and why an action is discriminatory), which is then used by the investigator to prepare for the follow-up interview. [7] Conducting a telephone survey was not in the team's original scope of work; however, upon learning about the telephone survey, the team considered replicating the survey on a smaller scale. [8] In order to do this we required historical office level data. [9] Even if the team could have located the data, given the deadline for the report draft, the team would have had to replicate the telephone survey in December 2005 or January 2006, which would have introduced confounds related to collecting data in dissimilar months (e.g. vacations/holidays in December and January). [10] Variables that obscure or make it impossible to identify and interpret the effect of the NCC on EEOC field office operations. [11] Because of the repositioning, equivalent data for the period January 1-March 31, 2006 were not available. [12] OFP authorized and encouraged but did not require offices to redirect unsolicited calls to the NCC. Whether and the extent to which offices implemented this change was left to the discretion of each individual office. [13] One of the five commissioner positions was vacant at the time the team conducted its interviews. [14] The Director Survey responses were compiled manually and therefore are not available with the survey instrument. [15] Convergys took the lead in evaluating the operation of the NCC. [16] SAS Institute, Inc., Statistical Software Version 9.1 [17] Based on Call Observation analysis [18] There were insufficient data to model the charge behavior or explain the change across time. [19] Outliers are typically a few extreme cases that can distort the interpretation of summary statistics. [20] This person's responses did not meet the 400-call cut-off for outliers, so the case was retained. [21] The months reported for the respective offices are somewhat different due to missing data. [22] Both offices were in the one-month pilot study during March 2005. [23] Per the Office of Personnel Management, 1 FTE=2,080 hours (http://www.opm.gov/feddata/html/2004/november/intro.asp) [24] As described in the Methodology section, the data to analyze NCC impact on case quality had major input errors on one of the critical variables, rendering results unreliable and invalid. [25] The JPS Team selected this period to ensure that findings would reflect current activities. [26] Calls where the caller wants to file a charge. [27] The NCC requirement is that offices respond to EASQs and GroupWise emails within four to six days; some offices have an internal requirement to respond within 48 hours. [28] Soft skills relate to the manner in which a CSR handles a transaction, such as professionalism in handling the transaction, controlling call flow, taking ownership of the call, and being empathetic with the caller. Hard skills are skills related to the product; in this case, understanding and knowledge of EEOC laws. [29] Customer satisfaction indexes were only available for FCIC, FTC, and CMS [30] Type of communication, e.g. web, email, calls handled by CSRs. [31] NICE is not an acronym. [32] It is possible that some responses across the two questions are from the same people. [33] Based upon an Average Investigator FTE equal to $73,000 (excluding benefits). [34] The team divided the total hours saved per annum by 2080. [35] Soft skills relate to the manner in which a CSR handles a transaction, such as professionalism in handling the transaction, controlling call flow, taking ownership of the call, and being empathetic with the caller. Hard skills are skills related to the product; in this case, understanding and knowledge of EEOC laws.
Report 2005-002- AMR - The EEOC's National Contact Center: An Evaluation of its Impacts
Published by the Equal Employment Opportunity Commission, Office of Inspector General on 2006-07-01.
Below is a raw (and likely hideous) rendition of the original report. (PDF)