Social Security Benefits: SSA Needs to Improve Oversight of Organizations that Manage Money for Vulnerable Beneficiaries

Nearly a million individuals relied on organizational payees to manage their Social Security benefits in 2018. Due to an aging population more beneficiaries may need organizational payees in the future. These beneficiaries are among the most vulnerable because, in addition to being deemed incapable of managing their own benefits, they lack family or another responsible party to assume this responsibility. SSA reports that misuse of benefits by payees is rare, but its Office of Inspector General has identified cases of misuse that have harmed vulnerable beneficiaries. GAO was asked to review SSA's organizational payee program.

This review examines, among other things SSA's process for approving payees and its monitoring efforts. GAO reviewed relevant federal laws, regulations, policies, and guidance; analyzed SSA data from fiscal year 2018; analyzed the predictive statistical model SSA uses to select low-volume payees for on-site reviews; and interviewed SSA central office staff and regional, area, and field office staff in four regions selected for geographic diversity.

The Social Security Administration (SSA) approves organizational payees—such as nursing homes or non-profits that manage the Social Security benefits of individuals unable to do so on their own—by assessing a range of suitability factors, such as whether the organizations have adequate staff to manage benefits for multiple individuals. However, GAO found that SSA's policy does not specify how to assess more complex suitability factors, such as whether an organization demonstrates sound financial management. Without clearer guidance, unqualified or ill-prepared organizational payees could be approved to manage benefits. Also, SSA does not currently require background checks for key employees of an organizational payee. In contrast, SSA requires background checks for individual payees—such as a relative or friend of the beneficiary. A comprehensive evaluation could help SSA determine whether and how to expand their use of background checks to organizational payees.

To ensure organizational payees are managing funds appropriately, SSA uses several monitoring tools, including resource-intensive onsite reviews. Certain organizational payees, such as those that charge fees for their services or have 50 or more beneficiaries (high-volume), receive onsite reviews every 3 to 4-years. In contrast, payees that serve fewer than 50 beneficiaries (low-volume)—the vast majority—are selected for review based on their estimated likelihood of misusing beneficiary funds, and a relatively low percent of them receive onsite reviews (see figure). SSA uses a predictive statistical model to identify higher risk low-volume payees, but the model's effectiveness cannot be fully assessed by GAO or others due to missing documentation on how it was designed. SSA officials said they will update the model in the future, but do not have a time frame for doing so. Establishing such a time frame and documenting design decisions are key steps toward assessing the model's effectiveness.

Another way SSA oversees organizational payees is by reviewing their annual accounting forms, but shortcomings exist in SSA's review of the form and in the form's content and design. For example, SSA lacks timeframes for following up on missing or problematic forms. Also, the accounting form does not capture complete information on whether payees co-mingle beneficiaries' funds in collective accounts, which can limit SSA's ability to monitor those risk-prone accounts. Establishing timeframes and revising the form could enhance the effectiveness of the annual accounting form as an oversight tool.

GAO is making nine recommendations in this report, including that SSA: clarify how to assess complex suitability factors; assess requiring background checks for organizational payees; establish a timeframe for reviewing the predictive model and document design decisions resulting from that review; and establish timeframes for, and conduct revisions of the accounting form. SSA agreed with all nine recommendations and provided technical comments that GAO incorporated as appropriate.

In fiscal year 2018, the Social Security Administration (SSA) paid over $1  trillion in retirement and disability benefits to over 67 million individuals.  Due to age or disability, some of these individuals are unable to manage  their benefits on their own or to direct someone else to manage them, so  SSA may assign a representative payee to do this for them. Generally,  when a family member or friend is not available to serve as the  representative payee, SSA allows certain organizations—such as  community-based non-profit social service agencies; nursing homes; or  government agencies—to serve this role. In 2018, SSA reported that  33,197 organizational representative payees served approximately  952,000 beneficiaries. The number of beneficiaries needing  organizational payees—who are particularly vulnerable among  represented beneficiaries because they lack family or friends to manage  their benefits in order to meet their most basic needs—could swell in the  coming years as the large generation of baby boomers retires and people  live longer. SSA generally relies on its network of field offices throughout  the country to approve and communicate with representative payees.

Although SSA reporting shows that misuse of benefits by organizational  payees is rare, SSA’s Office of Inspector General (OIG) has reported that  some cases of misuse and ill treatment by these organizations have  resulted in significant harm to the vulnerable beneficiaries they are  supposed to protect. For example, SSA’s OIG testified in 2017 that the  owner of an organizational payee had used beneficiary funds to purchase  properties and for the business’ use, while at the same time many  beneficiaries the organizational payee served were living in poor  conditions or homeless. In light of the important role organizational  payees play in ensuring that basic needs of vulnerable SSA beneficiaries  are met, you asked us to review SSA’s organizational payee program.  This report examines how SSA: (1) approves organizations to be  representative payees, (2) communicates with organizational payees, and  (3) oversees these organizations.

To address all three objectives, we reviewed relevant federal laws and  regulations and SSA policies and guidance. We also interviewed SSA  officials in its central office and staff in four regional offices that we  selected to reflect a range in the number of states and organizational  payees they collectively oversee, and to represent a range of geographic  locations. Within those regions, we interviewed staff at four area offices  and a total of eight field offices (in seven states) that were selected to  include both metropolitan and non-metropolitan areas. These interviews  with regional, area, and field office staff are intended to obtain  perspectives from SSA officials in different parts of the country and are  not intended to be representative of all SSA field offices or field office  staff. We also analyzed program data, including the number and type of  organizational payees and the number of beneficiaries they serve. We  assessed the reliability of these data by reviewing relevant documentation  and interviewing SSA staff knowledgeable about the systems used to  collect and maintain the data, and we determined the data were  sufficiently reliable for our use.

To obtain overall perspectives on the program, we interviewed staff of the  Social Security Advisory Board, representatives of an SSA managers’  association, and an organizational payee association. In addition, we  interviewed representatives of advocacy groups for the aged, persons  with physical disabilities, and persons with mental illness regarding their  constituents’ experiences with SSA’s organizational payee program.

To determine how organizations are approved to be representative  payees, in addition to interviewing officials at each field office we visited  about their role in the application process, we reviewed information SSA  maintained for up to six organizational payees at each visited field office,  where available. Specifically, at each visited field office we reviewed up to  two applications that SSA approved and up to two that SSA denied within  the last 5 years. At each visited field office we also reviewed files for up to  two organizational payees that closed or were terminated within the last 5  years. We selected the most recent approval, denial, and termination files  that were available.

To understand how SSA communicates with organizational payees, we  reviewed program guidance and visited eight organizational payees—one  in the local area of each field office we visited. We also interviewed SSA  officials at the central office and at the four regional and area offices.

To review SSA’s oversight of organizational payees—including onsite  reviews and other monitoring tools—we reviewed program guidance and  interviewed SSA officials at its central office and the four regional and  area offices. The onsite review process changed in 2018, when  legislation was enacted requiring SSA to award annual grants to state  protection and advocacy systems to conduct onsite reviews of  representative payees. To understand SSA’s new onsite review process,  we reviewed agency documents that describe the roles and  responsibilities of key players in the new process and interviewed SSA  officials and representatives of the National Disability Rights Network  about the status of its implementation. However, we did not assess the  efficacy of the new onsite review process because we determined it was  too early in its implementation. We analyzed the predictive model SSA  uses to select low-volume organizational payees for onsite reviews by  reviewing available documentation on its development and interviewing  SSA officials.

To determine the extent to which organizational payees are reviewed and  the outcomes of those reviews, we analyzed fiscal year 2018 program  data from SSA and reviewed SSA’s annual reports to Congress. To learn  how frequently organizational payees closed or were terminated, we  obtained data from SSA. We determined SSA data used in this report— including the extent of onsite reviews conducted by organizational payee  type, the number of organizational payee terminations, and outcomes  reported to Congress—were sufficiently reliable for our purposes. See  appendix I for more details on our scope and methodology.

We conducted this performance audit from April 2018 to September 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Different types of organizations can serve as representative payees, from  residential facilities where beneficiaries live, to organizations that only  manage individuals’ Social Security retirement, disability, or other  benefits. SSA’s organizational payees include social service agencies,  mental institutions (federal, state or local, non-profit, private), non-mental  institutions (federal, state or local, non-profit, private), financial  organizations, and entities represented by public officials (such as public  guardians, officers of the court, and other similar positions).

For certain oversight purposes, such as periodic reviews, SSA  categorizes organizational payees into several groups, including: (1) fee- for-service organizations, which charge beneficiaries a monthly fee for  expenses incurred in providing services; (2) organizations that serve 50  or more beneficiaries and do not charge a fee for their services, referred  to in this report as “high-volume”; (3) organizations that serve fewer than  50 beneficiaries and do not charge a fee for their services, referred to in  this report as “low-volume”, and (4) state mental institutions participating  in the State Onsite review program.

SSA data from fiscal year 2018 indicate that the vast majority (86 percent)  of organizational payees are low-volume payees, which serve 34 percent  of beneficiaries (see fig. 1).

Organizational payees decide how to spend beneficiaries’ funds, but must  do so for the beneficiary’s use and benefit in a manner the payee  determines to be in the beneficiary’s best interest. SSA considers it  acceptable if the funds are used for the beneficiary’s current  maintenance, which includes the costs of food, shelter, clothing, and  medical care. After the representative payee has used the funds  consistent with these guidelines, any remaining amounts must be  conserved or invested on behalf of the beneficiary.

Organizational payees are responsible for keeping records of SSA  payments made to them on behalf of each beneficiary and the  expenditures for each beneficiary. All organizational payees participate in  onsite reviews and—except for state mental institutions participating in  State Onsite reviews—are also required to file an annual accounting form  to show how benefit payments were used and any amounts that were  saved. Organizational payees also are required to notify SSA of certain  changes or situations, including:  changes that may affect the beneficiary’s eligibility for benefits or the  benefit amounts, such as when the beneficiary, or the beneficiary’s  spouse, dies; and when the beneficiary moves or is unable to be  contacted or located, starts or stops working, or no longer needs a  payee; when organizational payees learn that one or more of their employees  has stolen a beneficiary’s funds or determines they can no longer  serve as the payee for any reason; and when payees establish an account that mingles funds from multiple  beneficiaries in one account—referred to as collective accounts— because these accounts must be approved before use.

The process for administering the representative payee program is  guided largely by requirements in statute and SSA regulations, which  SSA communicates through its Program Operations Manual System  (POMS). Recent changes to the program—including the new onsite  review process—reflect requirements established by the Strengthening  Protections for Social Security Beneficiaries Act of 2018. For example,  the Commissioner of SSA must now: (1) reassess representative payee  selection and replacement policies, (2) award annual grants (totaling at  least $25 million nationwide) to each state’s protection and advocacy  system to conduct onsite reviews of representative payees, and (3)  award annual grants to a national association that can provide state  protection and advocacy systems with training, technical assistance, and  administrative oversight. In addition, the Act requires SSA to present the  results of reviews—including information on representative payees’  misuse of benefits—in an annual report to Congress.

SSA administers and manages the representative payee program through  three dedicated data systems:

The Electronic Representative Payee System (eRPS) is the system  used to process payee applications; record poor payee performance;  process changes (such as new addresses); and document misuse  allegations, significant information about the payee, and why  applications were approved or denied.

The Electronic Representative Payee Accounting (eRPA) system is  used to capture and review annual accounting forms that all  organizational payees except state mental institutions participating in  SSA’s State Onsite review program, are required to submit for each  beneficiary they represent. Field office staff also use this system to  track progress in resolving problems identified during reviews of the  form, such as representative payees failing to submit complete  information.

The newly created Representative Payee Monitoring Tool, which is  used to track and oversee the updated onsite review process.

SSA operates the representative payee program primarily through its  network of field offices. Field offices review and approve organizations’  applications to become representative payees, serve as the point of  contact when organizations report changes to beneficiary or organization  information, and play a role in monitoring and overseeing representative  payees.

SSA policy describes the required process for designating a  representative payee for a beneficiary whom SSA staff have determined  to be incapable of managing his or her benefits. First, organizations  apply to serve as a payee for specific individuals. Second, SSA staff  review applications to assess if the organization is qualified to serve as a  payee and is the most suitable payee for the individual beneficiary.  Additional qualifications are assessed when organizations apply to collect  fees for their payee services. For example, SSA requires that all fee-for- service applicants have already served at least five beneficiaries for a full  calendar month or more, and that non-governmental agencies be  licensed and bonded.

Once approved, organizational payees are subject to ongoing SSA  oversight. SSA reviews annual accounting forms from organizational  payees on each of the beneficiaries they represent. The accounting  forms are used to monitor how the payee spent or saved benefits on  behalf of the beneficiary; identify situations where payment to a payee  may no longer be appropriate; or determine if the payee is no longer  suitable. In addition to reviewing accounting forms annually, every 3 years  SSA must review collective accounts established by organizational  payees.

Whether or not additional oversight in the form of an onsite review is  provided, and the frequency of that oversight, generally depends on the  organizations’ characteristics (see table 1). Certain types of  organizational payees—such as high-volume, fee-for-service, and some  state mental institutions—receive onsite reviews every 3 or 4 years. Low- volume organizational payees do not receive periodic reviews; rather SSA  selects some of these payees for onsite reviews based on their likelihood  of misusing beneficiaries’ funds and may target additional organizational  payees because of an event that raises a question about the payee’s  performance or suitability or because a protection and advocacy grantee  thinks that a review is warranted. SSA’s purpose in conducting onsite  reviews is to: (1) ensure organizational payees perform their duties  satisfactorily, (2) deter misuse, (3) keep lines of communication open  between the organizational payee and the servicing field office, (4)  reinforce to the organizational payee their duties and responsibilities, and  (5) proactively address the needs of organizational payees.

Gaps in the Organizational Payee Approval Process Introduce Risks

We identified several gaps in SSA’s process for approving organizational  payees, including insufficient detail in SSA’s policies, insufficient  documentation of suitability decisions, and absence of background or  credit checks on most organizational payees—gaps that may increase the  risk of approving an unsuitable payee. We also identified challenges that  field offices may face when approving replacement representative  payees, such as a lack of local organizational payees and difficulty  locating some beneficiaries.

Some SSA Policies for Approving Organizational Payees Lack Detail, and SSA Lacks a Process to Ensure that Supplemental Guidance Is Compliant with Policy

When an organization applies to serve as a payee, SSA’s policy  stipulates that field office staff evaluate whether the organization is  suitable. All payee applicants—individuals and organizations—are subject  to the same general suitability factors, and organizations are evaluated  on an additional set of suitability factors. Organizations are generally  evaluated on the same suitability factors, whether they are a first-time  applicant, or applying to serve an additional beneficiary. Additional  requirements apply to organizations applying to collect fees (see  appendix II).

Two factors used to determine the suitability of organizations that are  applying to be representative payees are straight-forward: (1) whether the  payee agrees to receive benefits via direct deposit and (2) whether the  payee uses protected accounts for beneficiary funds. However, other  suitability factors are more complex, such as whether the applicant: demonstrates sound financial management policies (i.e., has a history  of being current in its own financial obligations), demonstrates effective internal communication (i.e., good  communication between case management and financial  management components), has adequate recordkeeping systems to ensure that the client’s needs  are met and benefits are properly administered.

We found that SSA’s policies on how to evaluate more complex issues do  not provide sufficient detail to ensure staff can fully assess an  organization’s suitability. Staff at one of the eight field offices we visited  told us the policies can leave room for interpretation, and staff at three  field offices use additional guidance developed by field and regional  offices that elaborates on how to assess some of the more complex  issues in SSA’s policies. For example, SSA’s policy on what constitutes  sound financial management states that an organization should have a  history of being current in its own financial obligations. However, it  generally does not provide direction on how to verify that an organization  meets that requirement. Moreover, the policy lacks details on what staff  should do to conduct a deeper assessment of an organization’s financial  management practices if they think further assessment is warranted.

Similarly, SSA policy directs staff to consider whether an organization has  effective internal communication, which it defines as good communication  between the organization’s case management and financial management  components. However, SSA’s policy does not specify what actions  constitute effective communication, such as the frequency and method of  communication, type of information to be shared, and time frames for  transmitting information. According to federal internal control standards,  agencies should establish policies to document responsibilities for a  process’s objectives and related risks and communicate these policies to  personnel so they can fulfill their assigned responsibilities. Although  SSA officials were able to point us to sections of agency policy that went  into more detail about some of these complex topics, these policies  pertain only to the few organizations that are applying to collect fees. In  the absence of specific guidance on how to consider factors when  assessing the suitability of all organizational payee applicants, SSA staff  may be approving some of them without a complete picture of their  financial health and ability to be good stewards of vulnerable  beneficiaries’ money.

According to central office officials, regions are generally given leeway to  create their own supplemental guidance documents based on SSA policy  to assist with training—documents that may also serve as resources to  help staff interpret SSA policy. Officials in field and area offices told us  this supplemental guidance is generally made available to staff on internal  websites maintained by area or regional offices. Staff in three field offices  told us they use supplemental guidance to evaluate organizations. For  example, staff in one field office told us they use a supplemental list of  questions to interview organizational payee applicants. These  supplemental interview questions address some suitability factors in  greater detail than SSA policy. For example:

SSA policy directs staff to consider whether the organization “has  adequate staff and resources to serve its clients.” The supplemental  guidance from a regional office includes five questions on the number,  type, relationships, and responsibilities of the staff; training and skills  of staff dealing with finances; and documentation.

SSA policy directs staff to consider whether the organization “has  adequate recordkeeping systems to ensure that the client’s needs are  met and benefits are properly administered.” The supplemental  guidance from a regional office includes nine questions on the  systems, records, procedures, and safeguards related to  recordkeeping.

Staff in another field office told us they created a desk guide on a range of  topics related to individual and organizational payees that includes  supplemental guidance documents and excerpts from SSA policy. The  desk guide is a reference for all employees that work on payee issues  and is also used to train new employees.

However, SSA lacks a process to ensure that supplemental guidance is  reviewed for compliance at the national level and that such guidance is  updated by the regional office in a timely manner. Officials told us that  because all regions are expected to follow SSA policy, central office staff  only review supplemental guidance when the regions request it.  Furthermore, SSA central office officials told us that although there is a  protocol for communicating policy updates to regional, area, and field  office staff, it is up to regions to refresh their own guidance. These  officials did not know how long it takes regions to incorporate policy  changes into regional guidance documents. As a result, field offices may  be using supplemental guidance that has not been updated to reflect  policy changes. For example, in a desk guide we reviewed, we identified  a policy excerpt that was not the most recent version of that policy.  Federal internal control standards stipulate that management should  periodically review policies and procedures for continued relevance and  effectiveness. Without processes to ensure that supplemental guidance  documents are reviewed for compliance or updated in a timely manner  when policy changes, decisions to approve organizational payees may be  made inconsistently across different regions and field offices.

SSA officials told us in May 2019 that they are currently reevaluating the  agency’s representative payee approval policies and procedures based  on feedback gathered through a forum hosted in September 2018 by the  Social Security Advisory Board and in response to a Federal Register  notice published in December 2018. However, SSA did not provide  additional information on the nature, scope, or timeframes of this effort.

Field Offices Do Not Always Fully Document Their Decisions to Approve Organizational Payees

SSA policy requires field office staff to document their assessment of an  applicant’s suitability as a payee and the rationale for deciding to approve  or deny an application. In addition, before approving a payee in eRPS,  the system SSA uses to manage representative payee information, field  office staff are to enter notes in accordance with the eRPS user guide.  Specifically, staff are directed to document their determination regarding  the beneficiary’s capability to manage their own finances and the  organization’s suitability as a payee for the beneficiary. In certain  situations, SSA policy directs staff to enter an additional note to document  the relationship between the beneficiary and the payee.

However, we found that staff in field offices we visited did not always fully  document their decisions before approving organizational payees for the  first time. Specifically, of the 21 first-time application files we reviewed, 16  did not contain a note about the organization’s suitability. Of the five  files that did contain such notes, three provided limited detail. For  example, two of the approved applications contained a note documenting  that the beneficiary currently lived in the facility applying to serve as  payee. However, notes in these two approved applications did not include  any details regarding the prospective payee’s suitability, such as  information about the facility or organization itself. Moreover, in two cases  where the payee was a creditor for the beneficiary, we found that SSA  staff had not documented why they approved these payees even though  they were creditors for the beneficiary. Applicants who are creditors for  beneficiaries are generally prohibited from serving as payees. Although  exceptions are allowed in certain situations—such as when the  organization is a care facility licensed or certified by the state, poses no  risk to the beneficiary, and whose financial relationship with the  beneficiary presents no substantial conflict of interest—staff are required  to document why a creditor was selected as the payee. Although being a  creditor could affect a payee’s suitability, we found that field office staff  had not recorded information about why they selected these two creditors  as the beneficiaries’ payees.

We found that SSA staff might not fully document their decisions to  approve organizational payees in part because eRPS, the system SSA  uses to process payee applications, lacks safeguards for certain  information entered into the system. As previously noted, staff use  eRPS notes to document their assessment of: the beneficiary’s capability, the payee applicant’s suitability, and, in some cases, the beneficiary- payee relationship. However, while eRPS prevents field office staff from  approving a payee without first documenting their assessment of a  beneficiary’s capability in a note, this automated safeguard does not  extend to the other note type. According to federal internal control  standards, agencies should clearly document significant events so that  that they are available for examination, and design their information  systems to obtain and process information that responds to the agency’s  objectives and risks. Because eRPS allows SSA staff to approve a  payee without fully documenting the decision, SSA staff may not be able  to reference that information to inform future decisions about the  organizational payee. Specifically, SSA staff will not be as well-prepared  to make fully informed decisions about an organizational payee’s  continuing eligibility, or whether the organizational payee should be  approved to manage benefits for additional vulnerable beneficiaries. This  creates a risk that SSA staff may unwittingly approve an inappropriate  organizational payee to serve other beneficiaries.

Without Screening Checks, SSA Lacks Additional Insight into the Suitability of Most Organizational Payee Applicants

SSA uses two types of external screening—background and credit  checks—to identify potential concerns regarding the suitability of certain  payee applicants. Whether such checks are required depends on the type  of applicant, but most organizational payees do not receive either check.

Background checks for individual representative payee  applicants: According to law and SSA policy, staff should conduct  background checks on individual payee applicants to determine if they  have a criminal history that would disqualify them from serving as a  payee. As part of the background check, policy directs staff to use  applicant interviews and tools embedded in eRPS to gather  information about the individual payee applicant’s criminal history,  including prison time or unsatisfied felony warrants. Unless the payee  is exempted by SSA policy, SSA staff will request the payee’s  permission to conduct a background check and, if permission is  granted, will then obtain a criminal report from eRPS.32,

Credit checks for some fee-for-service applicants: According to SSA policy, staff are directed to obtain and review a credit report from Dun & Bradstreet for all non-governmental organizational payees that are applying to collect fees for payee services. These credit reports include information on bankruptcies, pending or completed legal judgments, liens, payment history and risk, credit use, and how the applicant compares to other organizations in its industry.

According to POMS GN 00502.113, certain family members with custody of the  beneficiary are exempt from the background check.

For non-exempt individual representative payee applicants, field office staff must obtain  the applicant’s permission before conducting part of the background check. SSA policy  provides additional detail about the specific steps staff are directed to take to obtain  information on individual representative payee applicants’ past criminal history. If the  applicant does not give permission for a background check, their application to serve as  payee will be denied. For more information, see POMS GN 00502.113, “Interviewing the  Payee Applicant.” potential risk factors that create payee business losses due to fraud,  failure, or severe delinquency, and (2) may provide an indication of any  risk involved in the organization’s current or future performance as a  payee.

However, SSA does not assess these risk factors for most organizational  applicants because SSA policy generally does not require staff to conduct  background checks for organizational payees, and SSA only conducts  credit checks for organizational payees that apply to collect fees.  According to SSA data, as of July 2018, only 4 percent of organizational  payees were authorized to collect fees and, therefore, may have  undergone a credit check. Moreover, those credit checks that are  conducted for organizations occur after their initial approval—when they  are already serving beneficiaries—because organizations can not apply  to collect fees until they have regularly served as payee for at least five  beneficiaries for 1 calendar month or more.

SSA officials told us the agency does not conduct background checks on  organizations, in part because the process is more complicated than for  individuals. SSA recommends that organizational payees screen  employees who deal with beneficiary funds—identifying this as a best  practice—but officials told us this is not required. However, in addition to  employees who handle beneficiary funds, the criminal history of an  organization’s principals (e.g., chief executive and operating officers,  director, president, etc.) may also help inform SSA’s assessment of an  organizational payee’s suitability, as these individuals may exert great  influence over the tone and structure of the organization.

Without conducting credit or background checks, SSA risks unknowingly  approving questionable organizational applicants, therefore increasing the  risk that beneficiary funds may be misused. In May 2019, SSA officials  informed us that, while the agency has been focused on implementing  criminal background checks on non-exempt individual representative  payees, it is also exploring whether to conduct background checks on  organizational payees’ employees or require organizational payee  applicants to conduct background checks on their employees. In addition,  they told us that SSA has also been considering whether to conduct credit  checks on additional organizational payees, but has yet to make a  decision on this matter. However, SSA did not provide information on the  expected timeframes for this decision-making process. Further, SSA  lacks a comprehensive plan for evaluating if and how to expand  background and credit checks to organizational payees.

SSA Also Faces Challenges Approving Replacement Organizational Payees

When an organizational payee closes or is terminated, SSA must  ensure that all affected beneficiaries can continue to access their  benefits, either by finding a replacement payee or—when a beneficiary is  deemed capable of managing their own finances—paying the individual  directly. SSA officials told us they strive to avoid temporarily suspending  benefits. However, temporarily suspending benefits may be necessary to  avoid sending beneficiary funds to a former payee that is no longer able  or willing to manage them. SSA’s policies delineate when temporarily  suspending benefits may be necessary, such as when a beneficiary’s  whereabouts are unknown. In 2017, according to SSA data, 427  organizational payees closed or were terminated by SSA. According to  SSA data, SSA suspended benefits for more than 13,000 beneficiaries  affected by payee closures and terminations in fiscal year 2017; their  benefits were suspended for an average of 2.28 months.

SSA policy describes the steps that SSA staff must take when dealing  with the closure or termination of an organizational payee serving multiple  beneficiaries, but SSA’s level of involvement in finding replacement  payees varies depending on the situation. Staff at one field office said  that, for the only organizational payee that closed in the last several  years, they were involved in finding replacement payees for affected  beneficiaries before they terminated the organizational payee. However,  staff from two field offices told us that SSA is not always involved in  finding new payees. For example, staff at one of these field offices said  that when the state closed a nursing home in their jurisdiction, it was state  officials and not SSA who found new facilities for affected beneficiaries.  When these new facilities applied to serve as payee for their new  residents, SSA processed the applications (see sidebar). Staff at another  field office told us that before closing, some organizational payees  identified prospective payees for affected beneficiaries. In those cases,  payee staff submitted proposed payee changes to SSA, and SSA told  these prospective payees they must file an application to become the  approved payee. Officials in SSA’s central office told us that staff  determine if the applicant is the most suitable payee before approving  them.

According to SSA officials, in 2015, SSA enhanced its policy on what to  do when beneficiaries are affected by an organizational payee’s closure  or termination. Specifically, national officials told us SSA added new  procedures for appointing a new payee in cases of immediate payee  termination and emphasized the narrow circumstances when it is  appropriate to temporarily suspend benefits. Officials told us these  changes were in response to a challenging experience terminating a large  organizational payee in 2014 that served nearly 1,000 beneficiaries.

Despite this change to agency policy on replacing organizational payees  that are terminated or closed, SSA continues to face some challenges in  approving replacement payees. Specifically, SSA staff we interviewed  cited a number of challenges they had encountered, such as shortages of  local organizational payees and difficulties obtaining information from  terminated organizational payees. While these challenges may not apply  to all field offices, they provide examples of circumstances that can  complicate the process of reassigning beneficiaries.

Lack of local organizational payees. Officials in some field and  regional offices said they lack sufficient organizational payees in their  local area. For example, staff in three field offices said many  organizational payees in their area only serve certain types of  beneficiaries, such as the elderly or individuals with developmental  disabilities or specific medical conditions. Staff in two field offices told  us they had unsuccessfully tried to recruit additional organizational  payees in their jurisdiction. Similarly, a member of an SSA managers  association noted that it has been several years since a new  organizational payee was approved in her state.

Difficulty ensuring community presence for fee-for-service  organizational payees. Officials from SSA’s regional, area, and field  offices told us that it can be challenging to meet the agency’s  requirement that non-governmental fee-for-service organizational  payees be community based. For example, staff at an SSA area  office told us that finding payees within the community is challenging  in sparsely populated and remote areas, such as along Maine’s  border with Canada, where beneficiaries may not live near any  approved organizational payees. In March 2019, SSA updated the  policy on community presence for non-governmental fee-for-service  organizational payees to better specify what is required for a payee to  establish community presence, but it is not yet clear the extent to  which this update resolves field office concerns about remote areas.

Difficulty locating beneficiaries. Officials in some field and regional  offices noted that they sometimes struggle to locate beneficiaries,  which hinders reassignment. Homeless beneficiaries, in particular,  can be difficult to find, according to staff in one regional office.

Difficulty obtaining information from terminated organizational  payees. Officials in some SSA offices told us that they may lack  information necessary to complete the transfer of an affected  beneficiary to another payee. For example, staff in a regional office  said that terminated organizational payees may not always be  forthcoming about unspent beneficiary funds. Staff in another field  office told us that because a terminated organizational payee had not  maintained adequate records of beneficiaries’ guardians, SSA staff  had to go to court to identify them before approving replacement  payees.

SSA’s Communication with Organizational Payees Varies and Payee Feedback Is Not Systematically Collected SSA Has Various Opportunities to Communicate with Organizational Payees

SSA staff communicate with organizational payees at various points.  According to SSA policy, field offices should communicate with  organizational payees when they initially apply, and field office staff may  communicate with payees as part of periodic oversight activities—such as  through record change reporting requirements or following up on annual  accounting forms. During the application process, SSA field office staff  should explain the responsibilities and duties of a payee. For example,  they should explain that payees must submit an annual accounting form  and that payees must keep detailed records of how benefits are used in  order to provide an accurate report to SSA when requested. Field office  staff also should explain when payees must contact SSA, such as when a  payee’s address changes. Monitoring and oversight activities, such as  reviews of annual accounting forms, also provide opportunities for SSA  field staff to communicate with organizational payees. Similarly, SSA’s  ongoing reporting requirements—such as to update certain beneficiary or  payee information—provide another opportunity to interact. According to  field staff we interviewed, staff frequently communicate with  organizational payees regarding changes to a beneficiary’s address.  Finally, according to SSA officials, SSA also communicates with  organizational payees by providing information online and providing  guidance documents when payees are approved.

While all field offices communicate with their organizational payees, how  field offices communicate with payees can vary. Four of the eight field  offices we visited had designated specific staff either to work with each  organizational payee or with high-volume payees. In the other four field  offices, payees talk to whichever staff member is available. SSA officials  told us that the different workforce arrangements stem from varying  workflows and staffing resources at individual offices. Similarly, we found  variation across field offices regarding whether SSA staff reach out to  organizational payees even if changes do not need to be made or  problems addressed. For example, staff at four of eight field offices also  said that they have held training sessions for groups of organizational  payees. Further, staff at three field offices told us that SSA provides  training to specific organizational payees at their request, such as when  an organization experiences staff turnover.

Selected Organizational Payees Expressed Frustrations with SSA Communications

Seven of the eight organizational payees we spoke with expressed  frustration either with SSA’s follow-through on communications or with its  processes for receiving information from payees.

Application status updates. Three payees said that SSA staff did  not tell them how long it would take to review their application. They  also said SSA staff had not provided updates during the process,  which took 2 to 3 months or longer to complete.

Follow-up calls. These three payees also said that they were not told  how long it would take for SSA staff to return their calls, and two said  that sometimes they never received a call back.

Wait times. Two payees said that it takes too long to provide  information in person at SSA field offices. For example, after signing  in at a kiosk, a payee may have to wait for hours until their number is  called. This payee said that they often bring beneficiaries to the SSA  office and that long wait times can be very difficult for them,  particularly those with mental illness. In some cases, beneficiaries  have walked out or passed out while waiting in the SSA office,  according to the payee. The payee also said that long wait times are  sometimes compounded when field office staff require them to return  to the queue for each successive case rather than handling all the  payee’s cases at once. However, because field offices are allowed to  establish their own workflow processes, this issue may not apply to  other field offices.

Faxing documents versus sending them electronically. Three  payees said that having to fax documentation to SSA rather than send  this information electronically creates additional work. SSA officials  said that the agency has a plan to allow individual payees to securely  transmit personally identifiable information electronically, but has not  established a timeframe for allowing organizational payees to do so.

At the field offices we visited, managers had different expectations  regarding time frames for responding to payee requests. Three managers  we interviewed said that staff should respond to payees as soon as  possible, three managers said that staff should respond within 24-48  hours, and two managers said staff should respond within 7-14 days.  SSA officials told us that SSA has not set timeliness standards for field  offices because doing so could affect other workloads in unanticipated  ways and it is the agency’s goal to provide service and support to all  payees on an ongoing basis.

Organizational Payees May Provide Feedback to SSA but SSA Lacks Systematic or Formal Feedback Mechanisms for Collecting and Analyzing Feedback

SSA may receive feedback from organizational payees through various  mechanisms. Officials from SSA’s central office told us that organizational  payees can provide feedback either by contacting their local field office or  calling SSA’s national customer service number. Some field office staff  also said that they provide informal opportunities for payees to offer  feedback. For example, one field office manager told us that he spends  time building relationships with organizational payees, solicits feedback  by asking how things are going, and sometimes visits organizational  payees when he is nearby. Another manager emphasized the importance  of gathering and responding to organizational payee feedback. This  manager said that she established quarterly calls with multiple payees to  discuss issues and solicit feedback. Managers of two field offices told us  that they provide standardized SSA customer comment cards in their  waiting areas, although the cards do not ask respondents to identify  whether they are organizational payees.

However, SSA does not have a mechanism for payee feedback to be  systematically collected, compiled, and analyzed across field offices to  determine if programmatic changes are warranted. SSA officials said they  do not have or plan to develop a formal mechanism for collecting and  analyzing organizational payee feedback because the current process  allows field offices to respond to all public contacts in a consistent and  timely manner. However, federal internal control standards state that  management should establish reporting lines that allow the agency to  receive quality information from external stakeholders and specify that  quality information, among other things, should be complete, current, and  provided on a timely basis. Without a formal mechanism to systematically  collect and analyze payees’ feedback and ideas for program  improvement, SSA cannot be sure that it is receiving complete or current  impressions from organizational payees on how efficient its processes are  or how timely it responds to their needs. Being aware of and responding  to payees’ concerns might help the agency retain and attract  organizations to serve as payees and ensure it is well-positioned to meet  future challenges.

SSA Offices Use Several Methods to Oversee Organizational Payees, but the Methods Have Shortcomings

SSA uses several methods to oversee organizational payees, including  conducting onsite reviews, and reviewing annual accounting forms and  collective accounts. However, each of these methods has shortcomings  in its design and implementation, weakening SSA’s ability to effectively  oversee payees and prevent fraud. SSA officials said they plan to conduct  an over-arching assessment of fraud risks to the representative payee  program in 2019, but the robustness of such a plan is yet to be  determined.

SSA is Transitioning to a New Onsite Review Process but its Process for Targeting Low-Volume Payees for Review is Poorly Documented

Transition to SSA’s New Onsite Review Process

State protection and advocacy agencies (“state grantees”), the national  association grantee (which is currently the National Disability Rights  Network, or NDRN), and SSA regional offices play key roles in the new  onsite review process for organizational payees. Given the extent to  which onsite reviews uncover misuse and other problems, the onsite  review is a crucial control for the representative payee program. Under  the new process, state grantees generally interview selected payees,  beneficiaries, and legal guardians or third parties; review financial records  for selected beneficiaries over a 12-month period; transmit findings from  their reviews to SSA; and, in some cases, follow up on deficiencies they  identify. State grantees also suggest additional payees to review  (beyond those targeted by SSA) if they think such a review is warranted.  According to SSA, the national association grantee’s responsibilities  include: (1) training state grantees; (2) ensuring the quality of onsite  reviews; (3) serving as the first point of contact for state grantee  communication and questions; and (4) providing state grantees with  technical assistance, administrative support, and data collection services.  According to SSA, the regional offices are responsible for compiling  information to facilitate grantees’ onsite reviews that is not automatically  provided through SSA’s system and for clarifying procedural and  technical information for the grantees. Regional offices also address and  resolve all deficiencies the grantees do not resolve, according to SSA.  Lastly, under the new system, state grantees, the national association  grantee, and SSA input information from reviews and track progress  towards completing their assigned reviews using SSA’s new  Representative Payee Monitoring Tool, which is used to manage and  control the new onsite review process.

According to six NDRN representatives, transitioning to the new onsite  review system involved challenges with grantees gaining access to  equipment, working through bottlenecks at some regional offices,  responding to unanticipated workloads, and receiving timely responses to  feedback. Specifically, NDRN representatives said that while the process  of clearing grantees to access beneficiaries’ personally identifiable  information has been efficient, there have been delays providing grantees  with access to SSA laptops, printers, and scanners. As a result of these  equipment delays, grantees started to conduct reviews on paper and then  input the information later, according to NDRN representatives, thus using  less efficient, manual processes. NDRN representatives also said that the  new onsite review process involves multiple handoffs between grantee  and regional office staff, which has contributed to bottlenecks at some  regional offices. Moreover, NDRN representatives noted that, in addition  to the reviews SSA originally assigned to the grantees, regional offices  have tasked them with conducting quick response checks. Because  these reviews have generally involved assessing a large number of  financial records and conducting many beneficiary interviews, and were  not anticipated in SSA’s initial plan, NDRN representatives believe they  may affect the ability of some state grantees to complete the other  reviews SSA had initially planned. Lastly, an NDRN representative said  that the timeliness of SSA’s responses to grantee feedback and concerns  (communicated from state grantees via NDRN) has diminished in recent  months. Specifically, the NDRN representative said that the computer  program SSA staff developed to enable NDRN to submit questions to the  agency was initially working well. However, recently, as the volume of  NDRN’s questions has increased, the system is not working as well, and  NDRN has asked for clarification on some important issues, to which SSA  has not yet responded.

As of May 2019, SSA reported to us on progress state grantees had  made towards reaching the total number of reviews SSA had planned for  the fiscal year. Specifically, as of May 21, 2019, state grantees had  conducted 112 of 852 planned high-volume reviews; 45 of 461 planned  fee-for-service reviews; and 0 out of 60 planned state mental institution  reviews. Although SSA initially assigned 2,800 low-volume reviews in  grant year 2019, SSA estimated in July 2019 that it will have initiated  around 1600 low-volume reviews by the end of the first grant year—about  the same number as completed in fiscal year 2018 (1,691).

SSA officials acknowledged these challenges and said they have been  addressing them, and will continue to address them and to monitor  progress. SSA officials cited significant improvements in issuing laptops  since they began the process in September of 2018. Regarding delays in  distributing printers and scanners, SSA reported that they are in the final  stages of procuring printers but that as of May 2019, they had not  identified an acceptable scanner model. SSA officials also said they are  developing a policy to govern grantee use of printers. SSA acknowledged  that workflow bottlenecks involving regional offices may exist, and said  that they will continue to monitor all actions required to be taken by  regional office staff. SSA staff also acknowledged having initiated more  quick response checks than originally anticipated, and said they are  researching options to alleviate the impact of these reviews on NDRN and  state grantee resources. Finally, SSA staff said that they will continue to  evaluate how SSA collects and responds to state grantees’ feedback, and  to hold weekly discussions with NDRN to identify ways to improve the  new onsite review process. GAO is not making recommendations in this  area because the onsite review process is new and SSA continues to  implement it and work to address implementation challenges.

Targeting Onsite Reviews for Organizations That Are Low- Volume Payees

Onsite reviews are resource intensive because they involve examining  organizational payee financial records and interviewing payee staff and  beneficiaries; therefore, SSA uses a risk-based approach to select which  organizational payees receive onsite reviews and how frequently such  reviews occur. SSA reviews all fee-for-service, high-volume payees, and  certain state mental institutions—which together account for around 67  percent of all beneficiaries and about 14 percent of all organizational  payees—at regular intervals of every 3 or 4 years, depending on the type  of organization. However, for the vast majority of organizations that are  low-volume payees (29,082 of around 33,700), SSA selects a subset of  payees to receive onsite reviews each year. As shown in figure 2, more  than half of the onsite reviews SSA conducted in fiscal year 2018 were for  low-volume payees (1,619 of 2,774 reviews). However, because there are  so many low-volume payees, only about 6 percent of these payees  received an on-site review. In contrast, the lower number of high-volume  onsite reviews conducted (767) covered about 25 percent of high-volume  payees.

Given that only a fraction of low-volume payees are selected for onsite  review each year, it is critical that SSA effectively prioritize which payees  should receive onsite reviews so SSA can effectively allocate resources.  To this end, SSA uses a predictive statistical model it first implemented in  2012 to rank low-volume organizations based on their chance of misusing  beneficiary funds and selects for onsite reviews those organizations  identified as having the highest risk. SSA staff told us they determine  how many reviews to conduct based on available resources.

However, we were unable to fully assess SSA’s decisions in developing  its model, or the model’s accuracy at predicting misuse compared to  alternative targeting methods, because SSA did not fully document or  retain documentation that described in sufficient detail important  decisions it made when developing it. For example, the available  documentation does not explain in sufficient detail how SSA assembled  data on the target population; how SSA sampled organizational payees  for assessing characteristics; which variables SSA considered using to  help predict misuse but ultimately decided not to include; how, if at all, it  assessed or assured itself of the reliability of the data the model used;  and how it decided to account for multiple beneficiaries with the same  payee. An SSA official responsible for using the model said he was not  sure whether documentation existed but was not retained, because the  individuals who developed the model are no longer with the agency.  Office of Management and Budget (OMB) standards for federal censuses  and surveys—which contain accepted practices (but not requirements) for  federal statistical efforts not officially covered by the standards—call for  documentation that “includes those materials necessary to understand  how to properly analyze data” and “replicate and evaluate” statistical  estimates. Moreover, federal internal control standards state that  effective documentation enables agencies to retain organizational  knowledge, mitigate the risk of having knowledge limited to a few  personnel, and communicate knowledge as needed to external parties,  such as external auditors. Due to the absence of key documentation,  neither SSA itself nor an external party is able to affirm whether, in  comparison to other approaches, SSA’s predictive model is the optimal  approach to identify low-volume payees and beneficiaries with the highest  risk of misuse.

SSA officials told us that they will revise the model at some point in the  future—at which point they could improve the documentation—but that  they do not have a formal plan to do so. SSA officials said they do not  have imminent plans to update the model because the pool of identified  misuse cases, which is driven by the number of onsite reviews  conducted, is too small. Finally, SSA officials said they are hesitant to  re-evaluate the organizational payee predictive model because they  believe the current model is working effectively.

However, seven years have passed since SSA first developed the model,  and SSA cannot be assured that the current model remains as effective  as when it was last formally validated and compared to alternative models  or targeting methods. Accepted practices for developing predictive  statistical models call for periodic re-estimation and re-validation, using  data that are current and applicable to the conditions in which the model  will be applied. Moreover, federal internal control standards call for  agencies to conduct ongoing monitoring of the design and effectiveness  of the internal control system including evaluations of control design.  SSA reported conducting ongoing assessments of the model’s continued  effectiveness, and provided us with aggregate performance data for 2012  to 2016. However, inclusion of older data and absence of more recent  misuse data in aggregated results provide limited assurance of the  model’s ongoing effectiveness.

In addition, a recent report by SSA’s Office of the Inspector General (OIG)  suggests that it may be possible to assess the ongoing suitability of  nursing home payees by using additional data, although we did not  evaluate the validity of the study’s conclusions. The SSA OIG’s report  expressly looked at how data from the Centers for Medicare & Medicaid  Services (CMS) might be used to evaluate the suitability of nursing  homes and found that these data would help SSA more effectively assess  the ongoing suitability of existing nursing home payees. Specifically, the  OIG used CMS data reflecting penalties and other signs of  underperformance to identify poorly-performing nursing homes that might  also be poorly executing their duties as payees.

For instance, according to SSA policy documents, the form can help  SSA identify previously unreported changes to beneficiaries’ addresses; identify  unapproved collective accounts; determine if certain beneficiaries’ savings are too high to  qualify for benefits; or determine whether the organizational payee is authorized to charge  a fee, if the payee reports charging one. outcomes would significantly dilute the model’s ability to detect misuse,  which they consider to be the most important goal of the representative  payee review process.

Developing additional models to predict other types of poor payee  performance besides misuse (such as poor recordkeeping or payees’  failing to meet beneficiary needs, which were identified in the OIG study)  could reduce SSA’s reliance on a model for which the low number of  misuse findings affects the efficacy of ongoing performance assessments  and prevents timely updates. Since SSA has only identified 31 misuse  cases using the predictive model since 2012, decades may pass before  SSA has the approximately 5,300 misuse cases it wants in order to  formally evaluate the model, and before SSA and others can be assured  that low-volume payees are being optimally targeted for review. Without  re-evaluating whether the current model remains predictive, and periodic  assessments about whether it predicts high-risk payees better than an  alternative model or targeting method, it is unknown whether SSA has  maximized its ability to target low-volume payees.

SSA Reviews Annual Accounting Form Submissions and Collective Accounts, but these Reviews Have Shortcomings Timeframes for Annual Accounting Form Follow Up

The annual accounting form is a key oversight tool because it touches  most organizational payees, and reviewing the annual accounting form  helps SSA to maintain current beneficiary and payee information and to  identify and resolve potential problems.

For instance, according to SSA policy documents, the form can help SSA identify  previously unreported changes to beneficiaries’ addresses; identify unapproved collective  accounts; determine if certain beneficiaries’ savings are too high to qualify for benefits; or  determine whether the organizational payee is authorized to charge a fee, if the payee  reports charging one. electronic processing indicates a potential problem, field offices  sometimes follow up with the payee to resolve the issue.

However, SSA has not established time frames within which field offices  must initiate this follow-up. For example, SSA guidance states that when  organizational payees do not submit the form timely, field offices should  contact the payee by phone to find out why the required form was not  completed. However, the guidance does not establish time frames within  which field offices should initiate the call. Similarly, SSA told us they do  not have time frames within which staff should follow up to resolve  potential problems flagged during electronic testing. In the absence of  national guidance, area offices we interviewed varied in the extent to  which they established time frames for the field offices in their purview to  follow up with organizational payees that did not submit an annual  accounting form or whose form was flagged for potential errors. One area  office we talked with expected staff to follow up with payees within 30  days but did not track time frames, another area office had not  established time frames, and officials from one field office told us that  their area office considered follow-up over 120 days to be untimely.

Given the absence of SSA guidance and variation in area office practices  related to establishing timeframes, field offices may not ensure that this  oversight mechanism is attended to in a timely manner. Officials at one  field office we visited told us that they had a backlog of forms needing  follow-up because the designated point person had left the agency.  Officials from another field office attributed the backlog to multiple factors,  including a staff person being out sick and their workload not being  reassigned, and the office taking on a special project. While we heard  from several field offices that the majority of follow-up on annual  accounting forms is for clerical errors or mistakes, staff from one field  office said that when staff must follow up with organizational payees to  ensure they submit a simple accounting form, it raises concerns about  whether those payees are fulfilling their other duties.

Federal internal control standards state that managers should use quality  information to achieve the entity’s objectives and that they should ensure  information is complete and provided on a timely basis. In May 2019,  SSA officials told us that they are now exploring approaches to implement  a nation-wide time frame to address these forms because a 2018 law— which reduced the volume of annual accounting forms SSA has to  process—allows staff to focus on problematic forms more expeditiously.  SSA officials said that they had not previously established a time frame  because they expected organizational payees to have routine contact  with field offices and expected field offices to re-evaluate the payee’s  suitability if the payee did not cooperate when conducting SSA business.  In addition, SSA expects state grantees to follow up on accounting forms  as part of their onsite reviews. At the same time, one of SSA’s stated  purposes for using the annual accounting form is to evaluate payee  suitability on a regular basis rather than relying on ad hoc interactions  between the payee and field office, or relatively infrequent periodic and  targeted reviews. Until SSA establishes time frames within which staff  must follow up on issues identified during annual accounting reviews, the  agency cannot ensure that it is taking timely action to resolve potential  problems and maximize this monitoring tool.

Content and Design of the Annual Accounting Form

Although the accounting form is a key oversight tool for SSA,  shortcomings exist in the form’s content and design. For example, SSA’s  annual accounting form does not ask or remind organizational payees  about all collective account requirements, and as a result does not fully  support SSA’s oversight efforts. Collective accounts are permitted under  SSA policy, but SSA reviews and approves them to ensure that payees  comply with SSA’s policies and procedures. While the annual  accounting form asks payees whether they put any saved funds into a  collective account, the form does not ask whether the payee uses a  collective account for day-to-day expenses. Payees should disclose the  use of any collective account to SSA independent of the form but may  have neglected to, and SSA does not use the form to fully ascertain the  use of collective accounts. Consequently, SSA may not have up-to-date  information about all of the collective accounts that an organizational  payee might be using—information that could help place these risk-prone  accounts on SSA’s radar to initiate the approval process and provide  ongoing oversight. Federal internal control standards state that  agencies should design control activities to achieve objectives and  respond to risks. When SSA officials were asked why the annual  accounting form does not ask about all collective accounts, the officials  said this would be unnecessary because payees are required to notify the  field office if they wish to open such accounts. SSA also indicated that its  periodic and targeted onsite reviews will uncover collective account  issues for the highest risk payees.

However, SSA finds many instances of unapproved collective accounts  during its onsite reviews, suggesting that organizational payees might not  be proactively reporting opening such accounts to SSA as required. For  example, in fiscal year 2018, SSA found unapproved collective accounts  in nearly 17 percent of the onsite reviews it conducted of organizational  payees (in 477 instances out of 2,882 reviews). Staff we interviewed from  one field office also said they have identified organizational payees with  unapproved collective accounts. Specifically, staff said they have  identified at least three payees with unapproved accounts, one of which  they identified when reviewing the payee’s annual accounting form. This  suggests that some payees may be willing to report they have a collective  account, but not remember or understand their responsibility to seek  approval from SSA when they open such accounts. Although SSA’s  accounting form includes reminders of various payee responsibilities, the  form does not include a reminder to all payees that they should notify  SSA when they establish collective accounts. Reminding payees of these  responsibilities could serve as a regular reminder for payees to notify  SSA about the existence of these accounts, and thereby help ensure SSA  provides regular oversight of them.

Stakeholders have also identified shortcomings in the content and design  of the accounting form. For example, SSA currently provides payees’ total  benefit amounts in the form, and asks payees to report how they spent  those benefits. In a 2007 review of SSA’s representative payee program,  the National Academy of Sciences (NAS) reported that because SSA  preprints total annual benefit amounts on the annual accounting form, it is  easy for payees to report spending that matches the total provided by  SSA. Even if the amounts the payee reported were incorrect, SSA’s  electronic check would not trigger further review of these responses as  long as the numbers added up. NAS further suggested that omitting this  information would reinforce payees’ responsibility for keeping and  consulting their records. In light of this and other findings, NAS broadly  recommended redesigning the form to collect more meaningful data—a  recommendation echoed by the Social Security Advisory Board in 2018.  When asked why SSA did not adopt NAS’ recommendation, SSA  indicated to us that it believed that NAS signaled that other  recommendations were more important, and cited NAS’ statement that  “no form, by itself, is going to detect program misuse.” At the same time,  NAS restated its recommendation to redesign the form twice in its report  and in each instance noted how the form could complement other  oversight efforts.

Research also suggests that agencies can improve the quality of the data  they collect via forms by applying behavioral science insights. For  example, behavioral science research has shown that requiring a  signature at the beginning of an online form helps promote honest self- reporting and can lead to government savings.77,  Moreover, the Internal  Revenue Service has identified approaches based on behavioral science  insights for improving compliance and honest self-reporting, and for  encouraging people to make good choices when providing information.  Given the importance of the annual accounting forms for oversight of  payees, considering and applying, where appropriate, behavioral science  insights while redesigning the accounting forms could help SSA achieve  more reliable and accurate reporting.

Executive Office of the President. National Science and Technology Council. Social and  Behavioral Sciences Team Annual Report (September 2015). See ep 31-32. Lisa L. Shu,  Nina Mazar, Francesca Gino, Dan Ariely, and Max H. Bazerman, Signing at the beginning  makes ethics salient and decreases dishonest self-reports in comparison to signing at the  end.” Proceedings of the National Academy of Sciences of the United States of America.  (Nov. 18, 2012).

In contrast, the signature field in SSA’s accounting forms is located at the end of the  form. accounts without oversight. A regional office analyst referred to this as a  glitch in the system and told us this issue was recently raised during a  meeting with the central office.

In response to our inquiry about disappearing alerts and collective  account information, SSA staff indicated that removing alerts and  collective account information after approval expires is appropriate  because field offices should always renew collective accounts before this  occurs. SSA further explained that the alerts are not deleted from eRPS,  but rather removed from the system’s “Workload Action Center” 30 days  after the collective account expiration date. Similarly, SSA reported that  collective account information is not deleted from eRPS, but rather no  longer displayed as an active account. However, removing information on  accounts that were not renewed timely weakens the efficacy of its  collective account review process to the extent that accounts are  operating without SSA approval and oversight.

SSA Did Not Have a Process for Periodically Assessing Program Risks, but Recently Said it Plans to Conduct a Comprehensive Fraud Risk Assessment

SSA has taken steps to address risk associated with payee oversight, but  to date has not continuously assessed and responded to potential risks.  Federal internal control standards state that to manage risk, agencies  should identify risks that might prevent the agency from achieving its  objective; assess the significance of those risks; and design responses so  that analyzed risks are within the agency’s risk tolerance level. In June  2013, SSA formed a task team to conduct a comprehensive review of the  representative payee program and develop recommendations. This effort  resulted in, for example, a new process of sharing misuse information  with the Department of Veterans Affairs. While this was a positive step,  the task team disbanded in 2014 because it had generated a set of  recommendations and SSA wanted to shift to implementing those  recommendations, according to agency staff. However, resulting actions  did not include a forum or system for continuously assessing lessons  learned from audits and reviews or identifying solutions that might have  addressed gaps we identified in this report. For example, we found that  SSA discovers many instances of unapproved collective accounts during  onsite reviews, but we have not seen documentation that SSA has  assessed the risk of unapproved collective accounts existing among low- volume payees that do not receive any regular scrutiny. Having a process  for continuously assessing and responding to potential risks could better  position the agency to respond to pressure placed on the payee program  due to an aging population.

As of May 2019, SSA reported it was in the early stages of planning a  fraud risk assessment of the representative payee program (for both  individual and organizational payees). In January 2019, a staff person  within SSA’s Office of Anti-Fraud Programs, which provides centralized  accountability and oversight for the agency’s anti-fraud activities, told us  they had identified the representative payee program as one that might  benefit from a risk assessment, and that they were currently developing a  strategy for conducting such risk assessments for a number of programs.  At that time, the staff person did not know whether they would be doing a  fraud risk assessment of the representative payee program specifically.  SSA subsequently reported in May 2019 that the agency has established  a schedule and business process for conducting its risk assessments,  including one on the representative payee program. According to SSA,  the fraud risk assessment will provide a comprehensive and strategic look  at the fraud risks facing the representative payee program and the  controls SSA has in place to mitigate those risks. SSA also reported it  plans to begin the assessment of the representative payee program in  October 2019, and update it every 3 years beginning in 2024 to determine  whether there have been any changes to the risks and whether additional  actions are required. While promising, SSA plans have yet to take shape.  Ensuring that its fraud risk assessments periodically examine the results  of onsite reviews and audits will be an important element in the design of  SSA’s risk assessment efforts.

Organizational payees play a critical role in ensuring beneficiaries’ basic  needs are met. The beneficiaries these payees serve—individuals who  cannot manage their own finances and lack a family member or friend to  do so on their behalf—are dependent on their representative payees and  thus extremely vulnerable to financial abuse. It is therefore crucial that  SSA take steps to shore up a range of gaps in how the agency evaluates,  supports, and oversees payees to better ensure beneficiaries are  protected.

Carefully screening organizations applying to be representative payees is  key to proactively avoiding potential abuse. However, in the absence of  detailed and centrally-approved policy guidance on how to assess  complex suitability factors for approving payees, SSA cannot be sure that  field office staff are consistently and appropriately evaluating applicants’  suitability. Also, until SSA updates its electronic system to ensure staff’s  rationale for approving or denying payees is captured in accordance with  policy, SSA may not have the benefit of information to better monitor  payees and inform future suitability decisions. Lastly, without a  comprehensive plan, including timeframes, for evaluating if and how to  conduct background and credit checks to help staff vet organizational  payees—as it does for individual payee applicants—SSA may forgo  potentially valuable safeguards for further protecting vulnerable  beneficiaries.

Once approved, organizational payees rely on SSA for information or  action in order to effectively carry out their responsibilities. Absent a  formal mechanism whereby feedback from payees on SSA services and  processes can be collected, compiled, and analyzed, SSA may not be  sufficiently aware of payee needs and frustrations, which in turn could  result in lost opportunities to either retain or recruit organizations willing to  serve this critical function, or make program improvements.

To ensure payees are managing beneficiary funds appropriately, SSA  relies on a number of monitoring mechanisms, including onsite reviews.  Onsite reviews represent SSA’s most thorough and resource-intensive  monitoring tool and must be appropriately targeted. Until SSA develops a  plan to periodically review the predictive model’s design, considers  inclusion of additional relevant data in the current model or an alternative  model that predicts outcomes other than misuse, and documents any  subsequent design changes, the model’s efficacy cannot be fully  assessed or ultimately improved upon, and SSA may not be effectively  targeting high-risk, low-volume payees for review. SSA may detect payee  performance problems by reviewing annual accounting forms for all  organizational payees; however, without a process to ensure prompt  follow-up, SSA cannot be sure staff resolve problems in a timely manner.  Moreover, mingling beneficiaries’ funds in collective accounts can mask  misuse, and until SSA addresses gaps in the annual accounting form and  issues with eRPS, SSA cannot effectively monitor payees’ use of such  accounts.

Addressing gaps in existing processes could improve the integrity of  SSA’s representative payee program and reduce risks to SSA’s most  vulnerable beneficiaries, but may not be sufficient in light of challenges  posed by the nation’s aging population, which could swell the number of  vulnerable beneficiaries that need payees. Carrying through with its plan  to develop initial and periodic fraud risk assessments for the  representative payee program—and ensuring that the assessments  reflect consideration of findings from onsite reviews and audits—could  help SSA anticipate potential problems and develop strategies to mitigate  their impact.

We are making the following nine recommendations to SSA:  The Commissioner of the Social Security Administration should ensure  that (a) the agency’s policies and guidance are specific enough so field  office staff know how to apply complex suitability criteria for assessing  payee suitability, such as by providing a minimum set of specific  questions; and (b) additional regional guidance that is made available to  staff is centrally reviewed for compliance and completeness.  (Recommendation 1)

The Commissioner of the Social Security Administration should create  safeguards in the eRPS system to ensure that field office staff fully  document all required information, such as the rationale for their decision,  before approving an application. (Recommendation 2)

The Commissioner of the Social Security Administration should complete  a plan, including timeframes, for comprehensively evaluating if and how  to leverage external sources of information on organizations’ suitability,  such as by conducting background checks or credit checks on  organizations or key staff that handle beneficiaries’ funds or requiring  organizations to conduct their own background checks on key staff.  (Recommendation 3)

The Commissioner of the Social Security Administration should develop  and implement mechanisms to systematically obtain and review feedback  from organizational payees and communicate findings to SSA  management. (Recommendation 4)

The Commissioner of the Social Security Administration should (a)  establish a plan and time frame for periodically reviewing the predictive  model’s design; (b) consider additional data sources that would allow for  additional screening or modeling of potentially high-risk organizational  payees; and (c) ensure that subsequent design decisions are  documented in sufficient detail so the development process can be more  fully understood and replicated, either by SSA or a knowledgeable third  party, with minimal further explanation. (Recommendation 5)

The Commissioner of the Social Security Administration should require  field offices to contact payees about missing or problematic annual  accounting forms within a specific time frame. (Recommendation 6)

The Commissioner of the Social Security Administration should revise the  annual accounting form to enhance its effectiveness. Such revisions  could include (but not be limited to) more fully ascertaining the use of  collective accounts, adopting stakeholders’ recommendations on using  the form to collect more meaningful data, and reflecting best practices  from behavioral science insights in the design of the form.  (Recommendation 7)

The Commissioner of the Social Security Administration should enhance  the eRPS system to ensure that field offices are (a) alerted when  collective accounts are due to be reviewed; and (b) able to take action on  expired collective account information and thereby avoid payees’  continued use of these accounts without oversight. (Recommendation 8)

The Commissioner of the Social Security Administration should, as it  carries through with its plan to develop a risk assessment for the  organizational payee program, ensure that that the plan reflects periodic  consideration of findings from onsite reviews and audits.  (Recommendation 9)

We provided a draft of this report to SSA for review and comment. In  written comments, reproduced in appendix IV, SSA agreed with all nine of  our recommendations and outlined its planned actions to address several  of the recommendations. SSA also provided technical comments that we  incorporated into the report, as appropriate.

SSA provided additional comments on its plans to address four of our  recommendations. Specifically, with respect to our second  recommendation that SSA create safeguards in its Electronic  Representative Payee System (eRPS) to ensure that field office staff fully  document decisions to approve organizational payee applications, SSA  reported that, as part of implementing the Strengthening Protections for  Social Security Beneficiaries Act of 2018, planned changes to eRPS will  improve documentation of selection decisions. SSA also reported it will  also consider additional enhancements to eRPS in the future. We  welcome SSA’s intentions to improve documentation of selection  decisions and consider additional enhancements to eRPS.

With respect to our third recommendation that SSA complete a plan,  including timeframes, for evaluating if and how to leverage external  sources of information on organizations’ suitability, such as by conducting  background checks or credit checks on organizational payee applicants,  SSA officials reiterated that SSA is first focusing on implementing  provisions of the Strengthening Protections for Social Security  Beneficiaries Act of 2018 related to background checks for certain  individual payees. After completing this work, SSA plans to evaluate  conducting criminal background checks and credit checks on  organizational payees and their staff. While we agree that implementing  background screening pursuant to the law should take precedence, SSA  should seek opportunities to implement screening for organizational  payees at the earliest opportunity.

With respect to our fifth recommendation related to SSA reviewing,  enhancing and documenting its model for selecting low-volume  organizational payees for on-site reviews, SSA reported that it will pursue  other data sources to develop additional screening tools and models to  identify potentially high-risk organizational payees, but that it is unable to  incorporate additional data into the existing model. We recognize that the  current model, which focuses on misuse findings and is based on  historical data, presents challenges for both updating and including new  data sources. Therefore, as SSA considers additional screening tools and  models to identify high-risk, low-volume organizational payees, SSA  should develop a plan for revising the existing model that allows for more  timely updates and results in documentation of related design decisions.

With respect to our eighth recommendation that SSA enhance the eRPS  system to more effectively address expiring collective accounts, SSA  officials reported that they would work with staff to ensure staff know  where to find alerts for expiring accounts and enhance how eRPS  displays information on collective accounts that have already expired. We  agree with SSA’s proposed actions. However, we adjusted our  recommendation to clarify that SSA should enhance eRPS in a manner  that ensures staff take action on expired accounts and that payees do not  continue to use expired accounts without oversight.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Commissioner of the Social Security  Administration, and other interested parties. In addition, the report will be  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-4040 or curdae@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

The three objectives examined in this report are how the Social Security  Administration (SSA): (1) approves organizations to be representative  payees, (2) communicates with organizational payees, and (3) oversees  these organizations.

To address our three objectives, we reviewed relevant federal laws and  SSA policies and guidance. We interviewed SSA officials in its central  office and staff in four regional offices that we selected to reflect a range  in the number of states and organizational payees they collectively  oversee and to achieve diversity in geographic location. Within those  regions, we visited eight field offices covering seven states, which were  selected to include both metropolitan and non-metropolitan areas that  maximized the number of files we would have available for our review  (see next paragraph). We also interviewed officials in one area office per  region—two representing metropolitan area field offices, and two  representing non-metropolitan area offices that we visited. These  interviews with regional, area, and field office staff are intended to obtain  perspectives from SSA officials in different parts of the country and are  not intended to be representative of all SSA field offices and staff. We  also analyzed program data, including the number and type of  organizational payees and the number of beneficiaries they serve. We  assessed the reliability of these data by reviewing relevant documentation  and interviewing SSA staff knowledgeable about the systems used to  collect and maintain the data and determined the data were sufficiently  reliable for our use.

To determine how organizations are approved to be representative  payees, we reviewed SSA’s policies and relevant federal laws and  regulations. At each field office we visited, we (1) interviewed managers  about their role in the application process and (2) reviewed up to six  organizations’ electronic files in the Electronic Representative Payee  System (eRPS), the primary data system SSA uses to track  representative payees. Specifically, at each field office we reviewed up to  two applications that SSA had approved (either initial applications to  serve as representative payee or initial applications to collect fees); up to  two applications that SSA had denied (initial or to collect fees); and files  for up to two organizations that were terminated or closed in the past 5  years. In some cases, field offices we visited did not have the full number  of cases available, and we reviewed fewer files in those offices. We  selected the most recent approval, denial, and termination files that were  available. In all, we reviewed 15 recently approved applications, six  recently denied applications, and three recent terminations. We also  interviewed cognizant SSA officials at the central office and the four  regional and area offices we selected.

We conducted background checks on a stratified random sample of 205  current organizational payees. The sample was selected to include fee- for-service organizations with 50 or more beneficiaries, fee-for-service  organizations with fewer than 50 beneficiaries, non-fee-for-service  organizations with 50 or more beneficiaries, and non-fee-for-service  organizations with fewer than 50 beneficiaries. We entered information on  selected organizations into a database called CLEAR and reviewed the  resulting reports for any indication of criminal history. Many of these  reports included the criminal history of individuals who are or may be  associated with the organizational payee, and we reviewed these with  particular focus on the crimes that bar individuals from serving as  individual payees. For those reports that contained an indication of  criminal history, we selected reports that indicated there may have been  federal crimes or felonies at the state or local level and attempted to  obtain court records to provide further insight into the nature of the crimes  and the outcome of the cases. However, because we lacked information  that would have made it possible for us to definitively link a conviction to  staff in an organization—such as Social Security numbers for payee staff  that are in leadership or financial management roles—the results of our  analysis were not reliable enough to report. SSA collects Social Security  numbers for individual payee applicants but not for any principals or staff  from organizational payee applicants. Without this information, it is  impossible to definitively link criminal convictions to individuals associated  with organizational payees.

To help understand how SSA communicates with organizational payees,  we reviewed program guidance and interviewed representatives of eight  organizational payees—one in the local area of each field office we  visited, in addition to interviewing officials in each field office. We also  interviewed cognizant SSA officials at the central office and the four  regional and four area offices we selected.

To review SSA’s overall oversight of organizational payees—including  onsite reviews and reviews of the annual accounting form and payees’  use of collective accounts—we reviewed relevant federal laws and  regulations, program policies, and relevant SSA documents; analyzed  data; and interviewed SSA officials at the central office, the four regional  and four area offices we selected, and the eight field offices we visited. To  further understand SSA’s new onsite review process, we reviewed  agency documents that describe the roles and responsibilities of key  players in SSA’s new onsite review process. We also interviewed SSA  officials and representatives of the National Disability Rights Network  (NDRN) about the status of its implementation.

To determine the extent to which different types of organizational payees  receive onsite reviews, we analyzed SSA program data for fiscal year  2018. We assessed the reliability of these data by reviewing relevant  documentation and interviewing knowledgeable agency officials and  determined they were sufficiently reliable for our purposes. To learn about  the outcomes of onsite reviews, such as how frequently unapproved  collective accounts were identified, we reviewed SSA’s annual reports to  Congress. We determined SSA data on the number of onsite reviews  conducted and SSA data reported to Congress on unapproved collective  accounts were sufficiently reliable for our purposes. We did not assess  the efficacy of the new onsite review process or the quality of onsite  reviews because we determined it was too soon to evaluate recent  program changes. Instead, we described the roles and responsibilities of  key players in the new process and interviewed SSA and NDRN to  provide information on the status of implementation.

To assess the predictive model SSA uses to select low-volume  organizational payees for onsite reviews, we analyzed available  documentation and interviewed SSA officials knowledgeable about the  predictive model. This information included: (1) a list of variables; (2) the  code SSA uses to execute the model; and (3) a brief description of how  SSA developed the model, including a high-level description of its  methodology and an analysis of the predictive power of the model  compared to random chance. We compared the documentation SSA  provided us with accepted practices for maintaining documentation of  statistical models. For detailed results on the findings of this analysis, see  appendix III.

To obtain a range of perspectives on the organizational payee program,  we interviewed staff of the Social Security Advisory Board,  representatives of an SSA managers’ association, an organizational  representative payee association, and NDRN. In addition, we interviewed  representatives of advocacy groups for the aged, persons with physical  disabilities, and persons with mental illness regarding their constituents’  experiences with SSA’s organizational payee program.

Appendix II: SSA Policy Regarding How to Evaluate Organizational Payee Applicants

Per Social Security Administration (SSA) policy, field office staff should  consider certain factors when evaluating organizations’ suitability to serve  as payees. Some factors apply to all applicants, including both  individuals and organizations, while others apply only to organizational  payee applicants (see table 2). In addition, there are some requirements  for organizational payees applying to collect fees for their payee  services.

According to SSA policy, organizational payees that are applying to  collect fees must meet the following requirements:

Be regularly serving as a payee for at least five beneficiaries for at  least 1 calendar month;

Generally not be a creditor of the beneficiaries it serves; and

Be a state or local agency with a qualified mission, or a non-profit  social service agency that is community-based, bonded, and  licensed.

Appendix III: Additional Information on SSA’s Predictive Model and Our Assessment

How SSA Selects Payees for Review Based on the Model

The Social Security Administration (SSA) uses a predictive statistical  model it implemented in 2012 to rank low-volume organizations based on  their chance of misusing beneficiary funds and selects for onsite reviews  those organizations identified as having the highest risk. The predictive  model uses a logistic regression to estimate the chance that each payee  will misuse benefits, given the characteristics of the beneficiary and  payee, such as the length of time served as a payee and whether the  beneficiary received a large lump sum payment from the payee. SSA  takes the predictive model output, which is calculated for every payee and  beneficiary pair, and uses it to rank payees. SSA assigns organizations  for review depending on (a) their rank (organizations that have a higher  likelihood of misusing benefits are more likely to be selected); and (b)  available resources.

To review the predictive model, we interviewed SSA officials  knowledgeable about the model and reviewed available documentation.  This documentation included: (1) a list of variables; (2) the computer code  SSA uses to execute the model; (3) a brief explanation of how SSA  periodically assesses the model and related performance statistics; and  (4) 2 documents (totaling 5 pages) describing how SSA developed the  model. We compared this documentation to accepted practices for  maintaining documentation of statistical analysis, such as standards  published by the Office of Management and Budget (OMB).

The documents describe, at a high level, SSA’s methodology for developing the model. It  also includes an analysis of the predictive power of the model compared to random  chance. of predictor variables, and ultimately selected a final model using a step- wise selection process. However, available documentation does not  include information necessary to evaluate how SSA assessed other  candidate models or understand the rationale for SSA’s decision to  accept its final model. For example, there is limited documentation to:

Reproduce SSA’s Target Population: The documentation does not  describe in detail how SSA identified all organizational payees that  served from 1993 to 2009 (such as how SSA queried the  Representative Payee System), nor does it explain in detail how SSA  linked beneficiary and organizational level data, such as to count the  number of beneficiaries that each payee served. SSA subsequently  explained in its technical comments that it used Social Security  numbers to link information among several systems. However, SSA  did not describe steps it took to establish the linkages, or steps taken  to identify organizational payees that served from 1993 to 2009, in  enough detail for an independent analyst to reproduce the work.  Moreover, SSA did not provide this written documentation upon our  original request, which suggests that SSA did not maintain complete  records of the work.

Reproduce SSA’s Sample Design: The documentation does not  describe in detail how SSA designed the probability sample it used to  develop the model or how, if at all, it weighted the sample to account  for varying probabilities of selection in the sample. Selecting the  appropriate sampling method for a model and applying appropriate  weights generally increases its predictive accuracy.

Reproduce SSA’s Process for Assembling the Data and  Selecting the Final Model: The documentation provides limited  information about the input variables and models that SSA tested but  ultimately did not use. In addition, the documentation does not show  how SSA assessed and addressed potential correlations between the  variables it selected. For example, we could expect certain variables,  such as receipt of a lump sum payment and receipt of a lump sum  payment over $1,000, to be highly correlated. Although highly  correlated variables do not necessarily impair the model’s predictive  accuracy, they can influence which individual variables test as being  predictive during the model’s development. The documentation also  does not describe how SSA chose to split continuous variables into  categorical variables—a choice which can influence predictive  accuracy.

Understand How SSA Assessed Data Reliability: Available  documentation does not indicate whether SSA assessed the reliability  of data used in its model. The reliability of the outcome variable— misuse—is particularly important. Unreliable data regarding whether  misuse occurred, either due to incorrect data entry or other errors,  would compromise the model’s ability to accurately predict the  likelihood of misuse. In contrast, the reliability of variables that could  signal risk of misuse—such as whether the beneficiary received a  large disbursement of funds—is less critical. Even variables prone to  measurement error may still predict misuse accurately. Nevertheless,  assessing their reliability remains important, since reducing  measurement error can increase the model’s predictive power. Such  assessments could range from limited testing of the data—e.g., for  outliers, illogical values, and missing data—to broader, independent  verification of data reliability. Regardless of the approach used,  documenting all data reliability assessments allows internal and  external stakeholders to assess, and possibly improve, the model.

Explain whether, or how, SSA’s model addressed potential  patterns of misuse for beneficiaries served by the same payee:  Statistical models typically assume that estimates can be generated  independently for each unit of analysis—in this case, unique pairs of  beneficiaries and payees. However, in cases where multiple  beneficiaries are served by the same payee, this may not be the case.  Patterns of misuse might be similar for all beneficiaries served by a  given payee, such as if the payee were systematically defrauding all  of its beneficiaries. Accurately modeling data with this kind of nested  structure—which conflicts with typical statistical assumptions—often  requires multi-level modeling methods. However, SSA’s  documentation does not specify how or whether it applied these  methods, or otherwise assessed or adjusted for the nesting of  beneficiaries within payees.

Reproduce SSA’s process for ranking organizations: With the  current model, which assigns a score for each payee-beneficiary pair,  SSA uses the predictive model’s output to then rank payees.  However, there are various approaches for ranking payees, ranging in  sophistication, and SSA does not have sufficient documentation to  determine whether the approach currently being used best predicts  risk to beneficiaries.

Appendix IV: Comments from the Social Security Administration

Appendix V: GAO Contact and Staff Acknowledgments

In addition to the contact named above, Michele Grgich (Assistant  Director), Isabella P. Anderson, Dan Meyer and Amy E. MacDonald  (Analysts-in-Charge), Daniel Bibeault, Ted Burik, Daniel Concepcion,  Jennifer Cook, Gus Fernandez, Alex Galuten, Sheila R. McCoy, Arthur  Thomas Merriam, Jr., Mimi Nguyen, Ramon J. Rodriguez, Margie K.  Shields, Joy Solmonson, Almeta Spencer, Jeff M. Tessin, Walter K.  Vance, Kathleen van Gelder, Srinidhi Vijaykumar, and Khristi A. Wilkins  made significant contributions to this report.

In addition, Seto J. Bagdoyan, Joy Booth, Gabrielle M. Fagan, Robert H.  Graves, Rosalind C. Romain, and Helina P. Wong contributed to the  report.

Some Social Security beneficiaries rely on organizations like non-profits or nursing homes to help them manage their benefits. The Social Security Administration monitors these organizations to ensure benefits are not being misused.

Yet SSA does not require background checks for key employees of such organizations as it does for relatives or friends that assist beneficiaries. In addition, SSA accounting forms make it difficult to track in detail how these funds are spent.

We made 9 recommendations, including that SSA redesign its accounting forms and look into requiring background checks for employees at these organizations.