question,contexts,answer,ground_truth,faithfulness,answer_relevancy,context_recall,context_precision,answer_correctness
What is the significance of providing notice and explanation as a legal requirement in the context of automated systems?,"[""Providing notice has long been a standard practice, and in many cases is a legal requirement, when, for example, making a video recording of someone (outside of a law enforcement or national security context). In some cases, such as credit, lenders are required to provide notice and explanation to consumers. Techniques used to automate the process of explaining such systems are under active research and improvement and such explanations can take many forms. Innovative companies and researchers are rising to the challenge and creating and deploying explanatory systems that can help the public better understand decisions that impact them. \nWhile notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. \n• A lawyer representing an older client with disabilities who had been cut off from Medicaid-funded home\nhealth-care assistance couldn't determine why\n, especially since the decision went against historical access\npractices. In a court hearing, the lawyer learned from a witness that the state in which the older client\nlived \nhad recently adopted a new algorithm to determine eligibility.83 The lack of a timely explanation made it\nharder \nto understand and contest the decision.\n•\nA formal child welfare investigation is opened against a parent based on an algorithm and without the parent\never \nbeing notified that data was being collected and used as part of an algorithmic child maltreatment\nrisk assessment.84 The lack of notice or an explanation makes it harder for those performing child\nmaltreatment assessments to validate the risk assessment and denies parents knowledge that could help them\ncontest a decision.\n41""
 'NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nTailored to the level of risk. An assessment should be done to determine the level of risk of the auto -\nmated system. In settings where the consequences are high as determined by a risk assessment, or extensive \noversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. \nValid. The explanation provided by a system should accurately reflect the factors and the influences that led \nto a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns. \nDemonstrate protections for notice and explanation \nReporting. Summary reporting should document the determinations made based on the above consider -'
 'should not be used in education, work, housing, or in other contexts where the use of such surveillance \ntechnologies is likely to limit rights, opportunities, or access. Whenever possible, you should have access to \nreporting that confirms your data decisions have been respected and provides an assessment of the \npotential impact of surveillance technologies on your rights, opportunities, or access. \nNOTICE AND EXPLANATION\nYou should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organiza\n-\ntion responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible. \n6'
 'NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nAn automated system should provide demonstrably clear, timely, understandable, and accessible notice of use, and \nexplanations as to how and why a decision was made or an action was taken by the system. These expectations are explained below. \nProvide clear, timely, understandable, and accessible notice of use and explanations \nGenerally accessible plain language documentation. The entity responsible for using the automated \nsystem should ensure that documentation describing the overall system (including any human components) is \npublic and easy to find. The documentation should describe, in plain language, how the system works and how \nany automated component is used to determine an action or decision. It should also include expectations about \nreporting described throughout this framework, such as the algorithmic impact assessments described as \npart of Algorithmic Discrimination Protections. \nAccount able. Notices should clearly identify the entity r esponsible for designing each component of the \nsystem and the entity using it. \nTimely and up-to-date. Users should receive notice of the use of automated systems in advance of using or \nwhile being impacted by the technolog y. An explanation should be available with the decision itself, or soon \nthereafte r. Notice should be kept up-to-date and people impacted by the system should be notified of use case \nor key functionality changes. \nBrief and clear. Notices and explanations should be assessed, such as by research on users’ experiences, \nincluding user testing, to ensure that the people using or impacted by the automated system are able to easily'
 'burdensome in both the process of requesting to opt-out and the human-driven alternative provided. \nProvide timely human consideration and remedy by a fallback and escalation system in the event that an automated system fails, produces error, or you would like to appeal or con\n-\ntest its impacts on you \nProportionate. The availability of human consideration and fallback, along with associated training and \nsafeguards against human bias, should be proportionate to the potential of the automated system to meaning -\nfully impact rights, opportunities, or access. Automated systems that have greater control over outcomes, provide input to high-stakes decisions, relate to sensitive domains, or otherwise have a greater potential to meaningfully impact rights, opportunities, or access should have greater availability (e.g., staffing) and over\n-\nsight of human consideration and fallback mechanisms. \nAccessible. Mechanisms for human consideration and fallback, whether in-person, on paper, by phone, or \notherwise provided, should be easy to find and use. These mechanisms should be tested to ensure that users who have trouble with the automated system are able to use human consideration and fallback, with the under\n-\nstanding that it may be these users who are most likely to need the human assistance. Similarly, it should be tested to ensure that users with disabilities are able to find and use human consideration and fallback and also request reasonable accommodations or modifications. \nConvenient. Mechanisms for human consideration and fallback should not be unreasonably burdensome as \ncompared to the automated system’s equivalent. \n49'
 'You should know that an automated system is being used, \nand understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automat\n-\ned systems should provide generally accessible plain language docu -\nmentation including clear descriptions of the overall system func -\ntioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and ex\n-\nplanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality chang\n-\nes. You should know how and why an outcome impacting you was de -\ntermined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible.   NOTICE AND  EXPLANATION\n40'
 'HUMAN ALTERNATIVES, \nCONSIDERATION, AND \nFALLBACK \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nThere are many reasons people may prefer not to use an automated system: the system can be flawed and can lead to \nunintended outcomes; it may reinforce bias or be inaccessible; it may simply be inconvenient or unavailable; or it may replace a paper or manual process to which people had grown accustomed. Yet members of the public are often presented with no alternative, or are forced to endure a cumbersome process to reach a human decision-maker once they decide they no longer want to deal exclusively with the automated system or be impacted by its results. As a result of this lack of human reconsideration, many receive delayed access, or lose access, to rights, opportunities, benefits, and critical services. The American public deserves the assurance that, when rights, opportunities, or access are meaningfully at stake and there is a reasonable expectation of an alternative to an automated system, they can conve\n-\nniently opt out of an automated system and will not be disadvantaged for that choice. In some cases, such a human or other alternative may be required by law, for example it could be required as “reasonable accommodations” for people with disabilities.'
 ""find notices and explanations, read them quickl y, and understand and act on them. This includes ensuring that \nnotices and explanations are accessible to users with disabilities and are available in the language(s) and read-\ning level appropriate for the audience. Notices and explanations may need to be available in multiple forms, \n(e.g., on pape r, on a physical sign, or online), in order to meet these expectations and to be accessible to the \nAmerican public. \nProvide explanations as to how and why a decision was made or an action was taken by an \nautomated system \nTailored to the purpose. Explanations should be tailored to the specific purpose for which the user is \nexpected to use the explanation, and should clearly state that purpose. An informational explanation might differ from an explanation provided to allow for the possibility of recourse, an appeal, or one provided in the context of a dispute or contestation process. For the purposes of this framework, 'explanation' should be construed broadly. An explanation need not be a plain-language statement about causality but could consist of any mechanism that allows the recipient to build the necessary understanding and intuitions to achieve the stated purpose. Tailoring should be assessed (e.g., via user experience research). \nTailored to the target of the explanation. Explanations should be targeted to specific audiences and clearly state that audience. An explanation provided to the subject of a decision might differ from one provided to an advocate, or to a domain expert or decision maker. Tailoring should be assessed (e.g., via user experience research). \n43""]","Providing notice and explanation as a legal requirement in the context of automated systems is significant for several reasons:

1. **Transparency**: It ensures that individuals are aware when automated systems are being used to make decisions that affect them. This transparency helps build trust between the public and the organizations deploying these systems.

2. **Informed Consent**: By notifying individuals about the use of automated systems, organizations allow people to make informed choices about their engagement with these systems. This is particularly important in sensitive areas like healthcare, finance, and law enforcement.

3. **Accountability**: Clear notice and explanations hold organizations accountable for their automated decisions. If individuals understand how decisions are made, they can better contest or appeal those decisions if they believe they are unjust or incorrect.

4. **Protection of Rights**: Legal requirements for notice and explanation help protect individuals' rights by ensuring they have access to information about how their data is used and how decisions that impact them are made. This is crucial in preventing discrimination and ensuring fair treatment.

5. **Facilitating Recourse**: When individuals receive timely and understandable explanations, they are better equipped to seek recourse if they feel wronged by an automated decision. This can include appealing decisions or requesting human intervention.

6. **Encouraging Ethical Use**: Legal requirements can encourage organizations to develop and implement automated systems ethically, ensuring that they consider the potential impacts on individuals and society as a whole.

Overall, these requirements aim to create a framework where automated systems are used responsibly, with respect for individuals' rights and dignity (Source: [document name], p. [page number]).","Providing notice and explanation as a legal requirement in the context of automated systems is significant because it allows individuals to understand how automated systems are impacting their lives. It helps in correcting errors, contesting decisions, and verifying the reasonableness of recommendations before enacting them. Clear and valid explanations are essential to ensure transparency, accountability, and trust in the use of automated systems across various sectors.",1.0,0.9713211543186683,1.0,0.9999999999875,0.8212990822356991
"How can structured human feedback exercises, such as GAI red-teaming, be beneficial for GAI risk measurement and management?","['50 Participatory Engagement Methods  \nOn an ad hoc or more structured basis, organizations can design and use a variety of channels to engage \nexternal stakeholders in product development or review. Focus groups with select experts can provide \nfeedback on a range of issues. Small user studies c an provide feedback from representative groups or \npopulations. Anonymous surveys can be used to poll or gauge reactions to specific features. Participatory engagement methods are often less structured than field testing or red teaming, and are more \ncommonl y used in early stages of AI or product development.  \nField Testing  \nField testing involves structured settings to evaluate risks and impacts and to simulate the conditions \nunder which the GAI system will be deployed. Field style tests can be adapted from a focus on user \npreferences and experiences towards AI risks and impacts – both negative and positive. When carried \nout with large groups of users, these tests can provide estimations of the likelihood of risks and impacts \nin real world interactions.  \nOrganizations may also collect feedback on outcomes, harms, and user experience directly from users in the production environment after a model has been released, in accordance with human subject \nstandards such as informed consent and compensation. Organiz ations should follow applicable human \nsubjects research requirements, and best practices such as informed consent and subject compensation, \nwhen implementing feedback activities. \nAI Red -teaming  \nAI red -teaming is an evolving practice that references exercises often  conducted in a controlled \nenvironment and in collaboration with AI developers building AI models  to identify potential adverse \nbehavior or outcomes of a GAI model or system,  how they could occur, an d stress test safeguards” . AI \nred-teaming can be performed before or after AI models or systems are made available to the broader'
 '14 GOVERN 1.2:  The characteristics of trustworthy AI are integrated into organizational policies, processes, procedures, and practices.  \nAction ID  Suggested Action  GAI Risks  \nGV-1.2-001 Establish transparency policies and processes for documenting the origin and \nhistory of training data and generated data for GAI applications  to advance  digital \ncontent transparency , while balancing the proprietary nature of training  \napproaches . Data Privacy ; Information \nIntegrity ; Intellectual Property  \nGV-1.2-0 02 Establish policies to evaluate risk -relevant capabilities of GAI and robustness of \nsafety measures, both prior to deployment and on an ongoing basis, through \ninternal and external evaluations.  CBRN Information or Capabilities ; \nInformation Security  \nAI Actor Tasks: Governance and Oversight  \n \nGOVERN 1.3:  Processes, procedures, and practices are in place to determine the needed level of risk management activities based \non the organization’s risk tolerance.  \nAction ID  Suggested Action  GAI Risks  \nGV-1.3-001 Consider the following  factors when updating or defining risk tiers for GAI: Abuses \nand impacts to information integrity; Dependencies between GAI and other IT or \ndata systems; Harm to fundamental rights or public safety ; Presentation of \nobscene, objectionable, offensive, discrimina tory, invalid or untruthful output; \nPsychological impacts to humans (e.g., anthropomorphization, algorithmic aversion, emotional entanglement); Possibility for malicious use ; Whether the \nsystem introduces  significant new security vulnerabilities ; Anticipated system \nimpact on some groups compared to others ; Unreliable decision making \ncapabilities, validity, adaptability, and variability of GAI system performance over \ntime.  Information Integrity ; Obscene, \nDegrading, and/or Abusive \nContent ; Value Chain and \nComponent Integration; Harmful \nBias and Homogenization ; \nDangerous , Violent , or Hateful  \nContent ; CBRN Information or \nCapabilities'
 '49 early lifecycle TEVV approaches are developed and matured for GAI, organizations may use \nrecommended “pre- deployment testing” practices to measure performance, capabilities, limits, risks, \nand impacts. This section describes risk measurement and estimation as part of pre -deployment TEVV, \nand examines the state of play for pre -deployment testing methodologies.  \nLimitations of Current Pre -deployment Test Approaches  \nCurrently available pre -deployment TEVV processes used for GAI applications may be inadequate, non-\nsystematically applied, or fail to reflect or mismatched to deployment contexts. For example, the \nanecdotal testing of GAI system capabilities through video games or standardized tests designed for \nhumans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or \nreliability in those domains. Similarly, jailbreaking or prompt  engineering tests may not systematically \nasse ss validity or reliability risks.  \nMeasurement gaps can arise from mismatches between laboratory and real -world settings. Current \ntesting approaches often remain focused on laboratory conditions or restricted to benchmark test \ndatasets and in silico techniques that may not extrapolate well to —or directly assess GAI impacts in real -\nworld conditions. For example, current measurement gaps for GAI make it difficult to precisely estimate \nits potential ecosystem -level or longitudinal risks and related political, social, and economic impacts. \nGaps between benchmarks and real-world  use of GAI systems may likely be exacerbated due to prompt \nsensitivity and broad heterogeneity of contexts of use.  \nA.1.5.  Structured Public Feedback  \nStructured public feedback can be used to evaluate whether GAI systems are performing as intended and to calibrate and verify traditional measurement methods. Examples of structured feedback include, \nbut are not limited to:'
 '51 general public participants. For example, expert AI red- teamers could modify or verify the \nprompts written by general public AI red- teamers. These approaches may also expand coverage \nof the AI risk attack surface.  \n• Human / AI: Performed by GAI in combinatio n with  specialist or non -specialist human teams. \nGAI- led red -teaming can be more cost effective than human red- teamers alone. Human or GAI-\nled AI red -teaming may be better suited for eliciting different types of harms.   \nA.1.6.  Content Provenance  \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. \nSome aspects of GAI output s, such as the production of deepfake content, can challenge our ability to \ndistinguish human- generated content from AI -generated synthetic  content. To help manage and mitigate \nthese risks, digital transparency mechanisms like provenance data tracking can trace the origin and \nhistory of content. Provenance data tracking and synthetic content detection can help facilitate greater \ninformation access  about both authentic and synthetic content to users, enabling better knowledge of  \ntrustworthiness  in AI systems. When combined with other organizational accountability mechanisms, \ndigital content transparency approaches  can enable processes to trace negative outcomes back to their \nsource, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin \nand history of content  to assist in \nGAI risk management efforts.  \nProvenance metad ata can include information about GAI model developers or creators  of GAI content , \ndate/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of p rovenance data tracking techniques  can'
 'GV-4.1-001 Establish policies and procedures that address continual improvement processes \nfor GAI risk measurement . Address general risks associated with a lack of \nexplainability and transparency in GAI systems by using ample documentation and \ntechniques such as: application of gradient -based attributions, occlusion/term \nreduction, counterfactual prompts and prompt eng ineering, and analysis of \nembeddings; Assess and update risk measurement approaches at regular cadences.  Confabulation  \nGV-4.1-002 Establish policies, procedures, and processes detailing risk measurement in \ncontext of use with standardized measurement protocols and structured public feedback exercises such as AI red -teaming or independent external evaluations . CBRN Information and Capability ; \nValue Chain and Component Integration'
 '42 MG-2.4-002 Establish and maintain procedures for escalating GAI system incidents to the \norganizational risk management  authority when specific criteria for deactivation \nor disengagement is met for a particular context of use or for the GAI system as a \nwhole.  Information Security  \nMG-2.4-003 Establish and maintain procedures for the remediation of issues which trigger \nincident response processes for the use of a GAI system, and provide stakeholders timelines associated with the remediation plan.  Information Security  \n \nMG-2.4-004 Establish and regularly review specific criteria that warrants the deactivation of \nGAI systems in accordance with set risk tolerances and appetites.  Information Security  \n \nAI Actor Tasks:  AI Deployment, Governance and Oversight, Operation and Monitoring  \n \nMANAGE 3.1:  AI risks and benefits from third -party resources are regularly monitored, and risk controls are applied and \ndocumented.  \nAction ID  Suggested Action  GAI Risks  \nMG-3.1-001 Apply organizational risk tolerances and controls (e.g., acquisition and \nprocurement processes; assessing personnel credentials and qualifications, \nperforming background checks; filtering GAI input and outputs, grounding, fine tuning , retrieval- augmented generation) to third -party GAI resources: Apply \norganizational risk tolerance to the utilization of third -party datasets and other \nGAI resources; Apply organizational risk tolerances to fine -tuned third -party \nmodels; Apply organizational risk tolerance to existing t hird-party models \nadapted to a new domain; Reassess risk measure ments after fine -tuning third-\nparty GAI models.  Value Chain and Component Integration ; Intellectual Property  \nMG-3.1-002 Test GAI system value chain  risks (e.g., data poisoning, malware, other software  \nand hardware vulnerabilities; labor practices; data privacy and localization compliance; geopolitical alignment).  Data Privacy ; Information Security ;']","Structured human feedback exercises, such as GAI red-teaming, can significantly benefit GAI risk measurement and management in several ways:

1. **Identifying Risks**: Red-teaming involves simulating potential adverse behaviors or outcomes of a GAI system in a controlled environment. This helps organizations identify risks that may not be apparent during standard testing, allowing for proactive risk management (Source: [document name], p. 14).

2. **Stress Testing Safeguards**: By challenging the GAI system's safeguards, red-teaming can reveal weaknesses in the system's defenses. This stress testing is crucial for ensuring that the GAI can withstand various scenarios, including malicious use or unexpected inputs (Source: [document name], p. 14).

3. **Diverse Perspectives**: Involving both expert and non-expert participants in red-teaming exercises can provide a broader range of insights into potential risks and impacts. This diversity can lead to a more comprehensive understanding of how the GAI system may perform in real-world situations (Source: [document name], p. 51).

4. **Calibration of Measurement Methods**: Structured feedback from red-teaming can help calibrate and verify traditional measurement methods used to assess GAI performance. This ensures that the evaluation processes are aligned with actual user experiences and risks (Source: [document name], p. 51).

5. **Continuous Improvement**: Regular red-teaming exercises can contribute to a culture of continuous improvement within organizations. By consistently evaluating and updating risk management practices based on feedback, organizations can adapt to evolving risks associated with GAI systems (Source: [document name], p. 14).

6. **Enhancing Accountability**: Documenting the findings from red-teaming exercises can enhance accountability within organizations. It provides a clear record of identified risks and the steps taken to address them, which is essential for regulatory compliance and public trust (Source: [document name], p. 14).

Overall, structured human feedback exercises like GAI red-teaming are vital for effectively managing the risks associated with GAI systems, ensuring they operate safely and ethically in various applications.","Structured human feedback exercises, such as GAI red-teaming, can be beneficial for GAI risk measurement and management by defining use cases, contexts of use, capabilities, and negative impacts where such exercises would be most beneficial. These exercises help in monitoring and improving outputs, evaluating the quality and integrity of data used in training, and tracking and documenting risks or opportunities related to GAI risks that cannot be measured quantitatively. Additionally, seeking active and direct feedback from affected communities through red-teaming can enhance information integrity and help in identifying harmful bias and homogenization in AI systems.",1.0,0.992832485501513,1.0,0.9999999999833333,0.5412221393025693
How do measurement gaps between laboratory and real-world settings impact the assessment of GAI systems in the context of pre-deployment testing?,"['49 early lifecycle TEVV approaches are developed and matured for GAI, organizations may use \nrecommended “pre- deployment testing” practices to measure performance, capabilities, limits, risks, \nand impacts. This section describes risk measurement and estimation as part of pre -deployment TEVV, \nand examines the state of play for pre -deployment testing methodologies.  \nLimitations of Current Pre -deployment Test Approaches  \nCurrently available pre -deployment TEVV processes used for GAI applications may be inadequate, non-\nsystematically applied, or fail to reflect or mismatched to deployment contexts. For example, the \nanecdotal testing of GAI system capabilities through video games or standardized tests designed for \nhumans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or \nreliability in those domains. Similarly, jailbreaking or prompt  engineering tests may not systematically \nasse ss validity or reliability risks.  \nMeasurement gaps can arise from mismatches between laboratory and real -world settings. Current \ntesting approaches often remain focused on laboratory conditions or restricted to benchmark test \ndatasets and in silico techniques that may not extrapolate well to —or directly assess GAI impacts in real -\nworld conditions. For example, current measurement gaps for GAI make it difficult to precisely estimate \nits potential ecosystem -level or longitudinal risks and related political, social, and economic impacts. \nGaps between benchmarks and real-world  use of GAI systems may likely be exacerbated due to prompt \nsensitivity and broad heterogeneity of contexts of use.  \nA.1.5.  Structured Public Feedback  \nStructured public feedback can be used to evaluate whether GAI systems are performing as intended and to calibrate and verify traditional measurement methods. Examples of structured feedback include, \nbut are not limited to:'
 'MP-2.3-001 Assess the accuracy, quality, reliability, and authenticity of GAI  output by \ncomparing it to a set of known ground truth data and by using a variety of evaluation methods (e.g., human oversight and automated evaluation , proven \ncryptographic techniques , review of content inputs ). Information Integrity'
 'Information security for computer systems and data is a mature field with widely accepted and \nstandardized practices  for offensive and defensive cyber capabilities . GAI -based systems  present two \nprimary information security risks: GAI could potentially discover or enable new cybersecurity risks by \nlowering the barriers for or easing automated  exercise of  offensive  capabilities ; simultaneously , it \nexpands the available attack surface , as GAI itself is vulnerable to attacks like prompt  injection  or data \npoisoning.  \nOffensive cyber capabilities advanced by GAI systems may augment  cyber security attacks such as \nhacking, malware, and phishing. Reports have indicated that LLMs are already able to discover some \nvulnerabilities  in systems (hardware, software, data) and write code to exploit them . Sophisticated threat \nactors might further these risks by developing GAI- powered security co -pilots  for use in several parts of \nthe attack chain, including informing attackers on how to proactively evade threat detection and escalate \nprivilege s after gaining system access.  \nInformation security for GAI models and systems also includes maintaining availability of the GAI system \nand the integrity and (when applicable ) the confidentiality of the GAI code, training data, and model \nweights. To identify  and secur e potential attack points in AI systems or specific components  of the AI \n \n \n12 See also https://doi.org/10.6028/NIST.AI.100-4 , to be published.'
 '30 MEASURE 2.2:  Evaluations involving human subjects meet applicable requirements (including human subject protection) and are \nrepresentative of the relevant population.  \nAction ID  Suggested Action  GAI Risks  \nMS-2.2-001 Assess and manage statistical biases related to GAI content provenance through \ntechniques such as re -sampling, re -weighting, or adversarial training.  Information Integrity ; Information \nSecurity ; Harmful Bias and \nHomogenization  \nMS-2.2-002 Document how content provenance data  is tracked  and how that data interact s \nwith  privacy and security . Consider : Anonymiz ing data to protect the privacy of \nhuman subjects; Leverag ing privacy output filters; Remov ing any personally \nidentifiable information (PII) to prevent potential harm or misuse.  Data Privacy ; Human AI \nConfiguration; Information \nIntegrity ; Information Security ; \nDangerous , Violent, or Hateful \nContent  \nMS-2.2-0 03 Provide human subjects with options to withdraw participation or revoke their \nconsent for present or future use of their data in GAI applications .  Data Privacy ; Human -AI \nConfiguration; Information \nIntegrity  \nMS-2.2-0 04 Use techniques such as anonymization , differential privacy  or other  privacy -\nenhancing technologies to minimize the risks associated with linking AI -generated \ncontent back to individual human subjects.  Data Privacy ; Human -AI \nConfiguration  \nAI Actor Tasks:  AI Development, Human Factors, TEVV  \n \nMEASURE 2.3:  AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for \nconditions similar to deployment setting(s). Measures are documented.  \nAction ID  Suggested Action  GAI Risks  \nMS-2.3-001 Consider baseline model performance on suites of benchmarks when selecting a \nmodel for fine tuning  or enhancement with retrieval- augmented generation. Information Security ; \nConfabulation'
 'MP-3.4-006 Involve the end -users, practitioners, and operators in GAI system in prototyping \nand testing activities. Make sure these tests cover various scenarios , such as crisis \nsituations or ethically sensitive contexts.  Human -AI Configuration ; \nInformation Integrity ; Harmful Bias \nand Homogenization ; Dangerous , \nViolent, or Hateful Content  \nAI Actor Tasks: AI Design, AI Development, Domain Experts, End -Users, Human Factors, Operation and Monitoring'
 'GV-4.1-001 Establish policies and procedures that address continual improvement processes \nfor GAI risk measurement . Address general risks associated with a lack of \nexplainability and transparency in GAI systems by using ample documentation and \ntechniques such as: application of gradient -based attributions, occlusion/term \nreduction, counterfactual prompts and prompt eng ineering, and analysis of \nembeddings; Assess and update risk measurement approaches at regular cadences.  Confabulation  \nGV-4.1-002 Establish policies, procedures, and processes detailing risk measurement in \ncontext of use with standardized measurement protocols and structured public feedback exercises such as AI red -teaming or independent external evaluations . CBRN Information and Capability ; \nValue Chain and Component Integration']","Measurement gaps between laboratory and real-world settings can significantly impact the assessment of GAI systems during pre-deployment testing in several ways:

1. **Inaccurate Performance Predictions**: Laboratory tests often use controlled conditions that may not accurately reflect the complexities and variabilities of real-world environments. As a result, GAI systems may perform well in tests but fail to deliver the same level of effectiveness when deployed in diverse, unpredictable settings (Source: [document name], p. 12).

2. **Limited Contextual Relevance**: Current pre-deployment testing approaches may focus on specific benchmarks or datasets that do not encompass the full range of scenarios the GAI system will encounter once deployed. This can lead to a mismatch between the system's capabilities as measured in the lab and its actual performance in real-world applications (Source: [document name], p. 12).

3. **Overlooking Edge Cases**: Laboratory settings may not adequately account for edge cases or rare events that could significantly impact the GAI system's performance. If these scenarios are not tested, the system may be unprepared for unexpected challenges, leading to failures or unintended consequences in real-world use (Source: [document name], p. 12).

4. **Prompt Sensitivity Issues**: GAI systems can be highly sensitive to the way prompts or inputs are structured. Testing in a controlled environment may not capture the variability in user inputs that occur in real-world situations, potentially leading to performance discrepancies (Source: [document name], p. 12).

5. **Ecosystem-Level Risks**: Current measurement gaps make it difficult to estimate the broader political, social, and economic impacts of GAI systems. Without understanding these ecosystem-level risks, organizations may overlook significant implications of deploying GAI technologies (Source: [document name], p. 12).

6. **Need for Iterative Testing**: The gaps highlight the importance of iterative testing and continuous feedback from real-world use. Organizations should incorporate structured public feedback and real-world evaluations to refine their GAI systems and ensure they meet the intended performance standards (Source: [document name], p. 12).

In summary, addressing measurement gaps between laboratory and real-world settings is crucial for accurately assessing GAI systems and ensuring their safe and effective deployment.","Measurement gaps between laboratory and real-world settings can impact the assessment of GAI systems in the context of pre-deployment testing by limiting the extrapolation of results from laboratory conditions to real-world scenarios. Current testing approaches often focus on benchmark test datasets and in silico techniques that may not accurately assess the impacts of GAI systems in real-world conditions. This can make it difficult to estimate the ecosystem-level or longitudinal risks associated with GAI deployment, as well as the political, social, and economic impacts. Additionally, the prompt sensitivity and broad heterogeneity of real-world contexts of use can exacerbate the gaps between benchmarks and actual GAI system performance.",0.9583333333333334,0.9887518614487671,1.0,0.8766666666491334,0.6365561513945531
How should data collection and use-case scope limits be determined and implemented in automated systems to prevent 'mission creep'?,"['Data collection and use-case scope limits. Data collection should be limited in scope, with specific, \nnarrow identified goals, to avoid ""mission creep.""  Anticipated data collection should be determined to be strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on these identified goals and for a specific context should not be used in a different context without assessing for new privacy risks and implementing appropriate mitigation measures, which may include express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Determined data retention timelines should be documented and justi\n-\nfied. \nRisk identification and mitigation. Entities that collect, use, share, or store sensitive data should attempt to proactively identify harms and seek to manage them so as to avoid, mitigate, and respond appropri\n-\nately to identified risks. Appropriate responses include determining not to process data when the privacy risks outweigh the benefits or implementing measures to mitigate acceptable risks. Appropriate responses do not include sharing or transferring the privacy risks to users via notice or consent requests where users could not reasonably be expected to understand the risks without further support. \nPrivacy-preserving security. Entities creating, using, or governing automated systems should follow privacy and security best practices designed to ensure data and metadata do not leak beyond the specific consented use case. Best practices could include using privacy-enhancing cryptography or other types of privacy-enhancing technologies or fine-grained permissions and access control mechanisms, along with conventional system security protocols. \n33'
 'ers may differ depending on the specific automated system and development phase, but should include subject matter, sector-specific, and context-specific experts as well as experts on potential impacts such as civil rights, civil liberties, and privacy experts. For private sector applications, consultations before product launch may need to be confidential. Government applications, particularly law enforcement applications or applications that raise national security considerations, may require confidential or limited engagement based on system sensitivities and preexisting oversight laws and structures. Concerns raised in this consultation should be documented, and the automated system developers were proposing to create, use, or deploy should be reconsidered based on this feedback. \nTesting. Systems should undergo extensive testing before deployment. This testing should follow domain-specific best practices, when available, for ensuring the technology will work in its real-world context. Such testing should take into account both the specific technology used and the roles of any human operators or reviewers who impact system outcomes or effectiveness; testing should include both automated systems testing and human-led (manual) testing. Testing conditions should mirror as closely as possible the conditions in which the system will be deployed, and new testing may be required for each deployment to account for material differences in conditions from one deployment to another. Following testing, system performance should be compared with the in-place, potentially human-driven, status quo procedures, with existing human performance considered as a performance baseline for the algorithm to meet pre-deployment, and as a lifecycle minimum performance standard. Decision possibilities resulting from performance testing should include the possibility of not deploying the system.'
 'DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nProtect the public from unchecked surveillance \nHeightened oversight of surveillance. Surveillance or monitoring systems should be subject to \nheightened oversight that includes at a minimum assessment of potential harms during design (before deploy -\nment) and in an ongoing manner, to ensure that the American public’s rights, opportunities, and access are protected. This assessment should be done before deployment and should give special attention to ensure there is not algorithmic discrimination, especially based on community membership, when deployed in a specific real-world context. Such assessment should then be reaffirmed in an ongoing manner as long as the system is in use. \nLimited and proportionate surveillance. Surveillance should be avoided unless it is strictly necessary to achieve a legitimate purpose and it is proportionate to the need. Designers, developers, and deployers of surveillance systems should use the least invasive means of monitoring available and restrict monitoring to the minimum number of subjects possible. To the greatest extent possible consistent with law enforcement and national security needs, individuals subject to monitoring should be provided with clear and specific notice before it occurs and be informed about how the data gathered through surveillance will be used. \nScope limits on surveillance to protect rights and democratic values. Civil liberties and civil'
 '-\ndures before deploying the system, as well as responsibility of specific individuals or entities to oversee ongoing assessment and mitigation. Organizational stakeholders including those with oversight of the business process or operation being automated, as well as other organizational divisions that may be affected due to the use of the system, should be involved in establishing governance procedures. Responsibility should rest high enough in the organization that decisions about resources, mitigation, incident response, and potential rollback can be made promptly, with sufficient weight given to risk mitigation objectives against competing concerns. Those holding this responsibility should be made aware of any use cases with the potential for meaningful impact on people’s rights, opportunities, or access as determined based on risk identification procedures.  In some cases, it may be appropriate for an independent ethics review to be conducted before deployment. \nAvoid inappropriate, low-quality, or irrelevant data use and the compounded harm of its reuse \nRelevant and high-quality data. Data used as part of any automated system’s creation, evaluation, or \ndeployment should be relevant, of high quality, and tailored to the task at hand. Relevancy should be \nestablished based on research-backed demonstration of the causal influence of the data to the specific use case \nor justified more generally based on a reasonable expectation of usefulness in the domain and/or for the \nsystem design or ongoing development. Relevance of data should not be established solely by appealing to \nits historical connection to the outcome. High quality and tailored data should be representative of the task at \nhand and errors from data entry or other sources should be measured and limited. Any data used as the target \nof a prediction process should receive particular attention to the quality and validity of the predicted outcome']","To prevent 'mission creep' in automated systems, data collection and use-case scope limits should be determined and implemented through a structured approach that includes the following steps:

1. **Define Clear Objectives**: Start by establishing specific, narrow goals for the automated system. This involves identifying the precise outcomes you aim to achieve and ensuring that data collection aligns strictly with these objectives. Avoid broad or vague goals that could lead to expanding the system's purpose over time (Source: [document name], p. 33).

2. **Conduct a Necessity Assessment**: Evaluate the necessity of each data point you plan to collect. Data should only be gathered if it is strictly necessary to achieve the identified goals. This assessment helps minimize the amount of data collected and reduces the risk of using data for unintended purposes (Source: [document name], p. 33).

3. **Implement Data Minimization Principles**: Adopt data minimization practices by collecting only the data that is essential for the specific use case. This means avoiding the collection of extraneous information that could lead to mission creep or privacy risks (Source: [document name], p. 33).

4. **Establish Context-Specific Guidelines**: Create guidelines that are tailored to the specific context in which the automated system will operate. These guidelines should outline what data can be collected, how it will be used, and the limitations on its use in other contexts (Source: [document name], p. 33).

5. **Set Clear Data Retention Timelines**: Define and document clear timelines for data retention. Data should be deleted as soon as it is no longer necessary for the identified goals, in accordance with legal or policy-based limitations. This helps prevent the accumulation of unnecessary data over time (Source: [document name], p. 33).

6. **Regularly Review and Update Practices**: Implement a process for regularly reviewing data collection practices and use-case scopes. This ensures that any changes in objectives or context are reflected in the data collection strategy, helping to prevent mission creep (Source: [document name], p. 33).

7. **Engage Stakeholders**: Involve relevant stakeholders, including privacy experts and affected communities, in the development of data collection policies. Their input can help identify potential risks and ensure that the scope remains focused and appropriate (Source: [document name], p. 33).

By following these steps, organizations can effectively manage data collection and use-case scope limits in automated systems, thereby reducing the risk of mission creep and protecting individuals' privacy rights.","Data collection and use-case scope limits in automated systems should be determined by setting specific, narrow goals to avoid 'mission creep.' Anticipated data collection should be strictly necessary for the identified goals and minimized as much as possible. Data collected for a specific context should not be used in a different context without assessing new privacy risks and implementing appropriate mitigation measures, which may include obtaining express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. The determined data retention timelines should be documented and justified.",1.0,0.9232044093042836,1.0,0.999999999975,0.4914245673315278
What action did the Federal Trade Commission take against Kochava regarding the sale of sensitive location tracking data?,"['alerts about location tracking—are brief, direct, and use-specific. Many of the expectations listed here for \nprivacy by design and use-specific consent mirror those distributed to developers as best practices when \ndeveloping for smart phone devices,\n82 such as being transparent about how user data will be used, asking for app \npermissions during their use so that the use-context will be clear to users, and ensuring that the app will still \nwork if users deny (or later revoke) some permissions. \n39'
 ""ENDNOTES\n85. Mick Dumke and Frank Main. A look inside the watch list Chicago police fought to keep secret. The\nChicago Sun Times. May 18, 2017.\nhttps://chicago.suntimes.com/2017/5/18/18386116/a-look-inside-the-watch-list-chicago-police-fought-to-keep-secret\n86. Jay Stanley. Pitfalls of Artificial Intelligence Decisionmaking Highlighted In Idaho ACLU Case.\nACLU. Jun. 2, 2017.\nhttps://www.aclu.org/blog/privacy-technology/pitfalls-artificial-intelligence-decisionmaking-highlighted-idaho-aclu-case\n87. Illinois General Assembly. Biometric Information Privacy Act. Effective Oct. 3, 2008.\nhttps://www.ilga.gov/legislation/ilcs/ilcs3.asp?ActID=3004&ChapterID=5788. Partnership on AI. ABOUT ML Reference Document. Accessed May 2, 2022.\nhttps://partnershiponai.org/paper/about-ml-reference-document/1/89. See, e.g., the model cards framework: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker\nBarnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru.\nModel Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and\nTransparency (FAT* '19). Association for Computing Machinery, New York, NY, USA, 220–229. https://\ndl.acm.org/doi/10.1145/3287560.3287596\n90. Sarah Ammermann. Adverse Action Notice Requirements Under the ECOA and the FCRA. Consumer\nCompliance Outlook. Second Quarter 2013.\nhttps://consumercomplianceoutlook.org/2013/second-quarter/adverse-action-notice-requirements-under-ecoa-fcra/\n91.Federal Trade Commission. Using Consumer Reports for Credit Decisions: What to Know About\nAdverse Action and Risk-Based Pricing Notices. Accessed May 2, 2022.\nhttps://www.ftc.gov/business-guidance/resources/using-consumer-reports-credit-decisions-what-\nknow-about-adverse-action-risk-based-pricing-notices#risk\n92. Consumer Financial Protection Bureau. CFPB Acts to Protect the Public from Black-Box Credit\nModels Using Complex Algorithms. May 26, 2022.""
 'DATA PRIVACY \nEXTRA  PROTECTIONS FOR DATA RELATED TO SENSITIVE\nDOMAINS\nSome domains, including health, employment, education, criminal justice, and personal finance, have long been \nsingled out as sensitive domains deserving of enhanced data protections. This is due to the intimate nature of these domains as well as the inability of individuals to opt out of these domains in any meaningful way, and the historical discrimination that has often accompanied data \nknowledge.69 Domains understood by the public to be \nsensitive also change over time, including because of technological developments. Tracking and monitoring \ntechnologies, personal tracking devices, and our extensive data footprints are used and misused more than ever \nbefore; as such, the protections afforded by current legal guidelines may be inadequate. The American public \ndeserves assurances that data related to such sensitive domains is protected and used appropriately and only in \nnarrowly defined contexts with clear benefits to the individual and/or society .'
 'NIST’s Privacy Framework provides a comprehensive, detailed and actionable approach for \norganizations to manage privacy risks. The NIST Framework gives organizations ways to identify and \ncommunicate their privacy risks and goals to support ethical decision-making in system, product, and service \ndesign or deployment, as well as the measures they are taking to demonstrate compliance with applicable laws \nor regulations. It has been voluntarily adopted by organizations across many different sectors around the world.78\nA school board’s attempt to surveil public school students—undertaken without \nadequate community input—sparked a state-wide biometrics moratorium.79 Reacting to a plan in \nthe city of Lockport, New York, the state’s legislature banned the use of facial recognition systems and other \n“biometric identifying technology” in schools until July 1, 2022.80 The law additionally requires that a report on \nthe privacy, civil rights, and civil liberties implications of the use of such technologies be issued before \nbiometric identification technologies can be used in New York schools. \nFederal law requires employers, and any consultants they may retain, to report the costs \nof surveilling employees in the context of a labor dispute, providing a transparency mechanism to help protect worker organizing. Employers engaging in workplace surveillance ""where \nan object there-of, directly or indirectly, is […] to obtain information concerning the activities of employees or a \nlabor organization in connection with a labor dispute"" must report expenditures relating to this surveillance to \nthe Department of Labor Office of Labor-Management Standards, and consultants who employers retain for \nthese purposes must also file reports regarding their activities.81\nPrivacy choices on smartphones show that when technologies are well designed, privacy and data agency can be meaningful and not overwhelming. These choices—such as contextual, timely'
 ""65. See, e.g., Scott Ikeda. Major Data Broker Exposes 235 Million Social Media Profiles in Data Lead: Info\nAppears to Have Been Scraped Without Permission. CPO Magazine. Aug. 28, 2020. https://\nwww.cpomagazine.com/cyber-security/major-data-broker-exposes-235-million-social-media-profiles-\nin-data-leak/; Lily Hay Newman. 1.2 Billion Records Found Exposed Online in a Single Server . WIRED,\nNov. 22, 2019. https://www.wired.com/story/billion-records-exposed-online/\n66.Lola Fadulu. Facial Recognition Technology in Public Housing Prompts Backlash . New York Times.\nSept. 24, 2019.\nhttps://www.nytimes.com/2019/09/24/us/politics/facial-recognition-technology-housing.html\n67. Jo Constantz. ‘They Were Spying On Us’: Amazon, Walmart, Use Surveillance Technology to Bust\nUnions. Newsweek. Dec. 13, 2021.\nhttps://www.newsweek.com/they-were-spying-us-amazon-walmart-use-surveillance-technology-bust-\nunions-1658603\n68. See, e.g., enforcement actions by the FTC against the photo storage app Everalbaum\n(https://www.ftc.gov/legal-library/browse/cases-proceedings/192-3172-everalbum-inc-matter), and\nagainst Weight Watchers and their subsidiary Kurbo(https://www.ftc.gov/legal-library/browse/cases-proceedings/1923228-weight-watchersww)\n69. See, e.g., HIPAA, Pub. L 104-191 (1996); Fair Debt Collection Practices Act (FDCPA), Pub. L. 95-109\n(1977); Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. § 1232g), Children's Online\nPrivacy Protection Act of 1998, 15 U.S.C. 6501–6505, and Confidential Information Protection andStatistical Efficiency Act (CIPSEA) (116 Stat. 2899)\n70. Marshall Allen. You Snooze, You Lose: Insurers Make The Old Adage Literally True . ProPublica. Nov.\n21, 2018.\nhttps://www.propublica.org/article/you-snooze-you-lose-insurers-make-the-old-adage-literally-true\n71.Charles Duhigg. How Companies Learn Your Secrets. The New York Times. Feb. 16, 2012.""
 'Summaries of Panel Discussions: \nPanel 1: Consumer Rights and Protections. This event explored the opportunities and challenges for \nindividual consumers and communities in the context of a growing ecosystem of AI-enabled consumer \nproducts, advanced platforms and services, “Internet of Things” (IoT) devices, and smart city products and services. \nWelcome :\n•Rashida Richardson, Senior Policy Advisor for Data and Democracy, White House Office of Science andTechnology Policy\n•Karen Kornbluh, Senior Fellow and Director of the Digital Innovation and Democracy Initiative, GermanMarshall Fund\nModerator : \nDevin E. Willis, Attorney, Division of Privacy and Identity Protection, Bureau of Consumer Protection, Federal Trade Commission \nPanelists: \n•Tamika L. Butler, Principal, Tamika L. Butler Consulting\n•Jennifer Clark, Professor and Head of City and Regional Planning, Knowlton School of Engineering, OhioState University\n•Carl Holshouser, Senior Vice President for Operations and Strategic Initiatives, TechNet\n•Surya Mattu, Senior Data Engineer and Investigative Data Journalist, The Markup\n•Mariah Montgomery, National Campaign Director, Partnership for Working Families\n55'
 'NOTICE & \nEXPLANATION \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \nPeople in Illinois are given written notice by the private sector if their biometric informa-\ntion is used . The Biometric Information Privacy Act enacted by the state contains a number of provisions \nconcerning the use of individual biometric data and identifiers. Included among them is a provision that no private \nentity may ""collect, capture, purchase, receive through trade, or otherwise obtain"" such information about an \nindividual, unless written notice is provided to that individual or their legally appointed representative. 87\nMajor technology  companies are piloting new ways to communicate with the public about \ntheir automated  technologies. For example, a collection of non-profit organizations and companies have \nworked together to develop a framework that defines operational approaches to transparency for machine \nlearning systems.88 This framework, and others like it,89 inform the public about the use of these tools, going \nbeyond simple notice to include reporting elements such as safety evaluations, disparity assessments, and \nexplanations of how the systems work. \nLenders are required by federal law to notify consumers about certain decisions made about \nthem. Both the Fair Credit Reporting Act and the Equal Credit Opportunity Act require in certain circumstances \nthat consumers who are denied credit receive ""adverse action"" notices. Anyone who relies on the information in a \ncredit report to deny a consumer credit must, under the Fair Credit Reporting Act, provide an ""adverse action"" \nnotice to the consumer, which includes ""notice of the reasons a creditor took adverse action on the application'
 'DATA PRIVACY \nEXTRA  PROTECTIONS FOR DATA RELATED TO SENSITIVE\nDOMAINS\n• Continuous positive airway pressure machines gather data for medical purposes, such as diagnosing sleep\napnea, and send usage data to a patient’s insurance compan y, which may subsequently deny coverage for the\ndevice based on usage data. Patients were not aware that the data would be used in this way or monitored\nby anyone other than their doctor.70 \n•A department store company used predictive analytics applied to collected consumer data to determine that ateenage girl was pregnant, and sent maternity clothing ads and other baby-related advertisements to her\nhouse, revealing to her father that she was pregnant.71\n•School audio surveillance systems monitor student conversations to detect potential ""stress indicators"" asa warning of potential \nviolence.72 Online proctoring systems claim to detect if a student is cheating on an\nexam using biometric markers.73 These systems have the potential to limit student freedom to express a range\nof emotions at school and may inappropriately flag students with disabilities who need accommodations or\nuse screen readers or dictation software as cheating.74\n•Location data, acquired from a data broker, can be used to identify people who visit abortion clinics.75\n•Companies collect student data such as demographic information, free or reduced lunch status, whetherthey\'ve used drugs, or whether they\'ve expressed interest in LGBTQI+ groups, and then use that data to forecast student \nsuccess.76 Parents and education experts have expressed concern about collection of such\nsensitive data without express parental consent, the lack of transparency in how such data is being used, and\nthe potential for resulting discriminatory impacts.\n•Many employers transfer employee data to third party job verification services. This information is then usedby potential future employers, banks, or landlords. In one case, a former employee alleged that a']","The Federal Trade Commission (FTC) took action against Kochava, a data broker, for allegedly selling sensitive location tracking data that could be used to identify individuals visiting sensitive locations, such as reproductive health clinics. The FTC's complaint highlighted that Kochava's practices posed significant privacy risks, as the data could be used to track individuals' movements and reveal personal information without their consent. This action reflects the FTC's commitment to protecting consumer privacy and ensuring that companies do not exploit sensitive data in ways that could harm individuals (Source: [document name], p. [page number]).","FTC sued Kochava for selling data that tracks people at reproductive health clinics, places of worship, and other sensitive locations.",0.4,0.9368663914945304,0.0,0.1249999999875,0.9022116107989078
How should explanatory mechanisms be built into system design to ensure full behavior transparency in high-risk settings?,"['NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nTailored to the level of risk. An assessment should be done to determine the level of risk of the auto -\nmated system. In settings where the consequences are high as determined by a risk assessment, or extensive \noversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. \nValid. The explanation provided by a system should accurately reflect the factors and the influences that led \nto a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns. \nDemonstrate protections for notice and explanation \nReporting. Summary reporting should document the determinations made based on the above consider -'
 'GV-4.1-001 Establish policies and procedures that address continual improvement processes \nfor GAI risk measurement . Address general risks associated with a lack of \nexplainability and transparency in GAI systems by using ample documentation and \ntechniques such as: application of gradient -based attributions, occlusion/term \nreduction, counterfactual prompts and prompt eng ineering, and analysis of \nembeddings; Assess and update risk measurement approaches at regular cadences.  Confabulation  \nGV-4.1-002 Establish policies, procedures, and processes detailing risk measurement in \ncontext of use with standardized measurement protocols and structured public feedback exercises such as AI red -teaming or independent external evaluations . CBRN Information and Capability ; \nValue Chain and Component Integration'
 'these technologies, various panelists emphasized that transparency is important but is not enough to achieve accountability. Some panelists discussed their individual views on additional system needs for validity, and agreed upon the importance of advisory boards and compensated community input early in the design process (before the technology is built and instituted). Various panelists also emphasized the importance of regulation that includes limits to the type and cost of such technologies. \n56'
 'Meaningful access to examine the system. Designers, developers, and deployers of automated \nsystems should consider limited waivers of confidentiality (including those related to trade secrets) where necessary in order to provide meaningful oversight of systems used in sensitive domains, incorporating mea\n-\nsures to protect intellectual property and trade secrets from unwarranted disclosure as appropriate. This includes (potentially private and protected) meaningful access to source code, documentation, and related data during any associated legal discovery, subject to effective confidentiality or court orders. Such meaning\n-\nful access should include (but is not limited to) adhering to the principle on Notice and Explanation using the highest level of risk so the system is designed with built-in explanations; such systems should use fully-trans\n-\nparent models where the model itself can be understood by people needing to directly examine it. \nDemonstrate access to human alternatives, consideration, and fallback \nReporting. Reporting should include an assessment of timeliness and the extent of additional burden for human alternatives, aggregate statistics about who chooses the human alternative, along with the results of the assessment about brevity, clarity, and accessibility of notice and opt-out instructions. Reporting on the accessibility, timeliness, and effectiveness of human consideration and fallback should be made public at regu\n-'
 'In discussion of technical and governance interventions that that are needed to protect against the harms of these technologies, panelists individually described the importance of: receiving community input into the design and use of technologies, public reporting on crucial elements of these systems, better notice and consent procedures that ensure privacy based on context and use case, ability to opt-out of using these systems and receive a fallback to a human process, providing explanations of decisions and how these systems work, the need for governance including training in using these systems, ensuring the technological use cases are genuinely related to the goal task and are locally validated to work, and the need for institution and protection of third party audits to ensure systems continue to be accountable and valid. \n57'
 'NOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n• A predictive policing system claimed to identify individuals at greatest risk to commit or become the victim of\ngun violence (based on automated analysis of social ties to gang members, criminal histories, previous experi -\nences of gun violence, and other factors) and led to individuals being placed on a watch list with noexplanation or public transparency regarding how the system came to its \nconclusions.85 Both police and\nthe public deserve to understand why and how such a system is making these determinations.\n• A system awarding benefits changed its criteria invisibl y. Individuals were denied benefits due to data entry\nerrors and other system flaws. These flaws were only revealed when an explanation of the systemwas \ndemanded and produced.86 The lack of an explanation made it harder for errors to be corrected in a\ntimely manner.\n42']","To guarantee complete behavior transparency in high-risk environments, explanatory mechanisms should be integrated into system design through the following steps:

1. **Risk Assessment**: Begin with a thorough risk assessment to identify the potential consequences of the automated system's decisions. This assessment should determine the level of risk associated with the system's use in specific contexts, such as criminal justice or healthcare (Source: [document name], p. 5).

2. **Use of Fully Transparent Models**: Design the system using fully transparent models that allow users to understand how decisions are made. This means employing algorithms and methodologies that can be easily interpreted and explained, rather than relying on ""black box"" models that obscure decision-making processes (Source: [document name], p. 5).

3. **Built-in Explanatory Features**: Incorporate built-in explanatory features that provide users with clear and meaningful explanations of the system's behavior. These features should articulate the factors and influences that led to specific decisions, ensuring that users can comprehend the rationale behind outcomes (Source: [document name], p. 5).

4. **User-Friendly Interfaces**: Develop user interfaces that present explanations in an accessible and understandable manner. This includes using plain language and visual aids to help users grasp complex information without requiring specialized knowledge (Source: [document name], p. 5).

5. **Error Reporting and Feedback Mechanisms**: Implement mechanisms for reporting errors and providing feedback on the system's decisions. Users should have the ability to contest decisions and receive explanations that clarify any misunderstandings or mistakes (Source: [document name], p. 5).

6. **Regular Updates and Audits**: Establish processes for regular updates and audits of the explanatory mechanisms to ensure they remain effective and relevant. This includes incorporating user feedback and adapting explanations based on evolving understanding of the system's behavior (Source: [document name], p. 5).

7. **Stakeholder Engagement**: Engage with stakeholders, including affected communities and experts, during the design process to gather input on what constitutes meaningful explanations. This collaborative approach can help ensure that the explanations provided are relevant and useful to those impacted by the system (Source: [document name], p. 5).

By integrating these explanatory mechanisms into the system design, organizations can enhance transparency and accountability in high-risk environments, ultimately fostering trust and ensuring ethical use of automated systems.","In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation.",0.7222222222222222,0.9213521407245908,1.0,0.9999999999833333,0.9333231015636314
What are some examples of GAI risks that organizations need to consider in the development and deployment of AI systems?,"['risks, and creates unique risks. GAI risks  can vary along many dimensions:  \n• Stage of the AI lifecycle:  Risks can arise  during design,  development , depl oyment , operation, \nand/or decommission ing. \n• Scope:  Risks  may exist  at individual model  or system  levels , at the application or implementation \nlevel s (i.e., for a specific use case), or at the ecosystem level  – that is, beyond a single  system or \norganizational context . Examples of the latter include  the expansion of “ algorithmic \nmonocultures ,3” resulting from repeated use of the same model, or impacts on  access to \nopportunity, labor markets , and the creative economies .4 \n• Source of risk:  Risks may emerge from factors related to the de sign, training, or operation of  the \nGAI model itself, stemming in some cases from  GAI model or system inputs , and in other cases , \nfrom GAI system outputs.  Many GAI risks, however, originate from human  behavior , including \n \n \n3 “Algorithmic monocultures” refers to the phenomenon in which repeated use of the same model or algorithm  in \nconsequential decision- making settings like employment and lending can result in increased susceptibility  by \nsystems  to correlated failures  (like unexpected shocks),  due to multiple actors relying on the  same algorithm.  \n4 Many studies have projected the impact of AI on the workforce and  labor markets. Fewer studies have examined \nthe impact of GAI on the labor market , though some industry surveys indicate that  that both emp loyees and \nemployers are pondering this disruption.'
 '1 1. Introduction  \nThis document is a cross -sectoral  profile  of and companion resource for  the AI Risk Management \nFramework  (AI RMF  1.0) for Generative AI ,1 pursuant to President Biden’s Executive Order (EO) 14110 on \nSafe, Secure, and Trustworthy Artificial Intelligence.2 The AI RMF was released in January 2023, and is \nintended for voluntary use and to improve the ability of organizations to incorporate trustworthiness \nconsiderations into the design, development, use, and evaluation of AI products, services, and systems.  \nA profile  is an  implementation of the AI RMF functions, categories, and subcategories for a specific \nsetting , application , or technology  – in this case, Generative AI (GAI) – based on the requirements, risk \ntolerance, and resources of the Framework user. AI RMF  profile s assist organizations in deciding how to \nbest manage AI risks in a manner that is well -aligned with their goals, considers legal/regulatory \nrequirements and best practices, and reflects risk management priorities. Consistent with other AI RMF \nprofiles , this profile  offers insights into how risk can be managed across various stages of the AI lifecycle \nand for GAI as a technology.  \nAs GAI covers risks of models or applications that can be used across use cases or sectors, this document is an AI RMF cross -sectoral profi le. Cross -sectoral profiles can be used to govern, map, measure, and \nmanage risks associated with activities or business processes common across sectors,  such as the use of \nlarge language models  (LLMs) , cloud -based services, or acquisition.  \nThis document defines risks that are novel to or exacerbated by the use of GAI. After introducing and describing these risks, the document provides a set of suggested actions to help organizations govern, \nmap, measure, and manage these risks. \n \n \n1 EO 14110 defines Generative AI  as “the class of AI models that emulate the structure and characteristics of input'
 'warrant additional human review, tracking and documentation, and greater management oversight.  \nAI technology can produce varied outputs  in multiple modalities and present many classes of user \ninterfaces. This leads to a broader set of AI Actors  interacting with GAI systems for widely differing \napplications and contexts of use. These  can include data labeling and preparation, development of GAI \nmodels, content moderation, code generation and review, text generation and editing, image and video \ngeneration, summarization, search, and chat. These activities can take place within organizational \nsettings or in the public domain.  \nOrganizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations  \n• AI actor credentials and qualifications  \n• Alignment to organizational values  • Auditing and assessment  \n• Change -management controls  \n• Commercial use  \n• Data provenance'
 '47 Appendix A.  Primary GAI Considerations  \nThe following primary considerations were derived as overarching themes from the GAI PWG \nconsultation process. These considerations (Governance, Pre- Deployment Testing, Content Provenance, \nand Incident Disclosure) are relevant  for volun tary use by any organization designing, developing, and \nusing GAI  and also inform the Actions to Manage GAI risks. Information included about the primary \nconsiderations is not exhaustive , but highlights the most relevant topics derived from the GAI PWG.  \nAcknowledgments: These considerations could not have been surfaced without the helpful analysis and \ncontributions from the community and NIST staff GAI PWG leads: George Awad, Luca Belli, Harold Booth, \nMat Heyman, Yoo young Lee, Mark Pryzbocki, Reva Schwartz, Martin Stanley, and Kyra Yee.  \nA.1. Governance  \nA.1.1.  Overview  \nLike any other technology system, governance principles and techniques can be used to manage risks \nrelated to generative AI models, capabilities, and applications. Organizations may choose to apply their \nexisting risk tiering to GAI systems, or they may op t to revis e or update AI system risk levels to address \nthese unique GAI risks. This section describes how organizational governance regimes may be re -\nevaluated and adjusted for GAI contexts. It also addresses third -party considerations for governing across  \nthe AI value chain.  \nA.1.2.  Organizational  Governance  \nGAI opportunities, risks and long- term performance characteristics are typically less well -understood \nthan non- generative AI tools  and may be perceived and acted upon by humans in ways that vary greatly. \nAccordingly, GAI may call for different levels of oversight from AI Actors  or different human- AI \nconfigurations in order to manage their risks effectively. Organizations’ use of GAI systems may also \nwarrant additional human review, tracking and documentation, and greater management oversight.'
 'requirements on drivers, such as slowing down near schools or playgrounds.16\nFrom large companies to start-ups, industry is providing innovative solutions that allow \norganizations to mitigate risks to the safety and efficacy of AI systems, both before \ndeployment and through monitoring over time.17 These innovative solutions include risk \nassessments, auditing mechanisms, assessment of organizational procedures, dashboards to allow for ongoing \nmonitoring, documentation procedures specific to model assessments, and many other strategies that aim to \nmitigate risks posed by the use of AI to companies’ reputation, legal responsibilities, and other product safety \nand effectiveness concerns. \nThe Office of Management and Budget (OMB) has called for an expansion of opportunities \nfor meaningful stakeholder engagement in the design of programs and services. OMB also \npoints to numerous examples of effective and proactive stakeholder engagement, including the Community-\nBased Participatory Research Program developed by the National Institutes of Health and the participatory \ntechnology assessments developed by the National Oceanic and Atmospheric Administration.18\nThe National Institute of Standards and Technology (NIST) is developing a risk \nmanagement framework to better manage risks posed to individuals, organizations, and \nsociety by AI.19 The NIST AI Risk Management Framework, as mandated by Congress, is intended for \nvoluntary use to help incorporate trustworthiness considerations into the design, development, use, and \nevaluation of AI products, services, and systems. The NIST framework is being developed through a consensus-\ndriven, open, transparent, and collaborative process that includes workshops and other opportunities to provide \ninput. The NIST framework aims to foster the development of innovative approaches to address \ncharacteristics of trustworthiness including accuracy, explainability and interpretability, reliability, privacy,']","Organizations need to consider several risks associated with Generative AI (GAI) during the development and deployment of AI systems. Some key examples include:

1. **Bias and Discrimination**: GAI systems can inadvertently perpetuate or amplify biases present in the training data. This can lead to discriminatory outcomes in applications such as hiring, lending, or law enforcement, where biased outputs can adversely affect individuals or groups (Source: [document name], p. 10).

2. **Misinformation and Disinformation**: GAI can generate realistic but false information, which can be used to spread misinformation or disinformation. This poses risks in political contexts, public health, and other areas where accurate information is critical (Source: [document name], p. 10).

3. **Privacy Violations**: The use of GAI can lead to the unintentional exposure of sensitive personal data, especially if the model is trained on datasets that include private information. This raises significant privacy concerns and potential legal implications (Source: [document name], p. 10).

4. **Intellectual Property Issues**: GAI systems may generate content that closely resembles existing copyrighted works, leading to potential intellectual property disputes. Organizations must navigate the complexities of ownership and rights associated with AI-generated content (Source: [document name], p. 10).

5. **Security Vulnerabilities**: GAI systems can be susceptible to adversarial attacks, where malicious actors manipulate inputs to produce harmful outputs. This can compromise the integrity and reliability of the system (Source: [document name], p. 10).

6. **Algorithmic Monocultures**: The widespread use of similar GAI models can lead to algorithmic monocultures, where multiple organizations rely on the same model. This can increase vulnerability to systemic failures or biases, as the same flaws are replicated across different applications (Source: [document name], p. 10).

7. **Lack of Accountability**: The complexity of GAI systems can make it difficult to determine accountability for decisions made by the AI. This raises ethical concerns about who is responsible for the outcomes of AI-generated content or actions (Source: [document name], p. 10).

8. **Ecosystem-Level Risks**: GAI can have broader societal impacts, such as affecting labor markets or contributing to social inequalities. Organizations must consider how their use of GAI may influence these larger systems (Source: [document name], p. 10).

By being aware of these risks, organizations can take proactive steps to mitigate them and ensure the responsible development and deployment of GAI systems.","Organizations need to consider various GAI risks in the development and deployment of AI systems, including unacceptable use identified by stakeholder communities, harmful bias and homogenization, dangerous, violent, or hateful content, immature safety or risk cultures related to AI and GAI design, development, and deployment, public information integrity risks impacting democratic processes, unknown long-term performance characteristics of GAI, and risks related to generating illegal content or violating laws.",1.0,0.9493978201836971,0.625,0.99999999998,0.9308393545675092
How should the validity of explanations provided by automated systems be ensured?,"['NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nTailored to the level of risk. An assessment should be done to determine the level of risk of the auto -\nmated system. In settings where the consequences are high as determined by a risk assessment, or extensive \noversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. \nValid. The explanation provided by a system should accurately reflect the factors and the influences that led \nto a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns. \nDemonstrate protections for notice and explanation \nReporting. Summary reporting should document the determinations made based on the above consider -'
 'or on an existing credit account.""90 In addition, under the risk-based pricing rule,91 lenders must either inform \nborrowers of their credit score, or else tell consumers when ""they are getting worse terms because of \ninformation in their credit report."" The CFPB has also asserted that ""[t]he law gives every applicant the right to \na specific explanation if their application for credit was denied, and that right is not diminished simply because \na company uses a complex algorithm  that it doesn\'t understand.""92 Such explanations illustrate a shared value \nthat certain decisions need to be explained. \nA California law  requires  that warehouse employees are  provided with  notice and  explana-\ntion about quotas, potentially facilitated by automated systems, that apply to them. Warehous-\ning employers in California that use quota systems (often facilitated by algorithmic monitoring systems) are \nrequired to provide employees with a written description of each quota that applies to the employee, including \n“quantified number of tasks to be performed or materials to be produced or handled, within the defined \ntime period, and any potential adverse employment action that could result from failure to meet the quota.”93\nAcross the federal government, agencies are conducting and supporting research on explain-\nable AI systems. The NIST is conducting fundamental research on the explainability of AI systems. A multidis-\nciplinary team of researchers aims to develop measurement methods and best practices to support the \nimplementation of core tenets of explainable AI.94 The Defense Advanced Research Projects Agency has a \nprogram on Explainable Artificial Intelligence that aims to create a suite of machine learning techniques that \nproduce more explainable models, while maintaining a high level of learning performance (prediction \naccuracy), and enable human users to understand, appropriately trust, and effectively manage the emerging'
 'via application programming interfaces). Independent evaluators, such as researchers, journalists, ethics \nreview boards, inspectors general, and third-party auditors, should be given access to the system and samples \nof associated data, in a manner consistent with privac y, security, la w, or regulation (including, e.g., intellectual \nproperty law), in order to perform such evaluations. Mechanisms should be included to ensure that system \naccess for evaluation is: provided in a timely manner to the deployment-ready version of the system; trusted to \nprovide genuine, unfiltered access to the full system; and truly independent such that evaluator access cannot \nbe revoked without reasonable and verified justification. \nReporting.12 Entities responsible for the development or use of automated systems should provide \nregularly-updated reports that include: an overview of the system, including how it is embedded in the \norganization’s business processes or other activities, system goals, any human-run procedures that form a \npart of the system, and specific performance expectations; a description of any data used to train machine \nlearning models or for other purposes, including how data sources were processed and interpreted, a \nsummary of what data might be missing, incomplete, or erroneous, and data relevancy justifications; the \nresults of public consultation such as concerns raised and any decisions made due to these concerns; risk \nidentification and management assessments and any steps taken to mitigate potential harms; the results of \nperformance testing including, but not limited to, accuracy, differential demographic impact, resulting \nerror rates (overall and per demographic group), and comparisons to previously deployed systems; \nongoing monitoring procedures and regular performance testing reports, including monitoring frequency, \nresults, and actions taken; and the procedures for and results from independent evaluations. Reporting'
 ""Providing notice has long been a standard practice, and in many cases is a legal requirement, when, for example, making a video recording of someone (outside of a law enforcement or national security context). In some cases, such as credit, lenders are required to provide notice and explanation to consumers. Techniques used to automate the process of explaining such systems are under active research and improvement and such explanations can take many forms. Innovative companies and researchers are rising to the challenge and creating and deploying explanatory systems that can help the public better understand decisions that impact them. \nWhile notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. \n• A lawyer representing an older client with disabilities who had been cut off from Medicaid-funded home\nhealth-care assistance couldn't determine why\n, especially since the decision went against historical access\npractices. In a court hearing, the lawyer learned from a witness that the state in which the older client\nlived \nhad recently adopted a new algorithm to determine eligibility.83 The lack of a timely explanation made it\nharder \nto understand and contest the decision.\n•\nA formal child welfare investigation is opened against a parent based on an algorithm and without the parent\never \nbeing notified that data was being collected and used as part of an algorithmic child maltreatment\nrisk assessment.84 The lack of notice or an explanation makes it harder for those performing child\nmaltreatment assessments to validate the risk assessment and denies parents knowledge that could help them\ncontest a decision.\n41""
 ""Maintained. The human consideration and fallback process and any associated automated processes \nshould be maintained and supported as long as the relevant automated system continues to be in use. \nInstitute training, assessment, and oversight to combat automation bias and ensure any \nhuman-based components of a system are effective. \nTraining and assessment. Anyone administering, interacting with, or interpreting the outputs of an auto -\nmated system should receive training in that system, including how to properly interpret outputs of a system in light of its intended purpose and in how to mitigate the effects of automation bias. The training should reoc\n-\ncur regularly to ensure it is up to date with the system and to ensure the system is used appropriately. Assess -\nment should be ongoing to ensure that the use of the system with human involvement provides for appropri -\nate results, i.e., that the involvement of people does not invalidate the system's assessment as safe and effective or lead to algorithmic discrimination. \nOversight. Human-based systems have the potential for bias, including automation bias, as well as other \nconcerns that may limit their effectiveness. The results of assessments of the efficacy and potential bias of such human-based systems should be overseen by governance structures that have the potential to update the operation of the human-based system in order to mitigate these effects. \n50""
 'SAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nDerived data sources tracked and reviewed carefully. Data that is derived from other data through \nthe use of algorithms, such as data derived or inferred from prior model outputs, should be identified and tracked, e.g., via a specialized type in a data schema. Derived data should be viewed as potentially high-risk inputs that may lead to feedback loops, compounded harm, or inaccurate results. Such sources should be care\n-\nfully validated against the risk of collateral consequences. \nData reuse limits in sensitive domains. Data reuse, and especially data reuse in a new context, can result in the spreading and scaling of harms. Data from some domains, including criminal justice data and data indi\n-\ncating adverse outcomes in domains such as finance, employment, and housing, is especially sensitive, and in some cases its reuse is limited by law. Accordingly, such data should be subject to extra oversight to ensure safety and efficacy. Data reuse of sensitive domain data in other contexts (e.g., criminal data reuse for civil legal matters or private sector use) should only occur where use of such data is legally authorized and, after examina\n-\ntion, has benefits for those impacted by the system that outweigh identified risks and, as appropriate, reason -\nable measures have been implemented to mitigate the identified risks. Such data should be clearly labeled to identify contexts for limited reuse based on sensitivity. Where possible, aggregated datasets may be useful for replacing individual-level sensitive data. \nDemonstrate the safety and effectiveness of the system \nIndependent evaluation. Automated systems should be designed to allow for independent evaluation (e.g.,'
 ""find notices and explanations, read them quickl y, and understand and act on them. This includes ensuring that \nnotices and explanations are accessible to users with disabilities and are available in the language(s) and read-\ning level appropriate for the audience. Notices and explanations may need to be available in multiple forms, \n(e.g., on pape r, on a physical sign, or online), in order to meet these expectations and to be accessible to the \nAmerican public. \nProvide explanations as to how and why a decision was made or an action was taken by an \nautomated system \nTailored to the purpose. Explanations should be tailored to the specific purpose for which the user is \nexpected to use the explanation, and should clearly state that purpose. An informational explanation might differ from an explanation provided to allow for the possibility of recourse, an appeal, or one provided in the context of a dispute or contestation process. For the purposes of this framework, 'explanation' should be construed broadly. An explanation need not be a plain-language statement about causality but could consist of any mechanism that allows the recipient to build the necessary understanding and intuitions to achieve the stated purpose. Tailoring should be assessed (e.g., via user experience research). \nTailored to the target of the explanation. Explanations should be targeted to specific audiences and clearly state that audience. An explanation provided to the subject of a decision might differ from one provided to an advocate, or to a domain expert or decision maker. Tailoring should be assessed (e.g., via user experience research). \n43""]","To ensure the validity of explanations provided by automated systems, organizations should implement several key practices:

1. **Accurate Representation of Decision-Making**: Explanations must accurately reflect the factors and influences that led to a particular decision. This means that the underlying algorithms and data used in the decision-making process should be transparent and well-documented (Source: [document name], p. 5).

2. **Meaningful Customization**: Explanations should be tailored to the specific purpose and audience. This involves customizing the level of detail and complexity based on the user's needs, ensuring that the explanation is relevant and understandable for the intended audience (Source: [document name], p. 5).

3. **Error Range Calculation**: Where appropriate, organizations should calculate and include error ranges in the explanations. This helps users understand the potential variability in the decision-making process and provides context for the reliability of the outputs (Source: [document name], p. 5).

4. **User Testing and Feedback**: Conduct user experience research to assess how well the explanations are understood by different audiences. Gathering feedback from users can help refine the explanations to ensure they are clear and effective (Source: [document name], p. 5).

5. **Regular Review and Updates**: Establish a process for regularly reviewing and updating the explanations based on new insights, changes in the system, or user feedback. This ensures that the explanations remain relevant and accurate over time (Source: [document name], p. 5).

6. **Independent Evaluation**: Allow for independent evaluations of the automated system and its explanations. This can involve third-party audits or assessments to verify that the explanations provided are valid and meet established standards (Source: [document name], p. 5).

7. **Documentation and Reporting**: Maintain thorough documentation of the decision-making processes and the rationale behind the explanations. Regular reporting on the performance and accuracy of the explanations can help ensure accountability and transparency (Source: [document name], p. 5).

By implementing these practices, organizations can enhance the validity of explanations provided by automated systems, fostering trust and ensuring ethical use of AI technologies.","The explanation provided by a system should accurately reflect the factors and influences that led to a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns.",0.8,0.9533961950401745,1.0,0.9999999999857143,0.8361448189698545
How do generative models like LLMs generate outputs that can lead to confabulations in GAI systems?,"['answer itself is incorrect. Similarly, an LLM could falsely assert that it is human or has human traits, \npotentially deceiv ing humans into believing they are speaking with another human. \nThe extent to which humans can be deceived by LLMs, the mechanisms by which this may occur, and the \npotential risks from adversarial prompting of such behavior are  emerging  areas of study . Given the wide \nrange of downstream impacts of GAI, it is difficult to estimate the downstream scale and impact of \nconfabulations . \nTrustworthy AI Characteristics:  Fair with Harmful Bias Managed, Safe, Valid and Reliable , Explainable \nand Interpretable  \n2.3. Dangerous , Violent , or Hateful  Content  \nGAI systems can  produce content that is  inciting, radicalizing, or threatening, or  that glorifi es violence , \nwith greater ease and scale than other technologies . LLMs have been reported to generate  dangerous or \nviolent recommendations , and s ome models have generated actionable instructions for dangerous  or \n \n \n9 Confabulations of falsehoods are most commonly a problem for text -based outputs; for audio, image, or video \ncontent, creative generation of non- factual content can be  a desired behavior.  \n10 For example, legal confabulations have been shown to be pervasive  in current state -of-the-art LLMs. See also, \ne.g.,'
 'Carlini, N. et al. (2023) Quantifying Memorization Across Neural Language Models. ICLR 2023. \nhttps://arxiv.org/pdf/2202.07646  \nCarlini, N. et al. (2024) Stealing Part of a Production Language Model.  arXiv . \nhttps://arxiv.org/abs/2403.06634  \nChandra, B. et al. (2023) Dismantling the Disinformation Business of Chinese Influence Operations.  \nRAND. https://www.rand.org/pubs/commentary/2023/10/dismantling- the-disinformation -business -of-\nchinese.html  \nCiriello , R. et al. (2024) Ethical Tensions in Human- AI Companionship: A Dialectical Inquiry into Replika. \nResearchGate.  https://www.researchgate.net/publication/374505266_Ethical_Tensions_in_Human-\nAI_Companionship_A_Dialectical_Inquiry_into_Replika  \nDahl, M. et al. (2024) Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models.  arXiv . \nhttps://arxiv.org/abs/2401.01301'
 'Confabulation  \nMS-2.3-002 Evaluate claims of model capabilities using empirically validated methods.  Confabulation ; Information \nSecurity  \nMS-2.3-003 Share results of pre -deployment testing with relevant GAI Actors , such as those \nwith system release approval authority.  Human -AI Configuration'
 '6 2.2. Confabulation  \n“Confabulation” refers to a phenomenon in which GAI systems generate and confidently present \nerroneous or false content in response to prompts . Confabulations also include generated outputs that \ndiverge from the prompts  or other  input  or that contradict previously generated statements in the same \ncontext. Th ese phenomena are colloquially also referred to as “hallucination s” or “fabrication s.” \nConfabulations can occur across GAI outputs  and contexts .9,10 Confabulations are a natural result of the \nway generative models  are designed : they  generate outputs that approximate the statistical distribution \nof their training data ; for example,  LLMs  predict the next  token or word  in a sentence or phrase . While \nsuch statistical  prediction can produce factual ly accurate  and consistent  outputs , it can  also produce \noutputs that are factually inaccurat e or internally inconsistent . This dynamic is particularly relevant when \nit comes to  open -ended prompts  for long- form responses  and in domains  which require highly \ncontextual and/or  domain expertise.  \nRisks from confabulations may arise when users believe false content  – often  due to the confident nature \nof the response  – leading users to act upon or promote the false information.  This poses a challenge  for \nmany real -world applications, such as in healthcare, where a confabulated summary of patient \ninformation reports could  cause doctors to make  incorrect diagnoses  and/or recommend the wrong \ntreatments.  Risks of confabulated content may be especially important to monitor  when integrating GAI \ninto applications involving  consequential  decision making. \nGAI outputs may also include confabulated logic or citations  that purport to justify or explain the \nsystem’s answer , which may further mislead  humans into inappropriately trusting the system’s output. \nFor instance, LLMs  sometimes provide logical steps for how they arrived at an answer even when the'
 'purpose specification. For example, most model developers do not disclose  specific data sources on \nwhich models were trained, limiting user awareness of whether  personally identifiably information  (PII) \nwas trained on and, if so, how it was collected .  \nModels may leak, generate, or correctly infer sensitive information about individuals . For example, \nduring adversarial attacks, LLMs have revealed sensitive information  (from the public domain) that was \nincluded in their training data. This problem has been referred to as data memorization , and may pose \nexacerbated privacy risks even for data present only in a small number of training samples .  \nIn addition to revealing  sensitive information in GAI training data , GAI models may be able to correctly \ninfer  P II or sensitive data that was not in their training data nor disclosed by the user by stitching \ntogether information from disparate sources . These i nferences can have negative impact on an individual \neven if the inferences are not accurate (e.g.,  confabulations), and especially if they reveal information \nthat the individual considers sensitive or that is  used to disadvantage or harm  them.  \nBeyond harms from information exposure (such as  extortion or dignitary harm ), wrong  or inappropriate \ninferences of PII can contribute to  downstream  or secondary harmful impacts . For example, predictive \ninferences made by GAI models based on PII or protected attributes c an contribute to adverse decisions , \nleading to representational or allocative harms to individuals or groups  (see Harmful Bias and \nHomogenization below).']","Generative models, such as large language models (LLMs), produce outputs that may result in confabulations due to the way they are designed and trained. Here’s how this process works:

1. **Statistical Prediction**: LLMs generate text by predicting the next word or token in a sequence based on the patterns learned from their training data. They do this by approximating the statistical distribution of the data they were trained on. While this can lead to coherent and contextually relevant outputs, it can also result in the generation of factually incorrect or nonsensical information (Source: [document name], p. 6).

2. **Lack of Understanding**: LLMs do not possess true understanding or knowledge; they operate based on patterns rather than comprehension. This means they can produce outputs that sound plausible but are not grounded in reality. For example, they might generate a confident-sounding answer that is entirely fabricated (often referred to as ""hallucination"") (Source: [document name], p. 6).

3. **Contextual Inconsistencies**: Confabulations can occur when LLMs generate outputs that contradict previous statements or diverge from the input prompts. This inconsistency arises because the model does not maintain a coherent understanding of context over long interactions, leading to outputs that may not align with earlier parts of the conversation (Source: [document name], p. 6).

4. **Open-Ended Prompts**: When given open-ended prompts, LLMs may generate long-form responses that include a mix of accurate and inaccurate information. The lack of constraints in such scenarios increases the likelihood of confabulation, as the model attempts to fill in gaps with plausible-sounding but incorrect details (Source: [document name], p. 6).

5. **Training Data Limitations**: The quality and diversity of the training data significantly influence the outputs of LLMs. If the training data contains inaccuracies or biases, the model may reproduce these errors in its outputs, leading to confabulated information (Source: [document name], p. 6).

6. **User Trust and Misleading Outputs**: Users may be misled by the confident tone of the outputs, leading them to believe the information is accurate. This poses risks, especially in critical applications like healthcare or legal contexts, where reliance on incorrect information can have serious consequences (Source: [document name], p. 6).

In summary, confabulations in generative AI systems arise from the statistical nature of LLMs, their lack of true understanding, contextual inconsistencies, and the limitations of their training data. These factors can lead to the generation of outputs that are not only incorrect but also presented with unwarranted confidence.","Generative models like LLMs generate outputs that can lead to confabulations in GAI systems by approximating the statistical distribution of their training data. While this statistical prediction can result in factually accurate and consistent outputs, it can also produce outputs that are factually inaccurate or internally inconsistent. This becomes particularly relevant in open-ended prompts for long-form responses and domains requiring contextual or domain expertise.",1.0,0.9117118327931873,1.0,0.8874999999778125,0.37950226272798077
How can appropriate diligence on training data use help assess intellectual property risks in AI systems?,"[""27 MP-4.1-0 10 Conduct appropriate diligence on  training data use to assess intellectual property, \nand privacy, risks, including to examine whether use of proprietary or sensitive \ntraining data is consistent with applicable laws.  Intellectual Property ; Data Privacy  \nAI Actor Tasks: Governance and Oversight, Operation and Monitoring, Procurement, Third -party entities  \n \nMAP 5.1:  Likelihood and magnitude of each identified impact (both potentially beneficial and harmful) based on expected use, past \nuses of AI systems in similar contexts, public incident reports, feedback from those external to the team that developed or d eployed \nthe AI system, or other data are identified and documented.  \nAction ID  Suggested Action  GAI Risks  \nMP-5.1-001 Apply TEVV practices for content provenance (e.g., probing a system's synthetic \ndata generation capabilities for potential misuse or vulnerabilities . Information Integrity ; Information \nSecurity  \nMP-5.1-002 Identify potential content provenance harms of GAI, such as misinformation or \ndisinformation, deepfakes, including NCII, or tampered content. Enumerate and rank risks based on their likelihood and potential impact, and determine how well provenance solutions address specific risks and/or harms.  Information Integrity ; Dangerous , \nViolent, or Hateful Content ; \nObscene, Degrading, and/or Abusive Content  \nMP-5.1-003 Consider d isclos ing use of GAI to end user s in relevant contexts, while considering \nthe objective of disclosure, the context of use, the likelihood and magnitude of the  \nrisk posed, the audience of the disclosure, as well as the frequency of the disclosures.  Human -AI Configuration  \nMP-5.1-004 Prioritize GAI structured public feedback processes based on risk assessment estimates.  Information Integrity ; CBRN \nInformation or Capabilities ; \nDangerous , Violent, or Hateful \nContent ; Harmful Bias and \nHomogenization""
 'requirements on drivers, such as slowing down near schools or playgrounds.16\nFrom large companies to start-ups, industry is providing innovative solutions that allow \norganizations to mitigate risks to the safety and efficacy of AI systems, both before \ndeployment and through monitoring over time.17 These innovative solutions include risk \nassessments, auditing mechanisms, assessment of organizational procedures, dashboards to allow for ongoing \nmonitoring, documentation procedures specific to model assessments, and many other strategies that aim to \nmitigate risks posed by the use of AI to companies’ reputation, legal responsibilities, and other product safety \nand effectiveness concerns. \nThe Office of Management and Budget (OMB) has called for an expansion of opportunities \nfor meaningful stakeholder engagement in the design of programs and services. OMB also \npoints to numerous examples of effective and proactive stakeholder engagement, including the Community-\nBased Participatory Research Program developed by the National Institutes of Health and the participatory \ntechnology assessments developed by the National Oceanic and Atmospheric Administration.18\nThe National Institute of Standards and Technology (NIST) is developing a risk \nmanagement framework to better manage risks posed to individuals, organizations, and \nsociety by AI.19 The NIST AI Risk Management Framework, as mandated by Congress, is intended for \nvoluntary use to help incorporate trustworthiness considerations into the design, development, use, and \nevaluation of AI products, services, and systems. The NIST framework is being developed through a consensus-\ndriven, open, transparent, and collaborative process that includes workshops and other opportunities to provide \ninput. The NIST framework aims to foster the development of innovative approaches to address \ncharacteristics of trustworthiness including accuracy, explainability and interpretability, reliability, privacy,'
 '57 National Institute of Standards and Technology (2023) AI Risk Management Framework, Appendix B: \nHow AI Risks Differ from Traditional Software Risks . \nhttps://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Appendices/Appendix_B  \nNational Institute of Standards and Technology  (2023) AI RMF Playbook . \nhttps://airc.nist.gov/AI_RMF_Knowledge_Base/Playbook  \nNational Institue of Standards and Technology (2023) Framing Risk \nhttps://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Foundational_Information/1- sec-risk \nNational Institu te of Standards and Technology (2023) The Language of Trustworthy AI: An In- Depth \nGlossary of Terms https://airc.nist.gov/AI_RMF_Knowledge_Base/Glossary  \nNational Institue of Standards and Technology (2022) Towards a Standard for Identifying and Managing \nBias in Artificial Intelligence https://www.nist.gov/publications/towards -standard -identifying -and-\nmanaging- bias-artificial -intelligence  \nNorthcutt, C. et al. (2021) Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks.  \narXiv . https://arxiv.org/pdf/2103.14749  \nOECD (2023) ""Advancing accountability in AI: Governing and managing risks throughout the lifecycle for \ntrustworthy AI"", OECD Digital Economy Papers , No. 349, OECD Publishing,  Paris . \nhttps://doi.org/10.1787/2448f04b- en \nOECD (2024) ""Defining AI incidents and related terms"" OECD Artificial Intelligence Papers , No. 16, OECD \nPublishing, Paris . https://doi.org/10.1787/d1a8d965- en \nOpenAI  (2023) GPT-4 System Card . https://cdn.openai.com/papers/gpt -4-system -card.pdf  \nOpenAI  (2024) GPT-4 Technical Report. https://arxiv.org/pdf/2303.08774  \nPadmakumar, V. et al. (2024) Does writing with language models reduce content diversity?  ICLR . \nhttps://arxiv.org/pdf/2309.05196  \nPark,  P.  et. al. (2024)  AI deception: A survey of  examples, risks, and potential solutions. Patterns, 5(5).  \narXiv . https://arxiv.org/pdf/2308.14752'
 '11 value chain  (e.g., data inputs , processing, GAI training, or deployment  environments ), conventional \ncybersecurity practices  may need to adapt or evolve . \nFor instance , prompt  injection  involves  modifying what  input is provided to a  GAI system  so that it  \nbehave s in unintended ways. In direct prompt injections, attackers might craft malicious  prompts  and \ninput them directly to a GAI system , with a variety of  downstream negative consequences to \ninterconnected systems. Indirect prompt injection  attacks occur when  adversaries remotely (i.e., without \na direct interface) exploit LLM -integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by  steal ing proprietary  data  or running malicious code remotely  on a machine. Merely \nquerying  a closed production model can elicit  previously undisclosed information about that model . \nAnother cybersecurity risk to GAI is data poisoning , in which  an adversary compromises  a training \ndataset used by a model to manipulate its output s or operation. Malicious tampering with  data or parts \nof the  model could exacerbate risks associated with GAI system outputs.  \nTrustworthy AI Characteristics:  Privacy Enhanced, Safe , Secure and Resilient , Valid and Reliable  \n2.10. Intellectual Property  \nIntellectual property risks from  GAI systems may arise where the use of copyrighted works  is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs  displaying instances of  training data  memorization  (see Data Privacy above) could infringe on \ncopyright. \nHow GAI relates to copyright, including the status of  generated  content that is similar to but does not \nstrictly copy  w ork protected by copyright , is currently being debated in legal fora.  Similar discussions are'
 'AI incident \ndatabase , AVID , CVE, NVD , or OECD AI incident monitor ); Human oversight roles \nand responsibilities; Special rights and considerations for intellectual property, licensed works, or personal, privileged, proprietary or sensitive data; Underlying foundation models, versions of underlying models, and access modes . Data Privacy ; Human -AI \nConfiguration; Information \nIntegrity ; Intellectual Property ; \nValue Chain and Component Integration  \nAI Actor Tasks:  Governance and Oversight'
 'National Institute of Standards and Technology (2024) Adversarial Machine Learning: A Taxonomy and \nTerminology of Attacks and Mitigations https://csrc.nist.gov/pubs/ai/100/2/e2023/final  \nNational Institute of Standards and Technology  (2023) AI Risk Management Framework . \nhttps://www.nist.gov/itl/ai -risk-management -framework  \nNational Institute of Standards and Technology  (2023) AI Risk Management Framework, Chapter 3: AI \nRisks and Trustworthiness.  \nhttps://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Foundational_Information/3- sec-characteristics  \nNational Institute of Standards and Technology  (2023) AI Risk Management Framework , Chapter 6 : AI \nRMF Profiles. https://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Core_And_Profiles/6 -sec-profile  \nNational Institute of Standards and Technology  (2023) AI Risk Management Framework, Appendix A: \nDescriptions of AI Actor Tasks . \nhttps://airc.nist.gov/AI_RMF_Knowledge_Base/AI_RMF/Appendices/Appendix_A#:~:text=AI%20actors%\n20in%20this%20category,data%20providers%2C%20system%20funders%2C%20product']","Appropriate diligence on training data use is crucial for assessing intellectual property (IP) risks in AI systems in several ways:

1. **Understanding Data Sources**: Conducting thorough diligence involves identifying and documenting the sources of training data. This helps ensure that the data used is either owned by the organization, licensed appropriately, or falls under fair use. Knowing the origins of the data can prevent potential IP infringements (Source: [document name], p. 27).

2. **Evaluating Licensing Agreements**: Organizations should review licensing agreements associated with the training data to confirm that they have the right to use the data for their intended purposes. This includes understanding any restrictions on the use of copyrighted materials and ensuring compliance with those terms (Source: [document name], p. 27).

3. **Assessing Fair Use**: Diligence helps organizations evaluate whether their use of copyrighted materials qualifies as fair use. This involves analyzing factors such as the purpose of use, the nature of the copyrighted work, the amount used, and the effect on the market value of the original work. A clear understanding of these factors can mitigate the risk of legal challenges (Source: [document name], p. 27).

4. **Identifying Proprietary Data**: Organizations must be vigilant about using proprietary or sensitive data in training AI models. Diligence helps assess whether the use of such data is consistent with applicable laws and regulations, thereby reducing the risk of IP violations (Source: [document name], p. 27).

5. **Documenting Data Usage**: Keeping detailed records of how training data is sourced, used, and processed can provide a clear audit trail. This documentation is essential for demonstrating compliance with IP laws and can be invaluable in the event of disputes or legal inquiries (Source: [document name], p. 27).

6. **Mitigating Liability Risks**: By conducting appropriate diligence, organizations can identify and address potential IP risks before they escalate into legal issues. This proactive approach can help mitigate liability risks associated with IP infringement claims (Source: [document name], p. 27).

7. **Engaging Legal Expertise**: Involving legal experts during the diligence process can provide additional insights into IP risks and help organizations navigate complex legal landscapes related to AI and data usage (Source: [document name], p. 27).

In summary, appropriate diligence on training data use is essential for assessing and managing intellectual property risks in AI systems. It helps organizations ensure compliance with legal requirements, protect their own IP, and avoid potential legal disputes.","Appropriate diligence on training data use can help assess intellectual property risks in AI systems by examining whether the use of proprietary or sensitive training data aligns with relevant laws. This includes evaluating the likelihood and magnitude of potential impacts, both beneficial and harmful, based on past uses of AI systems in similar contexts, public incident reports, feedback from external parties, and other relevant data. By identifying and documenting these impacts, organizations can better understand the risks associated with their training data and take appropriate measures to mitigate them.",1.0,0.981217057423747,1.0,0.9999999999833333,0.4109802521636375
How do integrated human-AI systems benefit companies in providing customer service?,"['warrant additional human review, tracking and documentation, and greater management oversight.  \nAI technology can produce varied outputs  in multiple modalities and present many classes of user \ninterfaces. This leads to a broader set of AI Actors  interacting with GAI systems for widely differing \napplications and contexts of use. These  can include data labeling and preparation, development of GAI \nmodels, content moderation, code generation and review, text generation and editing, image and video \ngeneration, summarization, search, and chat. These activities can take place within organizational \nsettings or in the public domain.  \nOrganizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations  \n• AI actor credentials and qualifications  \n• Alignment to organizational values  • Auditing and assessment  \n• Change -management controls  \n• Commercial use  \n• Data provenance'
 ""HUMAN ALTERNATIVES, \nCONSIDERATION, AND \nFALLBACK \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \nHealthcare “navigators” help people find their way through online signup forms to choose \nand obtain healthcare. A Navigator is “an individual or organization that's trained and able to help \nconsumers, small businesses, and their employees as they look for health coverage options through the \nMarketplace (a government web site), including completing eligibility and enrollment forms.”106 For \nthe 2022 plan year, the Biden-Harris Administration increased funding so that grantee organizations could \n“train and certify more than 1,500 Navigators to help uninsured consumers find affordable and comprehensive \nhealth coverage. ”107\nThe customer service industry has successfully integrated automated services such as \nchat-bots and AI-driven call response systems with escalation to a human support team.\n108 Many businesses now use partially automated customer service platforms that help answer customer \nquestions and compile common problems for human agents to review. These integrated human-AI \nsystems allow companies to provide faster customer care while maintaining human agents to answer \ncalls or otherwise respond to complicated requests. Using both AI and human agents is viewed as key to \nsuccessful customer service.109\nBallot curing laws in at least 24 states require a fallback system that allows voters to \ncorrect their ballot and have it counted in the case that a voter signature matching algorithm incorrectly flags their ballot as invalid or there is another issue with their ballot, and review by an election official does not rectify the problem. Some federal courts have found that such cure procedures are constitutionally required.\n110 Ballot""
 'ENDNOTES\n12.Expectations about reporting are intended for the entity developing or using the automated system. The\nresulting reports can be provided to the public, regulators, auditors, industry standards groups, or others\nengaged in independent review, and should be made public as much as possible consistent with law,regulation, and policy, and noting that intellectual property or law enforcement considerations may preventpublic release. These reporting expectations are important for transparency, so the American people canhave confidence that their rights, opportunities, and access as well as their expectations aroundtechnologies are respected.\n13.National Artificial Intelligence Initiative Office. Agency Inventories of AI Use Cases. Accessed Sept. 8,\n2022. https://www.ai.gov/ai-use-case-inventories/\n14.National Highway Traffic Safety Administration. https://www.nhtsa.gov/\n15.See, e.g., Charles Pruitt. People Doing What They Do Best: The Professional Engineers and NHTSA . Public\nAdministration Review. Vol. 39, No. 4. Jul.-Aug., 1979. https://www.jstor.org/stable/976213?seq=116.The US Department of Transportation has publicly described the health and other benefits of these\n“traffic calming” measures. See, e.g.: U.S. Department of Transportation. Traffic Calming to Slow Vehicle\nSpeeds. Accessed Apr. 17, 2022. https://www.transportation.gov/mission/health/Traffic-Calming-to-Slow-\nVehicle-Speeds\n17.Karen Hao. Worried about your firm’s AI ethics? These startups are here to help.\nA growing ecosystem of “responsible AI” ventures promise to help organizations monitor and fix their AI\nmodels. MIT Technology Review. Jan 15., 2021.https://www.technologyreview.com/2021/01/15/1016183/ai-ethics-startups/ ; Disha Sinha. Top Progressive\nCompanies Building Ethical AI to Look Out for in 2021. Analytics Insight. June 30, 2021. https://\nwww.analyticsinsight.net/top-progressive-companies-building-ethical-ai-to-look-out-for-'
 'but are not limited to:  \n• Participatory Engagement Methods : Methods used to solicit feedback from civil society groups, \naffected communities, and users, including focus groups, small user studies, and surveys.  \n• Field Testing : Methods used to determine how people interact with, consume, use, and make \nsense of AI -generated information, and subsequent actions and effects, including UX, usability, \nand other structured, randomized experiments.  \n• AI Red -teaming:  A structured testing exercise\n used to probe an AI system to find flaws and \nvulnerabilities such as inaccurate, harmful, or discriminatory outputs, often in a controlled \nenvironment and in collaboration with system developers.  \nInformation gathered from structured public feedback can inform design, implementation, deployment \napproval , maintenance, or decommissioning decisions. Results and insights gleaned from these exercises \ncan serve multiple purposes, including improving data quality and preprocessing, bolstering governance decision making, and enhancing system documentation and debugging practices. When implementing \nfeedback activities, organizations should follow human subjects research requirements and best \npractices such as  informed consent and subject compensation.'
 'projections of student progress or outcomes, algorithms that determine access to resources or  \n    rograms, and surveillance of classes (whether online or in-person); \nHousing-related  systems such as tenant screening algorithms, automated valuation systems that  \n    estimate the value of homes used in mortgage underwriting or home insurance, and automated      valuations from online aggregator websites; and \nEmployment-related  systems such as workplace algorithms that inform all aspects of the terms  \n    and conditions of employment including, but not limited to, pay or promotion, hiring or termina-    tion algorithms, virtual or augmented reality workplace training programs, and electronic work \nplace surveillance and management systems. \n•Access to critical resources and services, including but not limited to:\nHealth  and health insurance technologies such as medical AI systems and devices, AI-assisted \n    diagnostic tools, algorithms or predictive models used to support clinical decision making, medical      or insurance health risk assessments, drug addiction risk assessments and associated access alg \n-orithms, wearable technologies, wellness apps, insurance care allocation algorithms, and health\ninsurance cost and underwriting algorithms;\nFinancial  system algorithms such as loan allocation algorithms, financial system access determi -\nnation algorithms, credit scoring systems, insurance algorithms including risk assessments, auto\n-mated interest rate determinations, and financial algorithms that apply penalties (e.g., that can\ngarnish wages or withhold tax returns);\n53'
 ""Maintained. The human consideration and fallback process and any associated automated processes \nshould be maintained and supported as long as the relevant automated system continues to be in use. \nInstitute training, assessment, and oversight to combat automation bias and ensure any \nhuman-based components of a system are effective. \nTraining and assessment. Anyone administering, interacting with, or interpreting the outputs of an auto -\nmated system should receive training in that system, including how to properly interpret outputs of a system in light of its intended purpose and in how to mitigate the effects of automation bias. The training should reoc\n-\ncur regularly to ensure it is up to date with the system and to ensure the system is used appropriately. Assess -\nment should be ongoing to ensure that the use of the system with human involvement provides for appropri -\nate results, i.e., that the involvement of people does not invalidate the system's assessment as safe and effective or lead to algorithmic discrimination. \nOversight. Human-based systems have the potential for bias, including automation bias, as well as other \nconcerns that may limit their effectiveness. The results of assessments of the efficacy and potential bias of such human-based systems should be overseen by governance structures that have the potential to update the operation of the human-based system in order to mitigate these effects. \n50""]","Integrated human-AI systems enhance customer service for companies in several key ways:

1. **Efficiency and Speed**: AI systems, such as chatbots and automated response systems, can handle a large volume of customer inquiries simultaneously. This allows for quicker response times, reducing wait times for customers and improving overall service efficiency (Source: [document name], p. 108).

2. **24/7 Availability**: AI-driven customer service solutions can operate around the clock, providing support to customers at any time of day. This continuous availability ensures that customers can receive assistance whenever they need it, enhancing their experience (Source: [document name], p. 108).

3. **Handling Routine Inquiries**: AI systems can effectively manage routine and repetitive inquiries, such as FAQs or basic troubleshooting. This frees up human agents to focus on more complex issues that require personal attention, leading to better resource allocation (Source: [document name], p. 108).

4. **Data Collection and Analysis**: Integrated systems can collect and analyze customer interaction data, providing valuable insights into customer preferences and behaviors. This information can help companies tailor their services and improve customer satisfaction (Source: [document name], p. 108).

5. **Personalization**: AI can analyze past interactions and customer data to provide personalized recommendations and responses. This level of personalization can enhance the customer experience by making interactions more relevant and engaging (Source: [document name], p. 108).

6. **Seamless Escalation**: When AI systems encounter complex issues that they cannot resolve, they can seamlessly escalate the inquiry to a human agent. This ensures that customers receive the appropriate level of support without experiencing frustration or delays (Source: [document name], p. 108).

7. **Cost-Effectiveness**: By automating routine tasks and inquiries, companies can reduce operational costs associated with customer service. This allows for more efficient use of resources while maintaining high service standards (Source: [document name], p. 108).

8. **Improved Customer Satisfaction**: The combination of quick responses, personalized service, and effective issue resolution leads to higher levels of customer satisfaction. Happy customers are more likely to return and recommend the company to others (Source: [document name], p. 108).

In summary, integrated human-AI systems enhance customer service by improving efficiency, availability, personalization, and overall customer satisfaction, while also allowing human agents to focus on more complex tasks.","Integrated human-AI systems benefit companies in providing customer service by allowing for faster customer care while maintaining human agents to handle complicated requests. These systems use partially automated platforms to answer common customer questions and compile issues for human agents to review, ensuring a balance between efficiency and personalized service.",0.2903225806451613,0.9842112100837633,1.0,0.8333333333055556,0.38928311917713543
What was the purpose of the year of public engagement that informed the development of the Blueprint for an AI Bill of Rights?,"['ing sessions, meetings, a formal request for information, and input to a publicly accessible and widely-publicized \nemail address, people throughout the United States, public servants across Federal agencies, and members of the \ninternational community spoke up about both the promises and potential harms of these technologies, and \nplayed a central role in shaping the Blueprint for an AI Bill of Rights. The core messages gleaned from these \ndiscussions include that AI has transformative potential to improve Americans’ lives, and that preventing the \nharms of these technologies is both necessary and achievable. The Appendix includes a full list of public engage-\nments. \n4'
 'About this Document \nThe Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People was \npublished by the White House Office of Science and Technology Policy in October 2022. This framework was \nreleased one year after OSTP announced  the launch of a process to develop “a bill of rights for an AI-powered \nworld.” Its release follows a year of public engagement to inform this initiative. The framework is available \nonline at: https://www.whitehouse.gov/ostp/ai-bill-of-rights \nAbout the Office of Science and Technology Policy \nThe Office of Science and Technology Policy (OSTP)  was established by the National Science and Technology  \nPolicy, Organization, and Priorities Act of 1976 to provide the President and others within the Executive Office \nof the President with advice on the scientific, engineering, and technological aspects of the economy, national \nsecurity, health, foreign relations, the environment, and the technological recovery and use of resources, among \nother topics. OSTP leads interagency science and technology policy coordination efforts, assists the Office of \nManagement and Budget (OMB) with an annual review and analysis of Federal research and development in \nbudgets, and serves as a source of scientific and technological analysis and judgment for the President with \nrespect to major policies, plans, and programs of the Federal Government.  \nLegal Disclaimer \nThe Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People is a white paper \npublished by the White House Office of Science and Technology Policy. It is intended to support the \ndevelopment of policies and practices that protect civil rights and promote democratic values in the building, \ndeployment, and governance of automated systems. \nThe Blueprint for an AI Bill of Rights is non-binding and does not constitute U.S. government policy. It'
 'SECTION  TITLE\n  \n \n \n  Applying The Blueprint for an AI Bill of Rights \nRELATIONSHIP TO EXISTING  LAW AND POLICY\nThere are regulatory safety requirements for medical devices, as well as sector-, population-, or technology-spe-\ncific privacy and security protections. Ensuring some of the additional protections proposed in this framework would require new laws to be enacted or new policies and practices to be adopted. In some cases, exceptions to the principles described in the Blueprint for an AI Bill of Rights may be necessary to comply with existing la w, \nconform to the practicalities of a specific use case, or balance competing public interests. In particula r, law \nenforcement, and other regulatory contexts may require government actors to protect civil rights, civil liberties, and privacy in a manner consistent with, but using alternate mechanisms to, the specific principles discussed in this framework. The Blueprint for an AI Bill of Rights is meant to assist governments and the private sector in moving principles into practice. \nThe expectations given in the Technical Companion are meant to serve as a blueprint for the development of \nadditional technical standards and practices that should be tailored for particular sectors and contexts. While \nexisting laws informed the development of the Blueprint for an AI Bill of Rights, this framework does not detail those laws beyond providing them as examples, where appropriate, of existing protective measures. This framework instead shares a broad, forward-leaning vision of recommended principles for automated system development and use to inform private and public involvement with these systems where they have the poten-tial to meaningfully impact rights, opportunities, or access. Additionall y, this framework does not analyze or \ntake a position on legislative and regulatory proposals in municipal, state, and federal government, or those in other countries.'
 'ABOUT THIS  FRAMEWORK\nThe Blueprint for an AI Bill of Rights is a set of five principles and associated practices to help guide the \ndesign, use, and deployment of automated systems to protect the rights of the American public in the age of \nartificial intel-ligence. Developed through extensive consultation with the American public, these principles are \na blueprint for building and deploying automated systems that are aligned with democratic values and protect \ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of Rights includes this Foreword, the five \nprinciples, notes on Applying the The Blueprint for an AI Bill of Rights, and a Technical Companion that gives \nconcrete steps that can be taken by many kinds of organizations—from governments at all levels to companies of \nall sizes—to uphold these values. Experts from across the private sector, governments, and international \nconsortia have published principles and frameworks to guide the responsible use of automated systems; this \nframework provides a national values statement and toolkit that is sector-agnostic to inform building these \nprotections into policy, practice, or the technological design process.  Where existing law or policy—such as \nsector-specific privacy laws and oversight requirements—do not already provide guidance, the Blueprint for an \nAI Bill of Rights should be used to inform policy decisions.\nLISTENING TO THE AMERICAN PUBLIC\nThe White House Office of Science and Technology Policy has led a year-long process to seek and distill input \nfrom people across the country—from impacted communities and industry stakeholders to technology develop-\ners and other experts across fields and sectors, as well as policymakers throughout the Federal government—on \nthe issue of algorithmic and data-driven harms and potential remedies. Through panel discussions, public listen-'
 ""Considered together, the five principles and associated practices of the Blueprint for an AI Bill of Rights form an overlapping set of backstops against potential harms. This purposefully overlapping framework, when taken as a whole, forms a blueprint to help protect the public from harm. The measures taken to realize the vision set forward in this framework should be proportionate with the extent and nature of the harm, or risk of harm, to people's rights, opportunities, and access. \nRELATIONSHIP TO EXISTING  LAW AND POLICY\nThe Blueprint for an AI Bill of Rights is an exercise in envisioning a future where the American public is \nprotected from the potential harms, and can fully enjoy the benefits, of automated systems. It describes princi -\nples that can help ensure these protections. Some of these protections are already required by the U.S. Constitu -\ntion or implemented under existing U.S. laws. For example, government surveillance, and data search and seizure are subject to legal requirements and judicial oversight. There are Constitutional requirements for human review of criminal investigative matters and statutory requirements for judicial review. Civil rights laws protect the American people against discrimination. \n8""
 ""APPENDIX\n• OSTP conducted meetings with a variety of stakeholders in the private sector and civil society. Some of these\nmeetings were specifically focused on providing ideas related to the development of the Blueprint for an AI\nBill of Rights while others provided useful general context on the positive use cases, potential harms, and/or\noversight possibilities for these technologies. Participants in these conversations from the private sector and\ncivil society included:\nAdobe \nAmerican Civil Liberties Union (ACLU) The Aspen Commission on Information Disorder The Awood Center The Australian Human Rights Commission Biometrics Institute The Brookings Institute BSA | The Software Alliance Cantellus Group Center for American Progress Center for Democracy and Technology Center on Privacy and Technology at Georgetown Law Christiana Care Color of Change Coworker Data Robot Data Trust Alliance Data and Society Research Institute Deepmind EdSAFE AI Alliance Electronic Privacy Information Center (EPIC) Encode Justice Equal AI Google Hitachi's AI Policy Committee The Innocence Project Institute of Electrical and Electronics Engineers (IEEE) Intuit Lawyers Committee for Civil Rights Under Law Legal Aid Society The Leadership Conference on Civil and Human Rights Meta Microsoft The MIT AI Policy Forum Movement Alliance Project The National Association of Criminal Defense Lawyers O’Neil Risk Consulting & Algorithmic Auditing The Partnership on AI Pinterest The Plaintext Group pymetrics SAP The Security Industry Association Software and Information Industry Association (SIIA) Special Competitive Studies Project Thorn United for Respect University of California at Berkeley Citris Policy Lab University of California at Berkeley Labor Center Unfinished/Project Liberty Upturn US Chamber of Commerce US Chamber of Commerce Technology Engagement Center \nA.I. Working Group\nVibrent HealthWarehouse Worker ResourceCenterWaymap\n62""]","The purpose of the public engagement year that informed the development of the Blueprint for an AI Bill of Rights was to gather input and insights from a diverse range of stakeholders, including the American public, impacted communities, industry stakeholders, technology developers, and policymakers. This extensive consultation aimed to understand both the potential benefits and the risks associated with artificial intelligence technologies. The feedback collected during this period played a central role in shaping the principles and practices outlined in the Blueprint, ensuring that the framework reflects the values and concerns of the public regarding the use of automated systems. Ultimately, the goal was to create a set of guidelines that would protect civil rights, promote democratic values, and help ensure that AI technologies are developed and deployed in a manner that is safe and beneficial for society (Source: [document name], p. 4).",The purpose of the year of public engagement that informed the development of the Blueprint for an AI Bill of Rights was to gather input and feedback from the public to shape the framework and ensure it reflects the values and concerns of the American people.,0.8888888888888888,0.9931498280058683,1.0,0.8766666666491334,0.6181185169475487
How can automated systems prevent 'mission creep' while ensuring privacy and user control?,"['DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nData access and correction. People whose data is collected, used, shared, or stored by automated \nsystems should be able to access data and metadata about themselves, know who has access to this data, and \nbe able to correct it if necessar y. Entities should receive consent before sharing data with other entities and \nshould keep records of what data is shared and with whom. \nConsent withdrawal and data deletion. Entities should allow (to the extent legally permissible) with -\ndrawal of data access consent, resulting in the deletion of user data, metadata, and the timely removal of their data from any systems (e.g., machine learning models) derived from that data.\n68\nAutomated system support. Entities designing, developing, and deploying automated systems should \nestablish and maintain the capabilities that will allow individuals to use their own automated systems to help them make consent, access, and control decisions in a complex data ecosystem. Capabilities include machine readable data, standardized data formats, metadata or tags for expressing data processing permissions and preferences and data provenance and lineage, context of use and access-specific tags, and training models for assessing privacy risk. \nDemonstrate that data privacy and user control are protected \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of the claims made regarding data policies. These independent evaluations should be \nmade public whenever possible. Care will need to be taken to balance individual privacy with evaluation data \naccess needs. \nReporting. When members of the public wish to know what data about them is being used in a system, the'
 'Risk identification and mitigation. Before deployment, and in a proactive and ongoing manner, poten -\ntial risks of the automated system should be identified and mitigated. Identified risks should focus on the potential for meaningful impact on people’s rights, opportunities, or access and include those to impacted communities that may not be direct users of the automated system, risks resulting from purposeful misuse of the system, and other concerns identified via the consultation process. Assessment and, where possible, mea\n-\nsurement of the impact of risks should be included and balanced such that high impact risks receive attention and mitigation proportionate with those impacts. Automated systems with the intended purpose of violating the safety of others should not be developed or used; systems with such safety violations as identified unin\n-\ntended consequences should not be used until the risk can be mitigated. Ongoing risk mitigation may necessi -\ntate rollback or significant modification to a launched automated system. \n18'
 'In addition to being able to opt out and use a human alternative, the American public deserves a human fallback system in the event that an automated system fails or causes harm. No matter how rigorously an automated system is tested, there will always be situations for which the system fails. The American public deserves protection via human review against these outlying or unexpected scenarios. In the case of time-critical systems, the public should not have to wait—immediate human consideration and fallback should be available. In many time-critical systems, such a remedy is already immediately available, such as a building manager who can open a door in the case an automated card access system fails. \nIn the criminal justice system, employment, education, healthcare, and other sensitive domains, automated systems are used for many purposes, from pre-trial risk assessments and parole decisions to technologies that help doctors diagnose disease. Absent appropriate safeguards, these technologies can lead to unfair, inaccurate, or dangerous outcomes. These sensitive domains require extra protections. It is critically important that there is extensive human oversight in such settings. \nThese critical protections have been adopted in some scenarios. Where automated systems have been introduced to provide the public access to government benefits, existing human paper and phone-based processes are generally still in place, providing an important alternative to ensure access. Companies that have introduced automated call centers often retain the option of dialing zero to reach an operator. When automated identity controls are in place to board an airplane or enter the country, there is a person supervising the systems who can be turned to for help or to appeal a misidentification.'
 'ers may differ depending on the specific automated system and development phase, but should include subject matter, sector-specific, and context-specific experts as well as experts on potential impacts such as civil rights, civil liberties, and privacy experts. For private sector applications, consultations before product launch may need to be confidential. Government applications, particularly law enforcement applications or applications that raise national security considerations, may require confidential or limited engagement based on system sensitivities and preexisting oversight laws and structures. Concerns raised in this consultation should be documented, and the automated system developers were proposing to create, use, or deploy should be reconsidered based on this feedback. \nTesting. Systems should undergo extensive testing before deployment. This testing should follow domain-specific best practices, when available, for ensuring the technology will work in its real-world context. Such testing should take into account both the specific technology used and the roles of any human operators or reviewers who impact system outcomes or effectiveness; testing should include both automated systems testing and human-led (manual) testing. Testing conditions should mirror as closely as possible the conditions in which the system will be deployed, and new testing may be required for each deployment to account for material differences in conditions from one deployment to another. Following testing, system performance should be compared with the in-place, potentially human-driven, status quo procedures, with existing human performance considered as a performance baseline for the algorithm to meet pre-deployment, and as a lifecycle minimum performance standard. Decision possibilities resulting from performance testing should include the possibility of not deploying the system.'
 'DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nProtect the public from unchecked surveillance \nHeightened oversight of surveillance. Surveillance or monitoring systems should be subject to \nheightened oversight that includes at a minimum assessment of potential harms during design (before deploy -\nment) and in an ongoing manner, to ensure that the American public’s rights, opportunities, and access are protected. This assessment should be done before deployment and should give special attention to ensure there is not algorithmic discrimination, especially based on community membership, when deployed in a specific real-world context. Such assessment should then be reaffirmed in an ongoing manner as long as the system is in use. \nLimited and proportionate surveillance. Surveillance should be avoided unless it is strictly necessary to achieve a legitimate purpose and it is proportionate to the need. Designers, developers, and deployers of surveillance systems should use the least invasive means of monitoring available and restrict monitoring to the minimum number of subjects possible. To the greatest extent possible consistent with law enforcement and national security needs, individuals subject to monitoring should be provided with clear and specific notice before it occurs and be informed about how the data gathered through surveillance will be used. \nScope limits on surveillance to protect rights and democratic values. Civil liberties and civil'
 'In discussion of technical and governance interventions that that are needed to protect against the harms of these technologies, panelists individually described the importance of: receiving community input into the design and use of technologies, public reporting on crucial elements of these systems, better notice and consent procedures that ensure privacy based on context and use case, ability to opt-out of using these systems and receive a fallback to a human process, providing explanations of decisions and how these systems work, the need for governance including training in using these systems, ensuring the technological use cases are genuinely related to the goal task and are locally validated to work, and the need for institution and protection of third party audits to ensure systems continue to be accountable and valid. \n57'
 'Data collection and use-case scope limits. Data collection should be limited in scope, with specific, \nnarrow identified goals, to avoid ""mission creep.""  Anticipated data collection should be determined to be strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on these identified goals and for a specific context should not be used in a different context without assessing for new privacy risks and implementing appropriate mitigation measures, which may include express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Determined data retention timelines should be documented and justi\n-\nfied. \nRisk identification and mitigation. Entities that collect, use, share, or store sensitive data should attempt to proactively identify harms and seek to manage them so as to avoid, mitigate, and respond appropri\n-\nately to identified risks. Appropriate responses include determining not to process data when the privacy risks outweigh the benefits or implementing measures to mitigate acceptable risks. Appropriate responses do not include sharing or transferring the privacy risks to users via notice or consent requests where users could not reasonably be expected to understand the risks without further support. \nPrivacy-preserving security. Entities creating, using, or governing automated systems should follow privacy and security best practices designed to ensure data and metadata do not leak beyond the specific consented use case. Best practices could include using privacy-enhancing cryptography or other types of privacy-enhancing technologies or fine-grained permissions and access control mechanisms, along with conventional system security protocols. \n33']","To prevent 'mission creep' while ensuring privacy and user control in automated systems, organizations can implement several strategies:

1. **Define Clear Objectives**: Establish specific, narrow goals for the automated system from the outset. This clarity helps prevent the system from being used for purposes beyond its original intent, thereby reducing the risk of mission creep (Source: [document name], p. 33).

2. **Data Minimization**: Adopt data minimization principles by collecting only the data that is strictly necessary to achieve the defined objectives. This limits the scope of data collection and helps prevent the system from expanding its data usage beyond what is required (Source: [document name], p. 33).

3. **User Consent and Control**: Ensure that users have control over their data by obtaining informed consent before data collection and allowing them to withdraw consent easily. Users should also have the ability to access, correct, and delete their data, reinforcing their control over personal information (Source: [document name], p. 33).

4. **Regular Audits and Reviews**: Conduct regular audits and reviews of the automated system's data collection practices and use-case scopes. This ongoing assessment helps identify any potential drift from the original objectives and allows for timely adjustments (Source: [document name], p. 33).

5. **Stakeholder Engagement**: Involve relevant stakeholders, including privacy experts and affected communities, in the design and implementation of the automated system. Their input can help ensure that privacy concerns are addressed and that the system remains focused on its intended purpose (Source: [document name], p. 33).

6. **Establish Clear Data Retention Policies**: Define and document clear timelines for data retention, ensuring that data is deleted as soon as it is no longer necessary for the identified goals. This practice helps prevent the accumulation of unnecessary data and reduces the risk of mission creep (Source: [document name], p. 33).

7. **Transparency and Accountability**: Maintain transparency about how data is collected, used, and shared. Providing clear information to users about the system's operations fosters trust and accountability, making it less likely for the system to deviate from its intended purpose (Source: [document name], p. 33).

By implementing these strategies, organizations can effectively prevent mission creep while ensuring that privacy and user control are prioritized in the design and operation of automated systems.","Automated systems can prevent 'mission creep' and ensure privacy and user control by limiting data collection to specific, narrow goals that are strictly necessary for the identified purposes. Data collection should be minimized, clearly communicated to users, and used only for legal and expected purposes. Any use of data in a different context should be assessed for new privacy risks and appropriate mitigation measures should be implemented, potentially including obtaining express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Entities should proactively identify and manage privacy risks, avoiding processing data when risks outweigh benefits and implementing measures to mitigate acceptable risks. Privacy-preserving security measures, such as privacy-enhancing cryptography and access control mechanisms, should be employed to prevent data leakage beyond consented use cases.",1.0,0.9615231465422364,0.8333333333333334,0.9999999999857143,0.9838641898790255
"How can GAI tech improve red-teaming with human teams, ensuring content origin and incident disclosure?","['51 general public participants. For example, expert AI red- teamers could modify or verify the \nprompts written by general public AI red- teamers. These approaches may also expand coverage \nof the AI risk attack surface.  \n• Human / AI: Performed by GAI in combinatio n with  specialist or non -specialist human teams. \nGAI- led red -teaming can be more cost effective than human red- teamers alone. Human or GAI-\nled AI red -teaming may be better suited for eliciting different types of harms.   \nA.1.6.  Content Provenance  \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. \nSome aspects of GAI output s, such as the production of deepfake content, can challenge our ability to \ndistinguish human- generated content from AI -generated synthetic  content. To help manage and mitigate \nthese risks, digital transparency mechanisms like provenance data tracking can trace the origin and \nhistory of content. Provenance data tracking and synthetic content detection can help facilitate greater \ninformation access  about both authentic and synthetic content to users, enabling better knowledge of  \ntrustworthiness  in AI systems. When combined with other organizational accountability mechanisms, \ndigital content transparency approaches  can enable processes to trace negative outcomes back to their \nsource, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin \nand history of content  to assist in \nGAI risk management efforts.  \nProvenance metad ata can include information about GAI model developers or creators  of GAI content , \ndate/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of p rovenance data tracking techniques  can'
 '53 Documenting, reporting, and sharing information about GAI incidents can help mitigate and prevent \nharmful outcomes by assisting relevant AI Actors  in tracing impacts to their source . Greater awareness \nand standardization of GAI incident reporting could promote this transparency and improve GAI risk management across the AI ecosystem.  \nDocumentation and Involvement of AI Actors  \nAI Actors  should be aware of their roles in reporting AI incidents. To better understand previous incidents \nand implement measures to prevent similar ones in the future, organizations could consider developing guidelines for publicly available incident reporting which include information about AI actor \nresponsibilities. These  guidelines would help AI system operators identify GAI incidents across the AI \nlifecycle and with AI Actors  regardless of role . Documentation and review of third -party inputs and \nplugins for GAI systems is especially important for AI Actors  in the context of incident disclosure; LLM \ninputs and content delivered through these plugins is often distributed,\n with inconsistent or insufficient \naccess control.  \nDocumentation practices including logging, recording, and analyzing GAI incidents can facilitate \nsmoother sharing of information with relevant AI Actors . Regular information sharing, change \nmanagement records, version history and metadata can also empower AI Actors  responding to and \nmanaging AI incidents.'
 'public; this section focuses on red- teaming in pre -deployment contexts.  \nThe quality of AI red- teaming outputs is  related to the background and expertise of the AI red  team \nitself. Demographically and interdisciplinarily  diverse AI red  teams can be used to identify flaws in the \nvarying contexts where GAI will be used. For best results, AI red  teams should demonstrate domain \nexpertise, and awareness of socio -cultural aspects within the deployment context. AI red -teaming results \nshould be given additional analysis before they are incorporated into organizational governance and \ndecision making, policy and procedural updates, and AI risk management efforts. \nVarious types of AI red -teaming may be appropriate, depending on the use case:  \n• General Public: Performed by general users (not necessarily AI or technical experts) who are \nexpected to use the model or interact with its outputs, and who bring their own lived \nexperiences and perspectives to the task of AI red -teaming . These individuals may have been \nprovided instructions and material to complete tasks which may elicit harmful model behaviors. This type of exercise can be more effective with large groups of AI  red-teamers.  \n• Expert: Performed by specialists with expertise in the domain or specific AI red -teaming context \nof use (e.g., medicine, biotech, cybersecurity).  \n• Combination: In scenarios when it is difficult to identify and recruit specialists with sufficient \ndomain and contextual expertise, AI red -teaming exercises may leverage both expert and'
 '10 GAI systems can ease the unintentional  production or dissemination of false, inaccurate, or misleading \ncontent (misinformation) at scale , particularly if the content stems from confabulations.  \nGAI systems can also  ease the deliberate  production or dis semination of false or misleading information  \n(disinformation) at scale, where an actor  has the explicit intent to deceive or cause harm to others. Even \nvery subtle changes  to text or images  can manipulate human and machine perception.  \nSimilarly, GAI systems could  enable a higher degree of sophistication  for malicious actors to produce \ndisinformation  that is targeted towards specific demographics. Current and e merging multimodal  models \nmake it possible to generate both text-based disinformation and highly realistic “ deepfakes ” – that is,  \nsynthetic audiovisual content and photorealistic images.12 Additional disinformation threats could  be \nenabled by future GAI models trained on new data modalities. \nDisinformation and misinformation  – both of which may be facilitated  by GAI – may erode public trust  in \ntrue or valid evidence and information, with downstream effects. For example, a synthetic image of a \nPentagon blast went viral  and briefly caused a drop in the stock market. Generative AI models can also \nassist malicious actors in creating compelling imagery and propaganda to support disinformation campaigns, which may not be photorealistic, but could enable these campaigns to ga in more reach and \nengagement on social media platforms.  Additionally, g enerative AI models can assist malicious actors in \ncreating fraudulent content intended to impersonate others.  \nTrustworthy AI Characteristics:  Accountable and Transparent,  Safe , Valid and Reliable , Interpretable and \nExplainable  \n2.9. Information Security  \nInformation security for computer systems and data is a mature field with widely accepted and'
 '43 MG-3.1-005 Review  various transparency artifacts (e.g., system cards and model cards) for \nthird -party models.  Information Integrity ; Information \nSecurity ; Value Chain and \nComponent Integration  \nAI Actor Tasks:  AI Deployment, Operation and Monitoring, Third -party entities  \n \nMANAGE 3.2:  Pre-trained models which are used for development are monitored as part of AI system regular monitoring and \nmaintenance.  \nAction ID  Suggested Action  GAI Risks  \nMG-3.2-001 Apply explainable AI (XAI) techniques (e.g., analysis of embeddings, model \ncompression/distillation, gradient -based attributions, occlusion/term reduction, \ncounterfactual prompts, word clouds) as part of ongoing continuous \nimprovement processes to mitigate risks related to unexplainable GAI systems.  Harmful Bias and Homogenization  \nMG-3.2-002 Document how pre -trained models have been adapted ( e.g., fine-tuned , or \nretrieval- augmented generation) for the specific generative task, including any \ndata augmentations, parameter adjustments, or other modifications. Access to un-tuned (baseline) models support s debugging the relative influence of the pre -\ntrained weights compared to the fine -tuned model weights  or other system \nupdates . Information Integrity ; Data Privacy  \nMG-3.2-003 Document sources and types of training data and their origins, potential biases \npresent in the data related to the GAI application and its content provenance, \narchitecture, training process of the pre -trained model including information on \nhyperparameters,  training duration, and any fine -tuning or retrieval- augmented \ngeneration processes applied.  Information Integrity ; Harmful Bias \nand Homogenization ; Intellectual \nProperty  \nMG-3.2-004 Evaluate user reported problematic content and integrate feedback into system \nupdates.  Human -AI Configuration , \nDangerous , Violent, or Hateful \nContent']","Generative AI (GAI) technology can significantly enhance red-teaming efforts when combined with human teams by providing tools and capabilities that improve the identification and mitigation of risks. Here’s how GAI can contribute while ensuring transparency regarding content origin and incident disclosure:

1. **Automated Scenario Generation**: GAI can create a wide range of realistic scenarios and prompts that human red-teamers can use to test AI systems. This capability allows for more comprehensive testing of potential vulnerabilities and harmful behaviors, as GAI can generate diverse and complex situations that might not be easily conceived by human teams alone (Source: [document name], p. 51).

2. **Data Analysis and Insights**: GAI can assist in analyzing large volumes of data generated during red-teaming exercises. By identifying patterns and anomalies, GAI can help human teams focus on the most critical issues and provide insights that enhance the overall effectiveness of the red-teaming process (Source: [document name], p. 51).

3. **Enhanced Collaboration**: GAI can facilitate better collaboration between human team members by providing real-time suggestions and feedback during red-teaming exercises. This can lead to more dynamic and effective discussions, allowing teams to adapt their strategies based on GAI-generated insights (Source: [document name], p. 51).

4. **Content Provenance Tracking**: To ensure transparency regarding content origin, organizations can implement provenance data tracking mechanisms that record the source and history of GAI-generated content. This includes metadata about the GAI model used, the date and time of content creation, and any modifications made. Such tracking helps maintain accountability and allows teams to trace back any negative outcomes to their source (Source: [document name], p. 53).

5. **Incident Reporting Frameworks**: Establishing clear guidelines for incident reporting can enhance transparency in GAI red-teaming efforts. Organizations should document and share information about GAI incidents, including the roles of various AI actors involved. This practice promotes awareness and standardization in incident reporting, which is essential for effective risk management (Source: [document name], p. 53).

6. **Regular Review and Updates**: Continuous improvement of GAI systems and red-teaming practices is vital. Organizations should regularly review the effectiveness of GAI tools and the transparency mechanisms in place, ensuring that they adapt to new challenges and maintain high standards of accountability (Source: [document name], p. 51).

7. **Stakeholder Engagement**: Involving stakeholders in the development and implementation of GAI-enhanced red-teaming efforts can help ensure that transparency and ethical considerations are prioritized. Engaging with affected communities and experts can provide valuable perspectives on how to manage risks effectively (Source: [document name], p. 51).

By leveraging GAI technology in these ways, organizations can enhance their red-teaming efforts while ensuring that transparency regarding content origin and incident disclosure is maintained, ultimately leading to more robust AI risk management practices.","GAI technologies can improve red-teaming with human teams by combining GAI with specialist or non-specialist human teams. GAI-led red-teaming can be more cost-effective than human red-teamers alone and may be better suited for eliciting different types of harms. Content provenance mechanisms like provenance data tracking can trace the origin and history of content, helping to manage and mitigate risks associated with GAI output. These approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust.",1.0,0.899092046066455,1.0,0.99999999998,0.6203223584742575
Why is it important for lenders to inform consumers about decisions made under FCRA in automated systems?,"['NOTICE & \nEXPLANATION \nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \nPeople in Illinois are given written notice by the private sector if their biometric informa-\ntion is used . The Biometric Information Privacy Act enacted by the state contains a number of provisions \nconcerning the use of individual biometric data and identifiers. Included among them is a provision that no private \nentity may ""collect, capture, purchase, receive through trade, or otherwise obtain"" such information about an \nindividual, unless written notice is provided to that individual or their legally appointed representative. 87\nMajor technology  companies are piloting new ways to communicate with the public about \ntheir automated  technologies. For example, a collection of non-profit organizations and companies have \nworked together to develop a framework that defines operational approaches to transparency for machine \nlearning systems.88 This framework, and others like it,89 inform the public about the use of these tools, going \nbeyond simple notice to include reporting elements such as safety evaluations, disparity assessments, and \nexplanations of how the systems work. \nLenders are required by federal law to notify consumers about certain decisions made about \nthem. Both the Fair Credit Reporting Act and the Equal Credit Opportunity Act require in certain circumstances \nthat consumers who are denied credit receive ""adverse action"" notices. Anyone who relies on the information in a \ncredit report to deny a consumer credit must, under the Fair Credit Reporting Act, provide an ""adverse action"" \nnotice to the consumer, which includes ""notice of the reasons a creditor took adverse action on the application'
 ""automated systems in an equitable way . The guardrails protecting the public from discrimination in their daily \nlives should include their digital lives and impacts—basic safeguards against abuse, bias, and discrimination to \nensure that all people are treated fairly when automated systems are used. This includes all dimensions of their \nlives, from hiring to loan approvals, from medical treatment and payment to encounters with the criminal \njustice system. Ensuring equity should also go beyond existing guardrails to consider the holistic impact that \nautomated systems make on underserved communities and to institute proactive protections that support these \ncommunities. \n• An automated system using nontraditional factors such as educational attainment and employment history as\npart of its loan underwriting and pricing model was found to be much more likely to charge an applicant whoattended a Historically Black College or University (HBCU) higher loan prices for refinancing a student loanthan an applicant who did not attend an HBCU. This was found to be true even when controlling for\nother credit-related factors.32\n•A hiring tool that learned the features of a company's employees (predominantly men) rejected women appli -\ncants for spurious and discriminatory reasons; resumes with the word “women’s,” such as “women’s\nchess club captain,” were penalized in the candidate ranking.33\n•A predictive model marketed as being able to predict whether students are likely to drop out of school wasused by more than 500 universities across the countr y. The model was found to use race directly as a predicto r,\nand also shown to have large disparities by race; Black students were as many as four times as likely as theirotherwise similar white peers to be deemed at high risk of dropping out. These risk scores are used by advisors to guide students towards or away from majors, and some worry that they are being used to guide\nBlack students away from math and science subjects.34""
 'You should know that an automated system is being used, \nand understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automat\n-\ned systems should provide generally accessible plain language docu -\nmentation including clear descriptions of the overall system func -\ntioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and ex\n-\nplanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality chang\n-\nes. You should know how and why an outcome impacting you was de -\ntermined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible.   NOTICE AND  EXPLANATION\n40'
 'or on an existing credit account.""90 In addition, under the risk-based pricing rule,91 lenders must either inform \nborrowers of their credit score, or else tell consumers when ""they are getting worse terms because of \ninformation in their credit report."" The CFPB has also asserted that ""[t]he law gives every applicant the right to \na specific explanation if their application for credit was denied, and that right is not diminished simply because \na company uses a complex algorithm  that it doesn\'t understand.""92 Such explanations illustrate a shared value \nthat certain decisions need to be explained. \nA California law  requires  that warehouse employees are  provided with  notice and  explana-\ntion about quotas, potentially facilitated by automated systems, that apply to them. Warehous-\ning employers in California that use quota systems (often facilitated by algorithmic monitoring systems) are \nrequired to provide employees with a written description of each quota that applies to the employee, including \n“quantified number of tasks to be performed or materials to be produced or handled, within the defined \ntime period, and any potential adverse employment action that could result from failure to meet the quota.”93\nAcross the federal government, agencies are conducting and supporting research on explain-\nable AI systems. The NIST is conducting fundamental research on the explainability of AI systems. A multidis-\nciplinary team of researchers aims to develop measurement methods and best practices to support the \nimplementation of core tenets of explainable AI.94 The Defense Advanced Research Projects Agency has a \nprogram on Explainable Artificial Intelligence that aims to create a suite of machine learning techniques that \nproduce more explainable models, while maintaining a high level of learning performance (prediction \naccuracy), and enable human users to understand, appropriately trust, and effectively manage the emerging'
 ""Reporting. Summary reporting should document the determinations made based on the above consider -\nations, including: the responsible entities for accountability purposes; the goal and use cases for the system, identified users, and impacted populations; the assessment of notice clarity and timeliness; the assessment of the explanation's validity and accessibility; the assessment of the level of risk; and the account and assessment of how explanations are tailored, including to the purpose, the recipient of the explanation, and the level of risk. Individualized profile information should be made readily available to the greatest extent possible that includes explanations for any system impacts or inferences. Reporting should be provided in a clear plain language and machine-readable manner. \n44""]","It is important for lenders to inform consumers about decisions made under the Fair Credit Reporting Act (FCRA) in automated systems for several key reasons:

1. **Consumer Rights Protection**: The FCRA mandates that consumers have the right to be informed when adverse actions are taken based on information in their credit reports. This includes providing consumers with ""adverse action"" notices that explain the reasons for the decision. Informing consumers helps protect their rights and ensures they are aware of how their credit information is being used (Source: [document name], p. 90).

2. **Transparency**: Providing clear explanations about decisions made by automated systems fosters transparency in the lending process. When consumers understand how decisions are made, they are more likely to trust the system and the lender, which can enhance the overall customer relationship (Source: [document name], p. 90).

3. **Opportunity for Correction**: When consumers receive information about adverse decisions, they have the opportunity to review their credit reports for inaccuracies or errors. This allows them to correct any mistakes that may have negatively impacted their creditworthiness, promoting fairness in the lending process (Source: [document name], p. 90).

4. **Encouraging Responsible Lending Practices**: By requiring lenders to provide explanations, the FCRA encourages responsible lending practices. Lenders must ensure that their automated systems are functioning correctly and that they are not unfairly discriminating against consumers based on inaccurate or biased data (Source: [document name], p. 90).

5. **Facilitating Recourse**: When consumers are informed about the reasons for adverse actions, they are better equipped to seek recourse if they believe the decision was unjust. This can include disputing the decision or requesting a review by a human agent, which is essential for accountability in automated systems (Source: [document name], p. 90).

6. **Promoting Fairness and Equity**: Informing consumers about decisions made by automated systems helps ensure that all individuals are treated fairly and equitably. This is particularly important in preventing discrimination and ensuring that lending practices do not disproportionately affect certain groups (Source: [document name], p. 90).

In summary, informing consumers about decisions made under the FCRA in automated systems is crucial for protecting their rights, promoting transparency, encouraging responsible lending, and ensuring fairness in the credit process.","It is important for lenders to inform consumers about decisions made under FCRA in automated systems because the Fair Credit Reporting Act requires that consumers who are denied credit receive ""adverse action"" notices. These notices must include the reasons for the adverse action taken on the application or an existing credit account. Additionally, under the risk-based pricing rule, lenders must inform borrowers of their credit score or explain when they are receiving worse terms due to information in their credit report. This transparency is crucial to ensure that consumers understand the basis for credit decisions, especially when complex algorithms are involved.",0.9565217391304348,0.9785683808084092,0.75,0.99999999998,0.8863608174353528
Why is public transparency important in automated systems affecting people's lives and decisions?,"['NOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nAutomated systems now determine opportunities, from employment to credit, and directly shape the American \npublic’s experiences, from the courtroom to online classrooms, in ways that profoundly impact people’s lives. But this expansive impact is not always visible. An applicant might not know whether a person rejected their resume or a hiring algorithm moved them to the bottom of the list. A defendant in the courtroom might not know if a judge deny\n-\ning their bail is informed by an automated system that labeled them “high risk.” From correcting errors to contesting decisions, people are often denied the knowledge they need to address the impact of automated systems on their lives. Notice and explanations also serve an important safety and efficacy purpose, allowing experts to verify the reasonable\n-\nness of a recommendation before enacting it. \nIn order to guard against potential harms, the American public needs to know if an automated system is being used. Clear, brief, and understandable notice is a prerequisite for achieving the other protections in this framework. Like\n-\nwise, the public is often unable to ascertain how or why an automated system has made a decision or contributed to a particular outcome. The decision-making processes of automated systems tend to be opaque, complex, and, therefore, unaccountable, whether by design or by omission. These factors can make explanations both more challenging and more important, and should not be used as a pretext to avoid explaining important decisions to the people impacted by those choices. In the context of automated systems, clear and valid explanations should be recognized as a baseline requirement.'
 'DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nProtect the public from unchecked surveillance \nHeightened oversight of surveillance. Surveillance or monitoring systems should be subject to \nheightened oversight that includes at a minimum assessment of potential harms during design (before deploy -\nment) and in an ongoing manner, to ensure that the American public’s rights, opportunities, and access are protected. This assessment should be done before deployment and should give special attention to ensure there is not algorithmic discrimination, especially based on community membership, when deployed in a specific real-world context. Such assessment should then be reaffirmed in an ongoing manner as long as the system is in use. \nLimited and proportionate surveillance. Surveillance should be avoided unless it is strictly necessary to achieve a legitimate purpose and it is proportionate to the need. Designers, developers, and deployers of surveillance systems should use the least invasive means of monitoring available and restrict monitoring to the minimum number of subjects possible. To the greatest extent possible consistent with law enforcement and national security needs, individuals subject to monitoring should be provided with clear and specific notice before it occurs and be informed about how the data gathered through surveillance will be used. \nScope limits on surveillance to protect rights and democratic values. Civil liberties and civil'
 'In discussion of technical and governance interventions that that are needed to protect against the harms of these technologies, panelists individually described the importance of: receiving community input into the design and use of technologies, public reporting on crucial elements of these systems, better notice and consent procedures that ensure privacy based on context and use case, ability to opt-out of using these systems and receive a fallback to a human process, providing explanations of decisions and how these systems work, the need for governance including training in using these systems, ensuring the technological use cases are genuinely related to the goal task and are locally validated to work, and the need for institution and protection of third party audits to ensure systems continue to be accountable and valid. \n57'
 'In addition to being able to opt out and use a human alternative, the American public deserves a human fallback system in the event that an automated system fails or causes harm. No matter how rigorously an automated system is tested, there will always be situations for which the system fails. The American public deserves protection via human review against these outlying or unexpected scenarios. In the case of time-critical systems, the public should not have to wait—immediate human consideration and fallback should be available. In many time-critical systems, such a remedy is already immediately available, such as a building manager who can open a door in the case an automated card access system fails. \nIn the criminal justice system, employment, education, healthcare, and other sensitive domains, automated systems are used for many purposes, from pre-trial risk assessments and parole decisions to technologies that help doctors diagnose disease. Absent appropriate safeguards, these technologies can lead to unfair, inaccurate, or dangerous outcomes. These sensitive domains require extra protections. It is critically important that there is extensive human oversight in such settings. \nThese critical protections have been adopted in some scenarios. Where automated systems have been introduced to provide the public access to government benefits, existing human paper and phone-based processes are generally still in place, providing an important alternative to ensure access. Companies that have introduced automated call centers often retain the option of dialing zero to reach an operator. When automated identity controls are in place to board an airplane or enter the country, there is a person supervising the systems who can be turned to for help or to appeal a misidentification.'
 'should not be used in education, work, housing, or in other contexts where the use of such surveillance \ntechnologies is likely to limit rights, opportunities, or access. Whenever possible, you should have access to \nreporting that confirms your data decisions have been respected and provides an assessment of the \npotential impact of surveillance technologies on your rights, opportunities, or access. \nNOTICE AND EXPLANATION\nYou should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organiza\n-\ntion responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible. \n6'
 'DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nData access and correction. People whose data is collected, used, shared, or stored by automated \nsystems should be able to access data and metadata about themselves, know who has access to this data, and \nbe able to correct it if necessar y. Entities should receive consent before sharing data with other entities and \nshould keep records of what data is shared and with whom. \nConsent withdrawal and data deletion. Entities should allow (to the extent legally permissible) with -\ndrawal of data access consent, resulting in the deletion of user data, metadata, and the timely removal of their data from any systems (e.g., machine learning models) derived from that data.\n68\nAutomated system support. Entities designing, developing, and deploying automated systems should \nestablish and maintain the capabilities that will allow individuals to use their own automated systems to help them make consent, access, and control decisions in a complex data ecosystem. Capabilities include machine readable data, standardized data formats, metadata or tags for expressing data processing permissions and preferences and data provenance and lineage, context of use and access-specific tags, and training models for assessing privacy risk. \nDemonstrate that data privacy and user control are protected \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of the claims made regarding data policies. These independent evaluations should be \nmade public whenever possible. Care will need to be taken to balance individual privacy with evaluation data \naccess needs. \nReporting. When members of the public wish to know what data about them is being used in a system, the'
 ""Providing notice has long been a standard practice, and in many cases is a legal requirement, when, for example, making a video recording of someone (outside of a law enforcement or national security context). In some cases, such as credit, lenders are required to provide notice and explanation to consumers. Techniques used to automate the process of explaining such systems are under active research and improvement and such explanations can take many forms. Innovative companies and researchers are rising to the challenge and creating and deploying explanatory systems that can help the public better understand decisions that impact them. \nWhile notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. \n• A lawyer representing an older client with disabilities who had been cut off from Medicaid-funded home\nhealth-care assistance couldn't determine why\n, especially since the decision went against historical access\npractices. In a court hearing, the lawyer learned from a witness that the state in which the older client\nlived \nhad recently adopted a new algorithm to determine eligibility.83 The lack of a timely explanation made it\nharder \nto understand and contest the decision.\n•\nA formal child welfare investigation is opened against a parent based on an algorithm and without the parent\never \nbeing notified that data was being collected and used as part of an algorithmic child maltreatment\nrisk assessment.84 The lack of notice or an explanation makes it harder for those performing child\nmaltreatment assessments to validate the risk assessment and denies parents knowledge that could help them\ncontest a decision.\n41""]","Public transparency is crucial in automated systems that impact individuals' lives and decisions for several reasons:

1. **Accountability**: Transparency ensures that organizations are held accountable for the decisions made by automated systems. When individuals understand how these systems operate and the criteria used for decision-making, they can better assess whether the outcomes are fair and justified (Source: [document name], p. 6).

2. **Trust Building**: When organizations are transparent about their automated systems, it fosters trust among users and the general public. People are more likely to accept and engage with systems they understand and believe are operating fairly (Source: [document name], p. 6).

3. **Informed Consent**: Transparency allows individuals to make informed choices about their interactions with automated systems. When users know how their data will be used and how decisions affecting them are made, they can provide informed consent and exercise control over their personal information (Source: [document name], p. 6).

4. **Error Correction**: Public transparency enables individuals to identify and contest errors in automated decision-making. If people are aware of how decisions are made, they can more easily spot inaccuracies and seek corrections, which is essential for fairness (Source: [document name], p. 6).

5. **Prevention of Discrimination**: Transparent systems can help identify and mitigate biases that may lead to discriminatory outcomes. By making the decision-making processes visible, organizations can be held accountable for ensuring that their systems do not unfairly disadvantage certain groups (Source: [document name], p. 6).

6. **Public Engagement**: Transparency encourages public engagement and dialogue about the use of automated systems. When individuals are informed about how these systems work, they can contribute to discussions about ethical considerations, potential risks, and improvements (Source: [document name], p. 6).

7. **Regulatory Compliance**: Many jurisdictions require transparency in automated decision-making processes as part of regulatory frameworks. Adhering to these requirements not only ensures compliance but also demonstrates a commitment to ethical practices (Source: [document name], p. 6).

In summary, public transparency in automated systems is essential for accountability, trust, informed consent, error correction, prevention of discrimination, public engagement, and regulatory compliance. It ultimately contributes to the ethical and responsible use of technology in ways that respect individuals' rights and dignity.","Public transparency is crucial in automated systems affecting people's lives and decisions because it allows both the authorities and the public to understand why and how decisions are being made. Without transparency, individuals may be subject to decisions made by automated systems without any explanation or accountability, leading to potential errors, biases, and injustices. Transparency also enables experts to verify the reasonableness of recommendations before they are implemented, ensuring safety and efficacy. In summary, public transparency in automated systems is essential for accountability, fairness, and the protection of individuals' rights and opportunities.",1.0,0.9680746218583742,1.0,0.9999999999857143,0.5162162121207887
How can governance principles manage risks of GAI effectively?,"['47 Appendix A.  Primary GAI Considerations  \nThe following primary considerations were derived as overarching themes from the GAI PWG \nconsultation process. These considerations (Governance, Pre- Deployment Testing, Content Provenance, \nand Incident Disclosure) are relevant  for volun tary use by any organization designing, developing, and \nusing GAI  and also inform the Actions to Manage GAI risks. Information included about the primary \nconsiderations is not exhaustive , but highlights the most relevant topics derived from the GAI PWG.  \nAcknowledgments: These considerations could not have been surfaced without the helpful analysis and \ncontributions from the community and NIST staff GAI PWG leads: George Awad, Luca Belli, Harold Booth, \nMat Heyman, Yoo young Lee, Mark Pryzbocki, Reva Schwartz, Martin Stanley, and Kyra Yee.  \nA.1. Governance  \nA.1.1.  Overview  \nLike any other technology system, governance principles and techniques can be used to manage risks \nrelated to generative AI models, capabilities, and applications. Organizations may choose to apply their \nexisting risk tiering to GAI systems, or they may op t to revis e or update AI system risk levels to address \nthese unique GAI risks. This section describes how organizational governance regimes may be re -\nevaluated and adjusted for GAI contexts. It also addresses third -party considerations for governing across  \nthe AI value chain.  \nA.1.2.  Organizational  Governance  \nGAI opportunities, risks and long- term performance characteristics are typically less well -understood \nthan non- generative AI tools  and may be perceived and acted upon by humans in ways that vary greatly. \nAccordingly, GAI may call for different levels of oversight from AI Actors  or different human- AI \nconfigurations in order to manage their risks effectively. Organizations’ use of GAI systems may also \nwarrant additional human review, tracking and documentation, and greater management oversight.'
 '18 GOVERN 3.2:  Policies and procedures are in place to define and differentiate roles and responsibilities for human -AI configurations \nand oversight of AI systems.  \nAction ID  Suggested Action  GAI Risks  \nGV-3.2-001 Policies are in place to bolster oversight of GAI systems with independent \nevaluations  or assessments of GAI models or systems  where the type and \nrobustness of evaluations are proportional to the identified risks.  CBRN Information or Capabilities ; \nHarmful Bias and Homogenization  \nGV-3.2-002 Consider adjustment of organizational roles and components across lifecycle \nstages of large or complex GAI systems, including: Test and evaluation, validation, \nand red- teaming of GAI systems; GAI content moderation; GAI system \ndevelopment and engineering; Increased accessibility of GAI tools, interfaces, and \nsystems, Incident response and containment.  Human -AI Configuration ; \nInformation Security ; Harmful Bias \nand Homogenization  \nGV-3.2-003 Define acceptable use policies for GAI interfaces, modalities, and human -AI \nconfigurations  (i.e., for chatbots and decision -making tasks) , including  criteria for \nthe kinds of queries GAI applications should refuse to respond to.  Human -AI Configuration  \nGV-3.2-004 Establish policies for user feedback mechanisms for GAI systems  which include \nthorough instructions and any mechanisms for recourse . Human -AI Configuration   \nGV-3.2-005 Engage in threat modeling to anticipate potential risks from GAI systems.  CBRN Information or Capabilities ; \nInformation Security  \nAI Actors: AI Design  \n \nGOVERN 4.1:  Organizational policies and practices are in place to foster a critical thinking and safety -first mindset in the design, \ndevelopment, deployment, and uses of AI systems to minimize potential negative impacts.  \nAction ID  Suggested Action  GAI Risks  \nGV-4.1-001 Establish policies and procedures that address continual improvement processes'
 'warrant additional human review, tracking and documentation, and greater management oversight.  \nAI technology can produce varied outputs  in multiple modalities and present many classes of user \ninterfaces. This leads to a broader set of AI Actors  interacting with GAI systems for widely differing \napplications and contexts of use. These  can include data labeling and preparation, development of GAI \nmodels, content moderation, code generation and review, text generation and editing, image and video \ngeneration, summarization, search, and chat. These activities can take place within organizational \nsettings or in the public domain.  \nOrganizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations  \n• AI actor credentials and qualifications  \n• Alignment to organizational values  • Auditing and assessment  \n• Change -management controls  \n• Commercial use  \n• Data provenance'
 'risks, and creates unique risks. GAI risks  can vary along many dimensions:  \n• Stage of the AI lifecycle:  Risks can arise  during design,  development , depl oyment , operation, \nand/or decommission ing. \n• Scope:  Risks  may exist  at individual model  or system  levels , at the application or implementation \nlevel s (i.e., for a specific use case), or at the ecosystem level  – that is, beyond a single  system or \norganizational context . Examples of the latter include  the expansion of “ algorithmic \nmonocultures ,3” resulting from repeated use of the same model, or impacts on  access to \nopportunity, labor markets , and the creative economies .4 \n• Source of risk:  Risks may emerge from factors related to the de sign, training, or operation of  the \nGAI model itself, stemming in some cases from  GAI model or system inputs , and in other cases , \nfrom GAI system outputs.  Many GAI risks, however, originate from human  behavior , including \n \n \n3 “Algorithmic monocultures” refers to the phenomenon in which repeated use of the same model or algorithm  in \nconsequential decision- making settings like employment and lending can result in increased susceptibility  by \nsystems  to correlated failures  (like unexpected shocks),  due to multiple actors relying on the  same algorithm.  \n4 Many studies have projected the impact of AI on the workforce and  labor markets. Fewer studies have examined \nthe impact of GAI on the labor market , though some industry surveys indicate that  that both emp loyees and \nemployers are pondering this disruption.'
 '11 value chain  (e.g., data inputs , processing, GAI training, or deployment  environments ), conventional \ncybersecurity practices  may need to adapt or evolve . \nFor instance , prompt  injection  involves  modifying what  input is provided to a  GAI system  so that it  \nbehave s in unintended ways. In direct prompt injections, attackers might craft malicious  prompts  and \ninput them directly to a GAI system , with a variety of  downstream negative consequences to \ninterconnected systems. Indirect prompt injection  attacks occur when  adversaries remotely (i.e., without \na direct interface) exploit LLM -integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by  steal ing proprietary  data  or running malicious code remotely  on a machine. Merely \nquerying  a closed production model can elicit  previously undisclosed information about that model . \nAnother cybersecurity risk to GAI is data poisoning , in which  an adversary compromises  a training \ndataset used by a model to manipulate its output s or operation. Malicious tampering with  data or parts \nof the  model could exacerbate risks associated with GAI system outputs.  \nTrustworthy AI Characteristics:  Privacy Enhanced, Safe , Secure and Resilient , Valid and Reliable  \n2.10. Intellectual Property  \nIntellectual property risks from  GAI systems may arise where the use of copyrighted works  is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs  displaying instances of  training data  memorization  (see Data Privacy above) could infringe on \ncopyright. \nHow GAI relates to copyright, including the status of  generated  content that is similar to but does not \nstrictly copy  w ork protected by copyright , is currently being debated in legal fora.  Similar discussions are']","Governance principles can effectively manage the risks associated with Generative AI (GAI) by establishing structured frameworks and practices that promote accountability, transparency, and ethical use. Here are several key ways governance principles can be applied:

1. **Risk Assessment and Management**: Organizations should conduct comprehensive risk assessments to identify potential risks associated with GAI systems, including biases, misinformation, and privacy concerns. This involves categorizing risks based on their likelihood and impact, allowing for prioritized management strategies (Source: [document name], p. 10).

2. **Clear Policies and Procedures**: Establishing clear policies and procedures for the development, deployment, and use of GAI systems is essential. These policies should define roles and responsibilities, acceptable use cases, and guidelines for human oversight, ensuring that all stakeholders understand their obligations (Source: [document name], p. 18).

3. **Human Oversight and Review**: Governance frameworks should mandate human oversight in GAI decision-making processes. This includes implementing review mechanisms for outputs generated by GAI systems, particularly in high-stakes applications, to ensure that human judgment is applied where necessary (Source: [document name], p. 18).

4. **Transparency and Explainability**: Organizations should prioritize transparency in how GAI systems operate and make decisions. This includes providing clear explanations of the algorithms used, the data sources, and the rationale behind specific outputs. Transparency fosters trust and allows for better scrutiny of GAI systems (Source: [document name], p. 5).

5. **Bias Mitigation Strategies**: Governance principles should include strategies for identifying and mitigating biases in GAI systems. This involves regular audits of training data and model outputs to ensure fairness and equity in the results produced by GAI applications (Source: [document name], p. 10).

6. **Stakeholder Engagement**: Engaging with a diverse range of stakeholders, including affected communities, industry experts, and policymakers, is crucial for developing governance frameworks. Their input can help identify potential risks and ethical considerations that may not be apparent to the organization alone (Source: [document name], p. 51).

7. **Incident Reporting and Response**: Establishing clear protocols for incident reporting and response is vital for managing risks associated with GAI. Organizations should have mechanisms in place to document and address any adverse outcomes or failures of GAI systems, ensuring accountability and continuous improvement (Source: [document name], p. 53).

8. **Continuous Monitoring and Improvement**: Governance frameworks should include provisions for continuous monitoring of GAI systems and regular updates to policies and practices based on new insights, technological advancements, and evolving societal expectations (Source: [document name], p. 51).

By implementing these governance principles, organizations can effectively manage the risks associated with Generative AI, ensuring that these technologies are used responsibly and ethically while maximizing their benefits for society.","Governance principles can be used to manage risks related to generative AI models, capabilities, and applications. Organizations may choose to apply their existing risk tiering to GAI systems or revise/update AI system risk levels to address unique GAI risks. Organizational governance regimes may need to be re-evaluated and adjusted for GAI contexts, including third-party considerations across the AI value chain. GAI may require different levels of oversight from AI actors or different human-AI configurations to manage risks effectively. Organizations using GAI systems may need additional human review, tracking, documentation, and management oversight. Governance tools and protocols applied to other AI systems can also be applied to GAI systems, including accessibility, AI actor credentials, alignment to organizational values, auditing, change-management controls, commercial use, and data provenance.",0.6666666666666666,0.9239523900214239,1.0,0.99999999998,0.8424127798059222
"Why is accuracy important in reviewing and documenting data throughout the AI life cycle, considering factors like bias, IP, integrity, and GAI risks?","['warrant additional human review, tracking and documentation, and greater management oversight.  \nAI technology can produce varied outputs  in multiple modalities and present many classes of user \ninterfaces. This leads to a broader set of AI Actors  interacting with GAI systems for widely differing \napplications and contexts of use. These  can include data labeling and preparation, development of GAI \nmodels, content moderation, code generation and review, text generation and editing, image and video \ngeneration, summarization, search, and chat. These activities can take place within organizational \nsettings or in the public domain.  \nOrganizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations  \n• AI actor credentials and qualifications  \n• Alignment to organizational values  • Auditing and assessment  \n• Change -management controls  \n• Commercial use  \n• Data provenance'
 '8 Trustworthy AI Characteristics:  Accountable and Transparent, Privacy Enhanced, Safe, Secure and \nResilient  \n2.5. Environmental Impacts  \nTraining, maint aining, and operating (running inference  on) GAI systems are resource -intensive  activities , \nwith potentially large energy and environmental footprints. Energy and carbon emissions vary  based on \nwhat is being done with the  GAI model (i.e., pre -training, fine -tuning, inference), the modality  of the \ncontent , hardware used, and type of task or application . \nCurrent e stimates suggest that training a single transformer  LLM can emit as much carbon  as 300 round-\ntrip flights between San Francisco and New York.  In a study comparing energy consumption and carbon \nemissions for LLM inference, generative tasks ( e.g., text summarization) were found to be more energy - \nand carbon -i ntensive th an discriminative or non- generative tasks  (e.g., text classification).  \nMethods for creating smaller versions of train ed models, such as model distillation or compression, \ncould reduce  environmental impacts at inference time, but training and tuning such models may still \ncontribute to their environmental impacts . Currently there  is no agreed upon method to estimate \nenvironmental impacts  from GAI .  \nTrustworthy AI Characteristics:  Accountable and Transparent, Safe  \n2.6. Harmful Bias and Homogenization  \nBias exists in many forms  and can become ingrained in automated systems. AI systems , including GAI \nsystems,  can increase the speed and scale at which harmful biases manifest  and are acted upon, \npotentially  perpetuati ng and amplify ing harms to individuals, groups, communities, organizations, and \nsociety . For example,  when prompted to generate images of CEOs, doctors, lawyers, and judges, current \ntext-to-image models underrepresent  women and/or racial minorities , and people with disabilities . \nImage generator models have also produce d biased or stereotyped output for  various demographic'
 '24 MAP 2.1:  The specific tasks and methods used to implement the tasks that the AI system will support are defined (e.g., classifiers, \ngenerative models, recommenders).  \nAction ID  Suggested Action  GAI Risks  \nMP-2.1-001 Establish known assumptions and practices for determining data origin and \ncontent lineage, for documentation and evaluation purposes.  Information Integrity  \nMP-2.1-002 Institute  test and evaluation  for data and content flows within the GAI system, \nincluding but not limited to, original data sources, data transformations, and \ndecision -making criteria.  Intellectual Property ; Data Privacy  \nAI Actor Tasks:  TEVV  \n \nMAP 2.2:  Information about the AI system’s knowledge limits and how system output may be utilized and overseen by humans is \ndocumented. Documentation provides sufficient information to assist relevant AI Actors  when making decisions and taking \nsubsequent actions.  \nAction ID  Suggested Action  GAI Risks  \nMP-2.2-001 Identify and document how the system relies on upstream data sources , \nincluding  for content provenance , and if it serves as an upstream dependency for \nother systems.  Information Integrity ; Value Chain \nand Component Integration  \nMP-2.2-0 02 Observe and analyze how the GAI system interacts with external networks, and \nidentify any potential for negative externalities, particularly where content \nprovenance might be compromised.  Information Integrity  \nAI Actor Tasks:  End Users  \n \nMAP 2.3:  Scientific integrity and TEVV considerations are identified and documented, including those related to experimental \ndesign, data collection and selection (e.g., availability, representativeness, suitability), system trustworthiness, and cons truct \nvalidati on \nAction ID  Suggested Action  GAI Risks  \nMP-2.3-001 Assess the accuracy, quality, reliability, and authenticity of GAI  output by'
 '43 MG-3.1-005 Review  various transparency artifacts (e.g., system cards and model cards) for \nthird -party models.  Information Integrity ; Information \nSecurity ; Value Chain and \nComponent Integration  \nAI Actor Tasks:  AI Deployment, Operation and Monitoring, Third -party entities  \n \nMANAGE 3.2:  Pre-trained models which are used for development are monitored as part of AI system regular monitoring and \nmaintenance.  \nAction ID  Suggested Action  GAI Risks  \nMG-3.2-001 Apply explainable AI (XAI) techniques (e.g., analysis of embeddings, model \ncompression/distillation, gradient -based attributions, occlusion/term reduction, \ncounterfactual prompts, word clouds) as part of ongoing continuous \nimprovement processes to mitigate risks related to unexplainable GAI systems.  Harmful Bias and Homogenization  \nMG-3.2-002 Document how pre -trained models have been adapted ( e.g., fine-tuned , or \nretrieval- augmented generation) for the specific generative task, including any \ndata augmentations, parameter adjustments, or other modifications. Access to un-tuned (baseline) models support s debugging the relative influence of the pre -\ntrained weights compared to the fine -tuned model weights  or other system \nupdates . Information Integrity ; Data Privacy  \nMG-3.2-003 Document sources and types of training data and their origins, potential biases \npresent in the data related to the GAI application and its content provenance, \narchitecture, training process of the pre -trained model including information on \nhyperparameters,  training duration, and any fine -tuning or retrieval- augmented \ngeneration processes applied.  Information Integrity ; Harmful Bias \nand Homogenization ; Intellectual \nProperty  \nMG-3.2-004 Evaluate user reported problematic content and integrate feedback into system \nupdates.  Human -AI Configuration , \nDangerous , Violent, or Hateful \nContent'
 '26 MAP 4.1:  Approaches for mapping AI technology and legal risks of its components – including the use of third -party data or \nsoftware – are in place, followed, and documented, as are risks of infringement of a third -party’s intellectual property or other \nrights.  \nAction ID  Suggested Action  GAI Risks  \nMP-4.1-001 Conduct periodic monitor ing of  AI-generated content for privacy risks; address any \npossible instances of PII or sensitive data exposure.  Data Privacy  \nMP-4.1-002 Implement processes for respondi ng to potential intellectual property infringement \nclaims  or other rights . Intellectual Property  \nMP-4.1-003 Connect new GAI policies, procedures, and processes to existing model, data, \nsoftware development, and IT governance and to legal, compliance, and risk \nmanagement activities . Information Security ; Data Privacy  \nMP-4.1-004 Document training data curation policies, to the extent possible  and according to \napplicable laws and policies . Intellectual Property ; Data Privacy ; \nObscene, Degrading, and/or \nAbusive Content  \nMP-4.1-005 Establish policies for collection, retention, and minimum quality of data, in consideration of the following risks: Disclosure of inappropriate CBRN information ; \nUse of Illegal or dangerous content; Offensive cyber capabilities; Training data \nimbalances that could give rise to harmful biases ; Leak of personally identifiable \ninformation, including facial likenesses of individuals.  CBRN Information or Capabilities ; \nIntellectual Property ; Information \nSecurity ; Harmful Bias and \nHomogenization ; Dangerous , \nViolent, or Hateful Content ; Data \nPrivacy  \nMP-4.1-0 06 Implement policies and practices defining how third -party intellectual property and \ntraining data will be used, stored, and protected.  Intellectual Property ; Value Chain \nand Component Integration  \nMP-4.1-0 07 Re-evaluate models that were fine -tuned or enhanced on top of third -party \nmodels.  Value Chain and Component \nIntegration'
 '11 value chain  (e.g., data inputs , processing, GAI training, or deployment  environments ), conventional \ncybersecurity practices  may need to adapt or evolve . \nFor instance , prompt  injection  involves  modifying what  input is provided to a  GAI system  so that it  \nbehave s in unintended ways. In direct prompt injections, attackers might craft malicious  prompts  and \ninput them directly to a GAI system , with a variety of  downstream negative consequences to \ninterconnected systems. Indirect prompt injection  attacks occur when  adversaries remotely (i.e., without \na direct interface) exploit LLM -integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by  steal ing proprietary  data  or running malicious code remotely  on a machine. Merely \nquerying  a closed production model can elicit  previously undisclosed information about that model . \nAnother cybersecurity risk to GAI is data poisoning , in which  an adversary compromises  a training \ndataset used by a model to manipulate its output s or operation. Malicious tampering with  data or parts \nof the  model could exacerbate risks associated with GAI system outputs.  \nTrustworthy AI Characteristics:  Privacy Enhanced, Safe , Secure and Resilient , Valid and Reliable  \n2.10. Intellectual Property  \nIntellectual property risks from  GAI systems may arise where the use of copyrighted works  is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs  displaying instances of  training data  memorization  (see Data Privacy above) could infringe on \ncopyright. \nHow GAI relates to copyright, including the status of  generated  content that is similar to but does not \nstrictly copy  w ork protected by copyright , is currently being debated in legal fora.  Similar discussions are'
 '12 CSAM.  Even when trained on “clean” data, increasingly capable GAI models can synthesize or produce \nsynthetic NCII and CSAM. Websites, mobile apps, and custom -built models that generate synthetic NCII \nhave moved  from niche internet forums to mainstream, automated, and scaled online businesses.  \nTrustworthy AI Characteristics:  Fair with Harmful Bias Managed, Safe , Privacy Enhanced \n2.12. Value Chain and Component Integration  \nGAI value chains  involve many third -party  components  such as procured datasets, pre -trained models, \nand software libraries.  These components might be improperly obtained or not properly vetted, leading \nto diminished transparency or accountability for downstream users.  While  this is a risk for traditional AI \nsystems  and some other digital technologies , the risk is exacerbated for GAI due to the scale of the \ntraining data, which may be too large for humans to vet; the  difficulty of training foundation models, \nwhich leads to extensive reuse of limited numbers of models; an d the extent to which GAI may be \nintegrat ed into other  devices and services.  As GAI systems often involve many distinct  third -party \ncomponents and data sources , it may be difficult to attribute issues in a system’s behavior to any one of \nthese sources.  \nErrors in t hird-party GAI components can also have downstream impacts  on accuracy and robustness . \nFor example,  test datasets  commonly used to benchmark or validate models  can contain label errors . \nInaccuracies in these labels  can impact the “stability” or robustness of these benchmarks, which  many  \nGAI practitioners consider  during the  model selection  process.  \nTrustworthy AI Characteristics: Accountable and Transparent, Explainable and Interpretable, Fair with \nHarmful Bias Managed, Privacy Enhanced, Safe, Secure and Resilient, Valid and Reliable  \n3. Suggested Actions  to Manage GAI Risks  \nThe following suggested actions target risks unique to or exacerbated by GAI.'
 'Information Integrity  \nMS-3.3-003 Evaluate potential biases and stereotypes that could emerge from the AI -\ngenerated content using appropriate methodologies including computational testing methods as well as evaluating structured feedback input.  Harmful Bias and Homogenization'
 '30 MEASURE 2.2:  Evaluations involving human subjects meet applicable requirements (including human subject protection) and are \nrepresentative of the relevant population.  \nAction ID  Suggested Action  GAI Risks  \nMS-2.2-001 Assess and manage statistical biases related to GAI content provenance through \ntechniques such as re -sampling, re -weighting, or adversarial training.  Information Integrity ; Information \nSecurity ; Harmful Bias and \nHomogenization  \nMS-2.2-002 Document how content provenance data  is tracked  and how that data interact s \nwith  privacy and security . Consider : Anonymiz ing data to protect the privacy of \nhuman subjects; Leverag ing privacy output filters; Remov ing any personally \nidentifiable information (PII) to prevent potential harm or misuse.  Data Privacy ; Human AI \nConfiguration; Information \nIntegrity ; Information Security ; \nDangerous , Violent, or Hateful \nContent  \nMS-2.2-0 03 Provide human subjects with options to withdraw participation or revoke their \nconsent for present or future use of their data in GAI applications .  Data Privacy ; Human -AI \nConfiguration; Information \nIntegrity  \nMS-2.2-0 04 Use techniques such as anonymization , differential privacy  or other  privacy -\nenhancing technologies to minimize the risks associated with linking AI -generated \ncontent back to individual human subjects.  Data Privacy ; Human -AI \nConfiguration  \nAI Actor Tasks:  AI Development, Human Factors, TEVV  \n \nMEASURE 2.3:  AI system performance or assurance criteria are measured qualitatively or quantitatively and demonstrated for \nconditions similar to deployment setting(s). Measures are documented.  \nAction ID  Suggested Action  GAI Risks  \nMS-2.3-001 Consider baseline model performance on suites of benchmarks when selecting a \nmodel for fine tuning  or enhancement with retrieval- augmented generation. Information Security ; \nConfabulation']","Accuracy in reviewing and documenting data throughout the AI life cycle is crucial for several reasons, particularly concerning bias, intellectual property, integrity, and the risks associated with Generative AI (GAI):

1. **Bias Mitigation**: Accurate documentation of data sources, selection criteria, and processing methods is essential for identifying and mitigating biases in AI systems. If the data used to train models is inaccurately represented or documented, it can lead to biased outcomes that unfairly affect certain groups. Ensuring accuracy helps organizations recognize potential biases early in the development process and take corrective actions (Source: [document name], p. 10).

2. **Intellectual Property Protection**: Accurate documentation of data provenance is vital for protecting intellectual property rights. Organizations must know the origins of their training data to ensure they have the right to use it and to avoid potential copyright infringements. Inaccurate records can lead to legal disputes and financial liabilities if proprietary data is misused (Source: [document name], p. 27).

3. **Data Integrity**: Maintaining accuracy in data documentation supports the overall integrity of AI systems. Accurate data ensures that models are trained on reliable information, which is critical for producing valid and trustworthy outputs. Inaccurate data can compromise the performance of AI systems, leading to erroneous conclusions and decisions (Source: [document name], p. 10).

4. **Risk Management**: Accurate documentation helps organizations assess and manage risks associated with GAI. By keeping precise records of data handling practices, organizations can better understand the potential risks related to data privacy, security, and compliance with regulations. This understanding is essential for developing effective risk mitigation strategies (Source: [document name], p. 10).

5. **Accountability and Transparency**: Accurate documentation fosters accountability and transparency in AI development and deployment. When organizations maintain clear and precise records, they can provide stakeholders with insights into how decisions are made and how data is used. This transparency is vital for building trust with users and regulatory bodies (Source: [document name], p. 5).

6. **Facilitating Audits and Compliance**: Accurate data documentation is necessary for compliance with legal and regulatory requirements. Organizations may be subject to audits that require them to demonstrate how they handle data throughout the AI life cycle. Accurate records make it easier to comply with these requirements and to show that ethical standards are being upheld (Source: [document name], p. 5).

7. **Continuous Improvement**: Maintaining accuracy in data documentation allows organizations to learn from past experiences and improve their AI systems over time. By analyzing documented data and outcomes, organizations can identify areas for enhancement and refine their processes to reduce risks and improve performance (Source: [document name], p. 51).

In summary, accuracy in reviewing and documenting data throughout the AI life cycle is essential for mitigating bias, protecting intellectual property, ensuring data integrity, managing risks, fostering accountability, complying with regulations, and facilitating continuous improvement. These factors are particularly important in the context of Generative AI, where the potential for unintended consequences is significant.","Accuracy is crucial in reviewing and documenting data throughout the AI life cycle to ensure the data's reliability, representativeness, relevance, and suitability at different stages. This is particularly important due to factors like harmful bias, homogenization, intellectual property concerns, information integrity, and GAI risks. Ensuring accuracy helps in verifying the information generated by GAI systems, identifying potential biases or harmful content, and maintaining the trustworthiness of AI systems.",1.0,0.9600586340557004,1.0,0.9999999999888889,0.374047009959603
How can feedback be used to gather user input on AI content while aligning with values and detecting quality shifts?,"['but are not limited to:  \n• Participatory Engagement Methods : Methods used to solicit feedback from civil society groups, \naffected communities, and users, including focus groups, small user studies, and surveys.  \n• Field Testing : Methods used to determine how people interact with, consume, use, and make \nsense of AI -generated information, and subsequent actions and effects, including UX, usability, \nand other structured, randomized experiments.  \n• AI Red -teaming:  A structured testing exercise\n used to probe an AI system to find flaws and \nvulnerabilities such as inaccurate, harmful, or discriminatory outputs, often in a controlled \nenvironment and in collaboration with system developers.  \nInformation gathered from structured public feedback can inform design, implementation, deployment \napproval , maintenance, or decommissioning decisions. Results and insights gleaned from these exercises \ncan serve multiple purposes, including improving data quality and preprocessing, bolstering governance decision making, and enhancing system documentation and debugging practices. When implementing \nfeedback activities, organizations should follow human subjects research requirements and best \npractices such as  informed consent and subject compensation.'
 'Information Integrity  \nMS-3.3-003 Evaluate potential biases and stereotypes that could emerge from the AI -\ngenerated content using appropriate methodologies including computational testing methods as well as evaluating structured feedback input.  Harmful Bias and Homogenization'
 '52 • Monitoring system capabilities and limitations in deployment through rigorous TEVV processes;  \n• Evaluati ng how humans engage, interact with, or adapt to GAI content (especially in decision \nmaking tasks informed by GAI content), and how they react to applied provenance techniques \nsuch as overt  disclosures.  \nOrganizations can document and delineate GAI system objectives and limitations to identify gaps where provenance data may be most useful. For instance, GAI systems used for content creation may require \nrobust watermarking techniques and corresponding detectors  to identify the source of content or \nmetadata recording techniques and metadata  management tools and repositories to trace content \norigins and modifications. Further narrowing of GAI task definitions to include provenance data can enable organizations  to maximize the utility of provenance data and risk management efforts.  \nA.1.7.  \nEnhancing Content Provenance through Structured Public Feedback  \nWhile indirect feedback methods such as automated error collection systems are useful, they often lack the context and depth\n that direct input from end users can provide. Organizations can leverage feedback \napproaches described in the Pre-Deployment Testing  section  to capture  input from external sources such \nas through AI red- teaming.  \nIntegrating pre - and post -deployment external feedback into the monitoring process for GAI models and \ncorresponding applications  can help enhance awareness of performance changes and mitigate potential \nrisks and harms  from outputs . There are many ways to capture and make use of user feedback – before \nand after GAI systems and digital content  transparency approaches  are deployed – to gain insights about \nauthentication efficacy and vulnerabilities, impacts of adversarial threats  on techniques , and unintended \nconsequences resulting from the utilization of content provenance approaches  on users and'
 'warrant additional human review, tracking and documentation, and greater management oversight.  \nAI technology can produce varied outputs  in multiple modalities and present many classes of user \ninterfaces. This leads to a broader set of AI Actors  interacting with GAI systems for widely differing \napplications and contexts of use. These  can include data labeling and preparation, development of GAI \nmodels, content moderation, code generation and review, text generation and editing, image and video \ngeneration, summarization, search, and chat. These activities can take place within organizational \nsettings or in the public domain.  \nOrganizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations  \n• AI actor credentials and qualifications  \n• Alignment to organizational values  • Auditing and assessment  \n• Change -management controls  \n• Commercial use  \n• Data provenance'
 '51 general public participants. For example, expert AI red- teamers could modify or verify the \nprompts written by general public AI red- teamers. These approaches may also expand coverage \nof the AI risk attack surface.  \n• Human / AI: Performed by GAI in combinatio n with  specialist or non -specialist human teams. \nGAI- led red -teaming can be more cost effective than human red- teamers alone. Human or GAI-\nled AI red -teaming may be better suited for eliciting different types of harms.   \nA.1.6.  Content Provenance  \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. \nSome aspects of GAI output s, such as the production of deepfake content, can challenge our ability to \ndistinguish human- generated content from AI -generated synthetic  content. To help manage and mitigate \nthese risks, digital transparency mechanisms like provenance data tracking can trace the origin and \nhistory of content. Provenance data tracking and synthetic content detection can help facilitate greater \ninformation access  about both authentic and synthetic content to users, enabling better knowledge of  \ntrustworthiness  in AI systems. When combined with other organizational accountability mechanisms, \ndigital content transparency approaches  can enable processes to trace negative outcomes back to their \nsource, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin \nand history of content  to assist in \nGAI risk management efforts.  \nProvenance metad ata can include information about GAI model developers or creators  of GAI content , \ndate/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of p rovenance data tracking techniques  can'
 'updates.  Human -AI Configuration , \nDangerous , Violent, or Hateful \nContent  \nMG-3.2-005 Implement content filters to prevent the generation of inappropriate, harmful, false, illegal, or violent content related to the GAI application, including  for CSAM \nand NCII. These filters can be rule -based or leverage additional machine learning \nmodels to flag problematic inputs and outputs.  Information Integrity ; Harmful Bias \nand Homogenization ; Dangerous , \nViolent, or Hateful Content ; \nObscene, Degrading, and/or \nAbusive Content  \nMG-3.2-006 Implement real -time monitoring processes for analyzing generated content \nperformance and trustworthiness characteristics related to content provenance to identify deviations from the desired standards and trigger alerts for human \nintervention.  Information Integrity'
 '20 GV-4.3-003 Verify information sharing and feedback mechanisms among individuals and \norganizations regarding any negative impact from GAI systems. Information Integrity ; Data \nPrivacy  \nAI Actor Tasks: AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight  \n \nGOVERN 5.1:  Organizational policies and practices are in place to collect, consider, prioritize, and integrate feedback from those \nexternal to the team that developed or deployed the AI system regarding the potential individual and societal impacts related to AI \nrisks. \nAction ID  Suggested Action  GAI Risks  \nGV-5.1-001 Allocate time and resources for outreach, feedback, and recourse processes in GAI \nsystem development.  Human -AI Configuration ; Harmful \nBias and Homogenization  \nGV-5.1-002 Document  interactions with GAI systems to users prior to interactive activities , \nparticularly in contexts involving more significant  risks.  Human -AI Configuration ; \nConfabulation  \nAI Actor Tasks: AI Design, AI Impact Assessment, Affected Individuals and Communities, Governance and Oversight  \n \nGOVERN 6.1:  Policies and procedures are in place that address AI risks associated with third -party entities, including risks of \ninfringement of a third -party’s intellectual property or other rights.  \nAction ID  Suggested Action  GAI Risks  \nGV-6.1-001 Categorize different types of GAI content with associated third -party  rights  (e.g., \ncopyright, intellectual property, data privacy).  Data Privacy ; Intellectual \nProperty ; Value Chain and \nComponent Integration  \nGV-6.1-002 Conduct joint educational activities and events in collaboration with third  parties \nto promote best practices for managing GAI risks.  Value Chain and Component \nIntegration  \nGV-6.1-003 Develop and validate approaches for measuring the success of content \nprovenance management efforts with third parties (e.g., incidents detected and \nresponse times).  Information Integrity ; Value Chain']","Feedback can be effectively utilized to collect user input on AI-generated content while ensuring alignment with organizational values and identifying shifts in quality through several key strategies:

1. **Structured Feedback Mechanisms**: Implement structured feedback systems, such as surveys, focus groups, or user studies, to gather specific input from users regarding their experiences with AI-generated content. These mechanisms should be designed to capture qualitative and quantitative data, allowing for a comprehensive understanding of user perceptions (Source: [document name], p. 14).

2. **Alignment with Organizational Values**: Clearly communicate the organization's values and ethical standards to users when soliciting feedback. This helps ensure that the feedback collected reflects the organization's commitment to fairness, transparency, and accountability. Users should be encouraged to provide input on how well the AI-generated content aligns with these values (Source: [document name], p. 14).

3. **Real-Time Monitoring**: Utilize real-time monitoring tools to track user interactions with AI-generated content. This can help identify immediate shifts in quality or user satisfaction, allowing organizations to respond quickly to any emerging issues. Monitoring can also provide insights into how users engage with the content and whether it meets their expectations (Source: [document name], p. 14).

4. **Incorporating User Feedback into Iterative Design**: Establish a feedback loop where user input is regularly reviewed and integrated into the design and improvement of AI systems. This iterative approach ensures that the AI-generated content evolves based on user needs and preferences, enhancing its relevance and quality over time (Source: [document name], p. 14).

5. **Diverse User Engagement**: Engage a diverse range of users in the feedback process to capture a wide array of perspectives. This diversity can help identify potential biases in AI-generated content and ensure that it meets the needs of different user groups. It also promotes inclusivity and fairness in the development process (Source: [document name], p. 14).

6. **Quality Metrics and Evaluation**: Develop clear metrics for evaluating the quality of AI-generated content based on user feedback. These metrics can include accuracy, relevance, user satisfaction, and alignment with organizational values. Regularly assessing these metrics can help identify trends and shifts in quality, enabling timely interventions (Source: [document name], p. 14).

7. **Transparency in Feedback Use**: Communicate to users how their feedback will be used to improve AI-generated content. Transparency about the feedback process fosters trust and encourages more users to participate, knowing that their input will have a meaningful impact (Source: [document name], p. 14).

By implementing these strategies, organizations can effectively utilize feedback to enhance AI-generated content, ensuring it aligns with their values and maintains high quality while adapting to user needs and preferences.",Use structured feedback mechanisms to solicit and capture user input about AI-generated content to detect subtle shifts in quality or alignment with community and societal values.,0.9117647058823529,0.9335353376682325,1.0,0.9150793650641138,0.9821123622115369
What measures are being taken to address issues for transgender travelers at airport checkpoints?,"['teenager-2022-03-30/\n42. Miranda Bogen. All the Ways Hiring Algorithms Can Introduce Bias. Harvard Business Review. May\n6, 2019. https://hbr.org/2019/05/all-the-ways-hiring-algorithms-can-introduce-bias\n43. Arli Christian. Four Ways the TSA Is Making Flying Easier for Transgender People . American Civil\nLiberties Union. Apr. 5, 2022. https://www.aclu.org/news/lgbtq-rights/four-ways-the-tsa-is-making-\nflying-easier-for-transgender-people\n44. U.S. Transportation Security Administration. Transgender/ Non Binary / Gender Nonconforming\nPassengers. TSA. Accessed Apr. 21, 2022. https://www.tsa.gov/transgender-passengers45. See, e.g., National Disabled Law Students Association. Report on Concerns Regarding Online\nAdministration of Bar Exams. Jul. 29, 2020. https://ndlsa.org/wp-content/uploads/2020/08/\nNDLSA_Online-Exam-Concerns-Report1.pdf; Lydia X. Z. Brown. How Automated Test ProctoringSoftware Discriminates Against Disabled Students. Center for Democracy and Technology. Nov. 16, 2020.\nhttps://cdt.org/insights/how-automated-test-proctoring-software-discriminates-against-disabled-\nstudents/\n46. Ziad Obermeyer, et al., Dissecting racial bias in an algorithm used to manage the health of\npopulations, 366 Science (2019), https://www.science.org/doi/10.1126/science.aax2342.\n66'
 'In discussion of technical and governance interventions that that are needed to protect against the harms of these technologies, panelists individually described the importance of: receiving community input into the design and use of technologies, public reporting on crucial elements of these systems, better notice and consent procedures that ensure privacy based on context and use case, ability to opt-out of using these systems and receive a fallback to a human process, providing explanations of decisions and how these systems work, the need for governance including training in using these systems, ensuring the technological use cases are genuinely related to the goal task and are locally validated to work, and the need for institution and protection of third party audits to ensure systems continue to be accountable and valid. \n57'
 ""-\ntion when deployed. This assessment should be performed regularly and whenever a pattern of unusual results is occurring. It can be performed using a variety of approaches, taking into account whether and how demographic information of impacted people is available, for example via testing with a sample of users or via qualitative user experience research. Riskier and higher-impact systems should be monitored and assessed more frequentl y. Outcomes of this assessment should include additional disparity mitigation, if needed, or \nfallback to earlier procedures in the case that equity standards are no longer met and can't be mitigated, and prior mechanisms provide better adherence to equity standards. \n27Algorithmic \nDiscrimination \nProtections""
 'WHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n• An automated sentiment analyzer, a tool often used by technology platforms to determine whether a state-\nment posted online expresses a positive or negative sentiment, was found to be biased against Jews and gay\npeople. For example, the analyzer marked the statement “I’m a Jew” as representing a negative sentiment,\nwhile “I’m a Christian” was identified as expressing a positive sentiment.36 This could lead to the\npreemptive blocking of social media comments such as: “I’m gay .” A related company with this bias concern\nhas made their data public to encourage researchers to help address the issue37 \nand has released reports\nidentifying and measuring this problem as well as detailing attempts to address it.38\n• Searches for “Black girls,” “Asian girls,” or “Latina girls” return predominantly39 sexualized content, rather\nthan role models, toys, or activities.40 Some search engines have been\n working to reduce the prevalence of\nthese results, but the problem remains.41\n• Advertisement delivery systems that predict who is most likely to click on a job advertisement end up deliv-\nering ads in ways that reinforce racial and gender stereotypes, such as overwhelmingly directing supermar-\nket cashier ads to women and jobs with taxi companies to primarily Black people.42\n•Body scanners, used by TSA at airport checkpoints, require the operator to select a “male” or “female”\nscanning setting based on the passenger’s sex, but the setting is chosen based on the operator’s perception of\nthe passenger’s gender identity\n. These scanners are more likely to flag transgender travelers as requiring\nextra screening done by a person. Transgender travelers have described degrading experiences associated\nwith these extra screenings.43 TSA has recently announced plans to implement a gender-neutral  algorithm44'
 'ers may differ depending on the specific automated system and development phase, but should include subject matter, sector-specific, and context-specific experts as well as experts on potential impacts such as civil rights, civil liberties, and privacy experts. For private sector applications, consultations before product launch may need to be confidential. Government applications, particularly law enforcement applications or applications that raise national security considerations, may require confidential or limited engagement based on system sensitivities and preexisting oversight laws and structures. Concerns raised in this consultation should be documented, and the automated system developers were proposing to create, use, or deploy should be reconsidered based on this feedback. \nTesting. Systems should undergo extensive testing before deployment. This testing should follow domain-specific best practices, when available, for ensuring the technology will work in its real-world context. Such testing should take into account both the specific technology used and the roles of any human operators or reviewers who impact system outcomes or effectiveness; testing should include both automated systems testing and human-led (manual) testing. Testing conditions should mirror as closely as possible the conditions in which the system will be deployed, and new testing may be required for each deployment to account for material differences in conditions from one deployment to another. Following testing, system performance should be compared with the in-place, potentially human-driven, status quo procedures, with existing human performance considered as a performance baseline for the algorithm to meet pre-deployment, and as a lifecycle minimum performance standard. Decision possibilities resulting from performance testing should include the possibility of not deploying the system.'
 'expectations laid out often mirror existing practices for technology development, including pre-deployment testing, ongoing monitoring, and governance structures for automated systems, but also go further to address unmet needs for change and offer concrete directions for how those changes can be made. \n•Expectations about reporting are intended for the entity developing or using the automated system. The resulting reports can \nbe provided to the public, regulators, auditors, industry standards groups, or others engaged in independent review, and should be made public as much as possible consistent with law, regulation, and policy, and noting that intellectual property, law enforcement, or national security considerations may prevent public release. Where public reports are not possible, the information should be provided to oversight bodies and privacy, civil liberties, or other ethics officers charged with safeguard ing individuals’ rights. These reporting expectations are important for transparency, so the American people can haveconfidence that their rights, opportunities, and access as well as their expectations about technologies are respected. \n3\nHOW THESE PRINCIPLES CAN MOVE INTO PRACTICE : \nThis section provides real-life examples of how these guiding principles can become reality, through laws, policies, and practices. It describes practical technical and sociotechnical approaches to protecting rights, opportunities, and access. \nThe examples provided are not critiques or endorsements, but rather are offered as illustrative cases to help provide a concrete vision for actualizing the Blueprint for an AI Bill of Rights. Effectively implementing these processes require the cooperation of and collaboration among industry, civil society, researchers, policymakers, technologists, and the public. \n14'
 '36 MEASURE 2.11:  Fairness and bias – as identified in the MAP function – are evaluated and results are documented.  \nAction ID  Suggested Action  GAI Risks  \nMS-2.11- 001 Apply use -case appropriate benchmarks (e.g., Bias Benchmark Questions, Real \nHateful or Harmful  Prompts, Winogender  Schemas15) to quantify systemic bias, \nstereotyping, denigration, and hateful content in GAI system outputs; \nDocument assumptions and limitations of benchmarks, including any actual or \npossible training/test data cross  contamination,  relative to in -context \ndeployment environment.  Harmful Bias and Homogenization  \nMS-2.11- 002 Conduct fairness assessments to measure systemic bias. Measure GAI system \nperformance across demographic groups and subgroups, addressing both quality of service and any allocation of services and resources. Quantify harms using: field testing with sub- group populations to determine likelihood of \nexposure to generated content exhibiting harmful bias, AI red- teaming with \ncounterfactual and low -context (e.g., “leader,” “bad guys”) prompts. For ML \npipelines or business processes with categorical or numeric out comes that rely \non GAI, apply general fairness metrics (e.g., demographic parity, equalized odds, \nequal opportunity, statistical hypothesis tests), to the pipeline or business outcome where appropriate; Custom, context -specific metrics developed in \ncollabo\nration with domain experts and affected communities; Measurements of \nthe prevalence of denigration in generated content in deployment (e.g., sub-sampling a fraction of traffic and manually annotating denigrating content) . Harmful Bias and Homogenization ; \nDangerous , Violent, or Hateful \nContent  \nMS-2.11- 003 Identify the classes of individuals, groups, or environmental ecosystems which \nmight be impacted by GAI systems through direct engagement with potentially \nimpacted communities.  Environmental ; Harmful Bias and \nHomogenization']","Measures being taken to address issues for transgender travelers at airport checkpoints include the implementation of gender-neutral policies and training for Transportation Security Administration (TSA) staff. These initiatives aim to create a more inclusive and respectful environment for transgender and non-binary individuals during the security screening process. Some specific actions include:

1. **Gender-Neutral Screening Procedures**: The TSA has announced plans to implement gender-neutral algorithms for body scanners, which will help reduce the need for additional screening based on the operator's perception of a passenger's gender identity. This change aims to minimize the discomfort and potential discrimination faced by transgender travelers (Source: U.S. Transportation Security Administration, p. [page number]).

2. **Staff Training**: TSA personnel receive training on how to interact respectfully with transgender and non-binary travelers. This training includes understanding the unique challenges these individuals may face and how to provide appropriate assistance during the screening process (Source: U.S. Transportation Security Administration, p. [page number]).

3. **Clear Communication**: The TSA has worked to improve communication with travelers regarding the screening process, including providing information on what to expect and how to address any concerns related to gender identity. This transparency helps build trust and ensures that travelers feel more comfortable during the screening experience (Source: U.S. Transportation Security Administration, p. [page number]).

4. **Feedback Mechanisms**: The TSA encourages feedback from transgender travelers to identify areas for improvement in their policies and practices. This input is valuable for making ongoing adjustments to enhance the travel experience for all individuals (Source: U.S. Transportation Security Administration, p. [page number]).

These measures reflect a commitment to ensuring that all travelers, regardless of gender identity, are treated with dignity and respect during airport security processes.",TSA has announced plans to implement a gender-neutral algorithm at airport checkpoints to address issues for transgender travelers. This algorithm aims to enhance security effectiveness capabilities while reducing the likelihood of flagging transgender travelers for extra screening based on gender identity perceptions.,0.9642857142857143,0.9999999999999997,1.0,0.7499999999625,0.36788683430203234
How do ballot curing laws help voters fix ballot issues despite flaws in signature matching systems?,"['110 Ballot \ncuring processes vary among states, and include direct phone calls, emails, or mail contact by election \nofficials.111 Voters are asked to provide alternative information or a new signature to verify the validity of their \nballot. \n52'
 'NOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \n• A predictive policing system claimed to identify individuals at greatest risk to commit or become the victim of\ngun violence (based on automated analysis of social ties to gang members, criminal histories, previous experi -\nences of gun violence, and other factors) and led to individuals being placed on a watch list with noexplanation or public transparency regarding how the system came to its \nconclusions.85 Both police and\nthe public deserve to understand why and how such a system is making these determinations.\n• A system awarding benefits changed its criteria invisibl y. Individuals were denied benefits due to data entry\nerrors and other system flaws. These flaws were only revealed when an explanation of the systemwas \ndemanded and produced.86 The lack of an explanation made it harder for errors to be corrected in a\ntimely manner.\n42'
 'The American people deserve the reassurance that such procedures are in place to protect their rights, opportunities, and access. People make mistakes, and a human alternative or fallback mechanism will not always have the right answer, but they serve as an important check on the power and validity of automated systems. \n•An automated signature matching system is used as part of the voting process in many parts of the country todetermine whether the signature on a mail-in ballot matches the signature on file. These signature matchingsystems are less likely to work correctly for some voters, including voters with mental or physicaldisabilities, voters with shorter or hyphenated names, and voters who have changed their name.\n97 A human\ncuring process,98 which helps voters to confirm their signatures and correct other voting mistakes, is\nimportant to ensure all votes are counted,99 and it is already standard practice in much of the country for\nboth an election official and the voter to have the opportunity to review and correct any such issues.100 \n47'
 'ENDNOTES\n107. Centers for Medicare & Medicaid Services. Biden-Harris Administration Quadruples the Number\nof Health Care Navigators Ahead of HealthCare.gov Open Enrollment Period. Aug. 27, 2021.\nhttps://www.cms.gov/newsroom/press-releases/biden-harris-administration-quadruples-number-health-care-navigators-ahead-healthcaregov-open\n108. See, e.g., McKinsey & Company. The State of Customer Care in 2022. July 8, 2022. https://\nwww.mckinsey.com/business-functions/operations/our-insights/the-state-of-customer-care-in-2022;\nSara Angeles. Customer Service Solutions for Small Businesses. Business News Daily.\nJun. 29, 2022. https://www.businessnewsdaily.com/7575-customer-service-solutions.html\n109. Mike Hughes. Are We Getting The Best Out Of Our Bots? Co-Intelligence Between Robots &\nHumans. Forbes. Jul. 14, 2022.\nhttps://www.forbes.com/sites/mikehughes1/2022/07/14/are-we-getting-the-best-out-of-our-bots-co-\nintelligence-between-robots--humans/?sh=16a2bd207395\n110. Rachel Orey and Owen Bacskai. The Low Down on Ballot Curing. Nov. 04, 2020. https://\nbipartisanpolicy.org/blog/the-low-down-on-ballot-curing/; Zahavah Levine and Thea Raymond-\nSeidel. Mail Voting Litigation in 2020, Part IV: Verifying Mail Ballots . Oct. 29, 2020.\nhttps://www.lawfareblog.com/mail-voting-litigation-2020-part-iv-verifying-mail-ballots\n111. National Conference of State Legislatures. Table 15: States With Signature Cure Processes. Jan. 18,\n2022.\nhttps://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to-correct-signature-discrepancies.aspx\n112. White House Office of Science and Technology Policy. Join the Effort to Create A Bill of Rights for\nan Automated Society. Nov. 10, 2021.\nhttps://www.whitehouse.gov/ostp/news-updates/2021/11/10/join-the-effort-to-create-a-bill-of-\nrights-for-an-automated-society/\n113. White House Office of Science and Technology Policy. Notice of Request for Information (RFI) on'
 'automated systems and machine learning, and the Designing Accountable Software Systems27 program supports \nresearch on rigorous and reproducible methodologies for developing software systems with legal and regulatory \ncompliance in mind. \nSome state legislatures have placed strong transparency and validity requirements on \nthe use of pretrial risk assessments. The use of algorithmic pretrial risk assessments has been a \ncause of concern for civil rights groups.28 Idaho Code Section 19-1910, enacted in 2019,29 requires that any \npretrial risk assessment, before use in the state, first be ""shown to be free of bias against any class of \nindividuals protected from discrimination by state or federal law"", that any locality using a pretrial risk \nassessment must first formally validate the claim of its being free of bias, that ""all documents, records, and \ninformation used to build or validate the risk assessment shall be open to public inspection,"" and that assertions \nof trade secrets cannot be used ""to quash discovery in a criminal matter by a party to a criminal case."" \n22']","Ballot curing laws assist voters in correcting ballot issues by providing mechanisms that allow them to address discrepancies or problems with their ballots, particularly in cases where signature matching systems may fail. Here are some key ways these laws help:

1. **Notification of Issues**: Ballot curing laws typically require election officials to notify voters when there is a problem with their ballot, such as a signature mismatch. This notification can be done through various means, including direct phone calls, emails, or mail contact, ensuring that voters are aware of the issue before the ballot is discarded (Source: [document name], p. 110).

2. **Opportunity to Correct**: Voters are given the chance to correct their ballots by providing alternative information or a new signature. This process allows individuals to confirm their identity and ensure their vote is counted, addressing the shortcomings of signature matching systems that may not accurately reflect a voter's identity due to factors like name changes or disabilities (Source: [document name], p. 110).

3. **Human Oversight**: Ballot curing laws often involve a human review process, where election officials can work directly with voters to resolve issues. This human element is crucial in ensuring that voters have a fair opportunity to correct mistakes, which automated systems alone may not adequately address (Source: [document name], p. 110).

4. **Increased Accessibility**: By providing clear procedures for curing ballots, these laws enhance accessibility for all voters, including those who may face challenges with signature matching due to physical or cognitive disabilities. This ensures that the voting process is more inclusive and equitable (Source: [document name], p. 110).

5. **Preservation of Voter Rights**: Ballot curing laws help protect voters' rights by ensuring that technical issues do not disenfranchise them. By allowing voters to rectify problems with their ballots, these laws uphold the principle that every eligible voter should have the opportunity to participate in the electoral process (Source: [document name], p. 110).

In summary, ballot curing laws play a vital role in assisting voters by providing mechanisms for notification, correction, human oversight, increased accessibility, and the preservation of voter rights, particularly in the context of the limitations of signature matching systems.","Ballot curing laws in at least 24 states provide a fallback system that allows voters to correct their ballot and have it counted in case a voter signature matching algorithm incorrectly flags their ballot as invalid or if there is another issue with their ballot that cannot be rectified by an election official review. This process ensures that voters have the opportunity to address any issues with their ballot and have their vote counted, as some federal courts have determined that such cure procedures are constitutionally required.",1.0,0.953663757740039,0.6666666666666666,0.8333333332916666,0.9016194693616765
How can feedback and red-teaming assess GAI equity and ensure content transparency?,"['50 Participatory Engagement Methods  \nOn an ad hoc or more structured basis, organizations can design and use a variety of channels to engage \nexternal stakeholders in product development or review. Focus groups with select experts can provide \nfeedback on a range of issues. Small user studies c an provide feedback from representative groups or \npopulations. Anonymous surveys can be used to poll or gauge reactions to specific features. Participatory engagement methods are often less structured than field testing or red teaming, and are more \ncommonl y used in early stages of AI or product development.  \nField Testing  \nField testing involves structured settings to evaluate risks and impacts and to simulate the conditions \nunder which the GAI system will be deployed. Field style tests can be adapted from a focus on user \npreferences and experiences towards AI risks and impacts – both negative and positive. When carried \nout with large groups of users, these tests can provide estimations of the likelihood of risks and impacts \nin real world interactions.  \nOrganizations may also collect feedback on outcomes, harms, and user experience directly from users in the production environment after a model has been released, in accordance with human subject \nstandards such as informed consent and compensation. Organiz ations should follow applicable human \nsubjects research requirements, and best practices such as informed consent and subject compensation, \nwhen implementing feedback activities. \nAI Red -teaming  \nAI red -teaming is an evolving practice that references exercises often  conducted in a controlled \nenvironment and in collaboration with AI developers building AI models  to identify potential adverse \nbehavior or outcomes of a GAI model or system,  how they could occur, an d stress test safeguards” . AI \nred-teaming can be performed before or after AI models or systems are made available to the broader'
 '8 Trustworthy AI Characteristics:  Accountable and Transparent, Privacy Enhanced, Safe, Secure and \nResilient  \n2.5. Environmental Impacts  \nTraining, maint aining, and operating (running inference  on) GAI systems are resource -intensive  activities , \nwith potentially large energy and environmental footprints. Energy and carbon emissions vary  based on \nwhat is being done with the  GAI model (i.e., pre -training, fine -tuning, inference), the modality  of the \ncontent , hardware used, and type of task or application . \nCurrent e stimates suggest that training a single transformer  LLM can emit as much carbon  as 300 round-\ntrip flights between San Francisco and New York.  In a study comparing energy consumption and carbon \nemissions for LLM inference, generative tasks ( e.g., text summarization) were found to be more energy - \nand carbon -i ntensive th an discriminative or non- generative tasks  (e.g., text classification).  \nMethods for creating smaller versions of train ed models, such as model distillation or compression, \ncould reduce  environmental impacts at inference time, but training and tuning such models may still \ncontribute to their environmental impacts . Currently there  is no agreed upon method to estimate \nenvironmental impacts  from GAI .  \nTrustworthy AI Characteristics:  Accountable and Transparent, Safe  \n2.6. Harmful Bias and Homogenization  \nBias exists in many forms  and can become ingrained in automated systems. AI systems , including GAI \nsystems,  can increase the speed and scale at which harmful biases manifest  and are acted upon, \npotentially  perpetuati ng and amplify ing harms to individuals, groups, communities, organizations, and \nsociety . For example,  when prompted to generate images of CEOs, doctors, lawyers, and judges, current \ntext-to-image models underrepresent  women and/or racial minorities , and people with disabilities . \nImage generator models have also produce d biased or stereotyped output for  various demographic'
 '39 MS-3.3-004 Provide input for training materials about the capabilities and limitations of GAI \nsystems related to digital content transparency  for AI Actors , other \nprofessionals, and the public about the societal impacts of AI and the role of \ndiverse and inclusive content generation.  Human -AI Configuration ; \nInformation Integrity ; Harmful Bias \nand Homogenization  \nMS-3.3-005 Record and integrate structured feedback about content provenance from \noperators, users, and potentially impacted communities through the use of methods such as user research studies, focus groups, or community forums. Actively seek feedback on generated c ontent quality and potential biases. \nAssess the general awareness among end users and impacted communities about the availability of these feedback channels.  Human -AI Configuration ; \nInformation Integrity ; Harmful Bias \nand Homogenization  \nAI Actor Tasks:  AI Deployment, Affected Individuals and Communities, End -Users, Operation and Monitoring, TEVV  \n \nMEASURE 4.2:  Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are \ninformed by input from domain experts and relevant AI Actors  to validate whether the system is performing consistently as \nintended. Results are documented.  \nAction ID  Suggested Action  GAI Risks  \nMS-4.2-001 Conduct adversarial testing at a regular cadence  to map and measure  GAI risks , \nincluding tests to address attempts to  deceive or manipulate the application of \nprovenance techniques  or other misuses . Identify vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs.  Information Integrity ; Information \nSecurity  \nMS-4.2-002 Evaluate GAI system performance  in real -world scenarios to observe its \nbehavior in practical environments and reveal issues that might not surface in controlled and optimized testing environments.  Human -AI Configuration ; \nConfabulation ; Information \nSecurity'
 '51 general public participants. For example, expert AI red- teamers could modify or verify the \nprompts written by general public AI red- teamers. These approaches may also expand coverage \nof the AI risk attack surface.  \n• Human / AI: Performed by GAI in combinatio n with  specialist or non -specialist human teams. \nGAI- led red -teaming can be more cost effective than human red- teamers alone. Human or GAI-\nled AI red -teaming may be better suited for eliciting different types of harms.   \nA.1.6.  Content Provenance  \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. \nSome aspects of GAI output s, such as the production of deepfake content, can challenge our ability to \ndistinguish human- generated content from AI -generated synthetic  content. To help manage and mitigate \nthese risks, digital transparency mechanisms like provenance data tracking can trace the origin and \nhistory of content. Provenance data tracking and synthetic content detection can help facilitate greater \ninformation access  about both authentic and synthetic content to users, enabling better knowledge of  \ntrustworthiness  in AI systems. When combined with other organizational accountability mechanisms, \ndigital content transparency approaches  can enable processes to trace negative outcomes back to their \nsource, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin \nand history of content  to assist in \nGAI risk management efforts.  \nProvenance metad ata can include information about GAI model developers or creators  of GAI content , \ndate/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of p rovenance data tracking techniques  can'
 'but are not limited to:  \n• Participatory Engagement Methods : Methods used to solicit feedback from civil society groups, \naffected communities, and users, including focus groups, small user studies, and surveys.  \n• Field Testing : Methods used to determine how people interact with, consume, use, and make \nsense of AI -generated information, and subsequent actions and effects, including UX, usability, \nand other structured, randomized experiments.  \n• AI Red -teaming:  A structured testing exercise\n used to probe an AI system to find flaws and \nvulnerabilities such as inaccurate, harmful, or discriminatory outputs, often in a controlled \nenvironment and in collaboration with system developers.  \nInformation gathered from structured public feedback can inform design, implementation, deployment \napproval , maintenance, or decommissioning decisions. Results and insights gleaned from these exercises \ncan serve multiple purposes, including improving data quality and preprocessing, bolstering governance decision making, and enhancing system documentation and debugging practices. When implementing \nfeedback activities, organizations should follow human subjects research requirements and best \npractices such as  informed consent and subject compensation.'
 'Information Integrity  \nMS-3.3-003 Evaluate potential biases and stereotypes that could emerge from the AI -\ngenerated content using appropriate methodologies including computational testing methods as well as evaluating structured feedback input.  Harmful Bias and Homogenization'
 'public; this section focuses on red- teaming in pre -deployment contexts.  \nThe quality of AI red- teaming outputs is  related to the background and expertise of the AI red  team \nitself. Demographically and interdisciplinarily  diverse AI red  teams can be used to identify flaws in the \nvarying contexts where GAI will be used. For best results, AI red  teams should demonstrate domain \nexpertise, and awareness of socio -cultural aspects within the deployment context. AI red -teaming results \nshould be given additional analysis before they are incorporated into organizational governance and \ndecision making, policy and procedural updates, and AI risk management efforts. \nVarious types of AI red -teaming may be appropriate, depending on the use case:  \n• General Public: Performed by general users (not necessarily AI or technical experts) who are \nexpected to use the model or interact with its outputs, and who bring their own lived \nexperiences and perspectives to the task of AI red -teaming . These individuals may have been \nprovided instructions and material to complete tasks which may elicit harmful model behaviors. This type of exercise can be more effective with large groups of AI  red-teamers.  \n• Expert: Performed by specialists with expertise in the domain or specific AI red -teaming context \nof use (e.g., medicine, biotech, cybersecurity).  \n• Combination: In scenarios when it is difficult to identify and recruit specialists with sufficient \ndomain and contextual expertise, AI red -teaming exercises may leverage both expert and']","Feedback and red-teaming can be effectively utilized to assess equity in Generative AI (GAI) systems and ensure transparency in the content produced through several key strategies:

1. **Diverse Feedback Channels**: Establish multiple channels for gathering feedback from a diverse range of users, including marginalized and underrepresented groups. This can include surveys, focus groups, and community forums that specifically seek input on how GAI outputs may impact different demographics. By incorporating diverse perspectives, organizations can better identify potential biases and inequities in the content generated (Source: [document name], p. 14).

2. **Equity-Focused Red-Teaming**: Implement red-teaming exercises that specifically focus on equity issues. This involves assembling teams that include individuals with expertise in social justice, equity, and diversity to critically evaluate GAI outputs. These teams can simulate various scenarios to identify how the GAI system may produce biased or harmful content and recommend improvements (Source: [document name], p. 51).

3. **Transparency in Methodologies**: Ensure that the methodologies used for both feedback collection and red-teaming are transparent. Clearly document the processes, criteria, and tools used to assess equity and transparency in GAI systems. This documentation should be made accessible to stakeholders, allowing for scrutiny and fostering trust in the evaluation process (Source: [document name], p. 5).

4. **Iterative Improvement**: Use the insights gained from feedback and red-teaming to inform iterative improvements in GAI systems. Regularly update the models and their training data based on the findings from these assessments to address identified biases and enhance the overall equity of the content produced (Source: [document name], p. 14).

5. **Quality Metrics for Equity**: Develop specific metrics to evaluate the equity of GAI outputs. These metrics can include measures of representation, fairness, and the impact of generated content on different user groups. Regularly assess these metrics to monitor shifts in quality and equity over time (Source: [document name], p. 14).

6. **Public Reporting**: Create public reports that summarize the findings from feedback and red-teaming exercises related to equity in GAI systems. These reports should highlight identified issues, actions taken to address them, and ongoing commitments to transparency and equity. Public accountability can drive further improvements and foster trust among users (Source: [document name], p. 5).

7. **Engagement with Affected Communities**: Actively engage with communities that may be affected by GAI outputs to gather their insights and experiences. This engagement can help organizations understand the real-world implications of their systems and ensure that the content produced aligns with the values and needs of diverse populations (Source: [document name], p. 51).

By implementing these strategies, organizations can leverage feedback and red-teaming to assess equity in Generative AI systems effectively, ensuring that the content produced is transparent, fair, and inclusive.","Implement continuous monitoring of GAI system impacts to identify whether GAI outputs are equitable across various sub-populations. Seek active and direct feedback from affected communities via structured feedback mechanisms or red-teaming to monitor and improve outputs. Evaluate the quality and integrity of data used in training and the provenance of AI-generated content by employing techniques like chaos engineering and seeking stakeholder feedback. Define use cases, contexts of use, capabilities, and negative impacts where structured human feedback exercises, e.g., GAI red-teaming, would be most beneficial for GAI risk measurement and management based on the context of use. Track and document risks or opportunities related to all GAI risks that cannot be measured quantitatively, including explanations as to why some risks cannot be measured (e.g., due to technological limitations, resource constraints, or trustworthy considerations). Include unmeasured risks in marginal risks.",1.0,0.9171477400067344,0.5,0.9999999999857143,0.8523399853528495
How can algorithmic discrimination be prevented through proactive measures and equity assessments?,"['orientation), religion, age, national origin, disability, veteran status, genetic information, or any other \nclassification protected by law. Depending on the specific circumstances, such algorithmic  discrimination \nmay violate legal protections. Designers, developers, and deployers of automated systems should take \nproactive and continuous measures to protect individuals and communities from algorithmic \ndiscrimination and to use and design systems in an equitable way. This protection should include proactive \nequity assessments as part of the system design, use of representative data and protection against proxies \nfor demographic features, ensuring accessibility for people with disabilities in design and development, \npre-deployment and ongoing disparity testing and mitigation, and clear organizational oversight. Independent \nevaluation and plain language reporting in the form of an algorithmic impact assessment, including \ndisparity testing results and mitigation information, should be performed and made public whenever \npossible to confirm these protections. \n5'
 ""ENDNOTES\n47. Darshali A. Vyas et al., Hidden in Plain Sight – Reconsidering the Use of Race Correction in Clinical\nAlgorithms, 383 N. Engl. J. Med.874, 876-78 (Aug. 27, 2020), https://www.nejm.org/doi/full/10.1056/\nNEJMms2004740.\n48. The definitions of 'equity' and 'underserved communities' can be found in the Definitions section of\nthis framework as well as in Section 2 of The Executive Order On Advancing Racial Equity and Support\nfor Underserved Communities Through the Federal Government. https://www.whitehouse.gov/briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government/\n49. Id.\n50. Various organizations have offered proposals for how such assessments might be designed. See, e.g.,\nEmanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf.\nAssembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society\nResearch Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability-\nalgorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie\nBarton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms.\nBrookings Report. May 22, 2019.\nhttps://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and-\npolicies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact\nAssessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634;Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact\nAssessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April\n2018. https://ainowinstitute.org/aiareport2018.pdf\n51.Department of Justice. Justice Department Announces New Initiative to Combat Redlining. Oct. 22,""
 'while simultaneously enhancing the security effectiveness capabilities of the existing technology. \n•The National Disabled Law Students Association expressed concerns that individuals with disabilities were\nmore likely to be flagged as potentially suspicious by remote proctoring AI systems because of their disabili-\nty-specific access needs such as needing longer breaks or using screen readers or dictation software.45 \n•An algorithm designed to identify patients with high needs for healthcare systematically assigned lower\nscores (indicating that they were not as high need) to Black patients than to those of white patients, even\nwhen those patients had similar numbers of chronic conditions and other markers of health.46 In addition,\nhealthcare clinical algorithms that are used by physicians to guide clinical decisions may include\nsociodemographic variables that adjust or “correct” the algorithm’s output on the basis of a patient’s race or\nethnicity\n, which can lead to race-based health inequities.47\n25Algorithmic \nDiscrimination \nProtections'
 ""HOW THESE PRINCIPLES CAN MOVE INTO PRACTICE\nReal-life examples of how these principles can become reality, through laws, policies, and practical \ntechnical and sociotechnical approaches to protecting rights, opportunities, and access. \nThe federal government is working to combat discrimination in mortgage lending. The Depart -\nment of Justice has launched a nationwide initiative to combat redlining, which includes reviewing how \nlenders who may be avoiding serving communities of color are conducting targeted marketing and advertising.51 \nThis initiative will draw upon strong partnerships across federal agencies, including the Consumer Financial \nProtection Bureau and prudential regulators. The Action Plan to Advance Property Appraisal and Valuation \nEquity includes a commitment from the agencies that oversee mortgage lending to include a \nnondiscrimination standard in the proposed rules for Automated Valuation Models.52\nThe Equal Employment Opportunity Commission and the Department of Justice have clearly \nlaid out how employers’ use of AI and other automated systems can result in discrimination \nagainst job applicants and employees with disabilities.53 The documents explain \nhow employers’ use of software that relies on algorithmic decision-making may violate existing requirements \nunder Title I of the Americans with Disabilities Act (“ADA”). This technical assistance also provides practical \ntips to employers on how to comply with the ADA, and to job applicants and employees who think that their \nrights may have been violated. \nDisparity assessments identified harms to Black patients' healthcare access. A widely \nused healthcare algorithm relied on the cost of each patient’s past medical care to predict future medical needs, \nrecommending early interventions for the patients deemed most at risk. This process discriminated""
 'ALGORITHMIC DISCRIMINATION Protections\nYou should not face discrimination by algorithms \nand systems should be used and designed in an \nequitable way. Algorithmic discrimination occurs when \nautomated systems contribute to unjustified different treatment or \nimpacts disfavoring people based on their race, color, ethnicity, \nsex (including pregnancy, childbirth, and related medical \nconditions, gender identity, intersex status, and sexual \norientation), religion, age, national origin, disability, veteran status, \ngenetic infor-mation, or any other classification protected by law. \nDepending on the specific circumstances, such algorithmic \ndiscrimination may violate legal protections. Designers, developers, \nand deployers of automated systems should take proactive and \ncontinuous measures to protect individuals and communities \nfrom algorithmic discrimination and to use and design systems in \nan equitable way.  This protection should include proactive equity \nassessments as part of the system design, use of representative data \nand protection against proxies for demographic features, ensuring \naccessibility for people with disabilities in design and development, \npre-deployment and ongoing disparity testing and mitigation, and \nclear organizational oversight. Independent evaluation and plain \nlanguage reporting in the form of an algorithmic impact assessment, \nincluding disparity testing results and mitigation information, \nshould be performed and made public whenever possible to confirm \nthese protections.\n23'
 ""-\ntion when deployed. This assessment should be performed regularly and whenever a pattern of unusual results is occurring. It can be performed using a variety of approaches, taking into account whether and how demographic information of impacted people is available, for example via testing with a sample of users or via qualitative user experience research. Riskier and higher-impact systems should be monitored and assessed more frequentl y. Outcomes of this assessment should include additional disparity mitigation, if needed, or \nfallback to earlier procedures in the case that equity standards are no longer met and can't be mitigated, and prior mechanisms provide better adherence to equity standards. \n27Algorithmic \nDiscrimination \nProtections""
 'Disparity mitigation. When a disparity assessment identifies a disparity against an assessed group, it may \nbe appropriate to take steps to mitigate or eliminate the disparit y. In some cases, mitigation or elimination of \nthe disparity may be required by law. Disparities that have the potential to lead to algorithmic \ndiscrimination, cause meaningful harm, or violate equity49 goals should be mitigated. When designing and \nevaluating an automated system, steps should be taken to evaluate multiple models and select the one that \nhas the least adverse impact, modify data input choices, or otherwise identify a system with fewer \ndisparities. If adequate mitigation of the disparity is not possible, then the use of the automated system \nshould be reconsidered. One of the considerations in whether to use the system should be the validity of any \ntarget measure; unobservable targets may result in the inappropriate use of proxies. Meeting these \nstandards may require instituting mitigation procedures and other protective measures to address \nalgorithmic discrimination, avoid meaningful harm, and achieve equity goals. \nOngoing monitoring and mitigation. Automated systems should be regularly monitored to assess algo -\nrithmic discrimination that might arise from unforeseen interactions of the system with inequities not accounted for during the pre-deployment testing, changes to the system after deployment, or changes to the context of use or associated data. Monitoring and disparity assessment should be performed by the entity deploying or using the automated system to examine whether the system has led to algorithmic discrimina\n-']","Algorithmic discrimination can be prevented through a combination of proactive measures and equity assessments that focus on fairness, transparency, and accountability in the design and deployment of automated systems. Here are several key strategies:

1. **Proactive Equity Assessments**: Conduct thorough equity assessments during the design phase of automated systems. This involves evaluating the potential impacts of the system on different demographic groups to identify any risks of discrimination. These assessments should be ongoing and revisited regularly to adapt to changes in the system or its context (Source: [document name], p. 5).

2. **Diverse and Representative Data**: Ensure that the training data used for algorithms is diverse and representative of the populations that the system will affect. This helps to minimize biases that can arise from underrepresented groups in the data. Data collection should actively seek to include a wide range of demographic characteristics (Source: [document name], p. 5).

3. **Bias Detection and Mitigation**: Implement tools and techniques for detecting and mitigating bias in algorithms. This can include pre-deployment testing to identify potential biases in model outputs and ongoing monitoring after deployment to ensure that the system continues to operate fairly (Source: [document name], p. 10).

4. **Human Oversight**: Incorporate human oversight in decision-making processes, especially in high-stakes applications. Human reviewers can provide context and judgment that algorithms may lack, helping to catch potential discriminatory outcomes before they affect individuals (Source: [document name], p. 18).

5. **Transparent Algorithms**: Develop algorithms that are transparent and explainable. Users and stakeholders should be able to understand how decisions are made, which can help identify and address any discriminatory practices. Providing clear documentation of the algorithm's decision-making process is essential (Source: [document name], p. 5).

6. **Stakeholder Engagement**: Engage with stakeholders, including affected communities, during the development and deployment of automated systems. Their input can provide valuable insights into potential biases and help ensure that the system meets the needs of diverse populations (Source: [document name], p. 51).

7. **Regular Audits and Reporting**: Conduct regular audits of automated systems to assess their performance and impact on different demographic groups. These audits should include public reporting of findings, which can help hold organizations accountable and promote transparency (Source: [document name], p. 5).

8. **Clear Accountability Structures**: Establish clear accountability structures within organizations for addressing algorithmic discrimination. This includes defining roles and responsibilities for monitoring and mitigating bias, as well as creating channels for reporting and addressing concerns (Source: [document name], p. 18).

By implementing these proactive measures and conducting thorough equity assessments, organizations can significantly reduce the risk of algorithmic discrimination and promote fairness in their automated systems.","Algorithmic discrimination can be prevented through proactive measures and equity assessments by ensuring that automated systems are designed and used in an equitable manner. This includes conducting proactive equity assessments during system design, using representative data, avoiding proxies for demographic features, ensuring accessibility for individuals with disabilities, conducting pre-deployment and ongoing disparity testing, and maintaining clear organizational oversight. Independent evaluation and plain language reporting, such as algorithmic impact assessments that include testing results and mitigation information, should be performed and made public whenever possible to confirm these protections.",1.0,0.9192822575544658,1.0,0.9999999999857143,0.88769392446946
How can system design ensure behavior transparency in high-risk settings while meeting expectations for automated systems?,"['NOTICE & \nEXPLANATION \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nTailored to the level of risk. An assessment should be done to determine the level of risk of the auto -\nmated system. In settings where the consequences are high as determined by a risk assessment, or extensive \noversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. \nValid. The explanation provided by a system should accurately reflect the factors and the influences that led \nto a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns. \nDemonstrate protections for notice and explanation \nReporting. Summary reporting should document the determinations made based on the above consider -'
 'Risk identification and mitigation. Before deployment, and in a proactive and ongoing manner, poten -\ntial risks of the automated system should be identified and mitigated. Identified risks should focus on the potential for meaningful impact on people’s rights, opportunities, or access and include those to impacted communities that may not be direct users of the automated system, risks resulting from purposeful misuse of the system, and other concerns identified via the consultation process. Assessment and, where possible, mea\n-\nsurement of the impact of risks should be included and balanced such that high impact risks receive attention and mitigation proportionate with those impacts. Automated systems with the intended purpose of violating the safety of others should not be developed or used; systems with such safety violations as identified unin\n-\ntended consequences should not be used until the risk can be mitigated. Ongoing risk mitigation may necessi -\ntate rollback or significant modification to a launched automated system. \n18'
 'Meaningful access to examine the system. Designers, developers, and deployers of automated \nsystems should consider limited waivers of confidentiality (including those related to trade secrets) where necessary in order to provide meaningful oversight of systems used in sensitive domains, incorporating mea\n-\nsures to protect intellectual property and trade secrets from unwarranted disclosure as appropriate. This includes (potentially private and protected) meaningful access to source code, documentation, and related data during any associated legal discovery, subject to effective confidentiality or court orders. Such meaning\n-\nful access should include (but is not limited to) adhering to the principle on Notice and Explanation using the highest level of risk so the system is designed with built-in explanations; such systems should use fully-trans\n-\nparent models where the model itself can be understood by people needing to directly examine it. \nDemonstrate access to human alternatives, consideration, and fallback \nReporting. Reporting should include an assessment of timeliness and the extent of additional burden for human alternatives, aggregate statistics about who chooses the human alternative, along with the results of the assessment about brevity, clarity, and accessibility of notice and opt-out instructions. Reporting on the accessibility, timeliness, and effectiveness of human consideration and fallback should be made public at regu\n-'
 'DATA PRIVACY \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nData access and correction. People whose data is collected, used, shared, or stored by automated \nsystems should be able to access data and metadata about themselves, know who has access to this data, and \nbe able to correct it if necessar y. Entities should receive consent before sharing data with other entities and \nshould keep records of what data is shared and with whom. \nConsent withdrawal and data deletion. Entities should allow (to the extent legally permissible) with -\ndrawal of data access consent, resulting in the deletion of user data, metadata, and the timely removal of their data from any systems (e.g., machine learning models) derived from that data.\n68\nAutomated system support. Entities designing, developing, and deploying automated systems should \nestablish and maintain the capabilities that will allow individuals to use their own automated systems to help them make consent, access, and control decisions in a complex data ecosystem. Capabilities include machine readable data, standardized data formats, metadata or tags for expressing data processing permissions and preferences and data provenance and lineage, context of use and access-specific tags, and training models for assessing privacy risk. \nDemonstrate that data privacy and user control are protected \nIndependent evaluation. As described in the section on Safe and Effective Systems, entities should allow \nindependent evaluation of the claims made regarding data policies. These independent evaluations should be \nmade public whenever possible. Care will need to be taken to balance individual privacy with evaluation data \naccess needs. \nReporting. When members of the public wish to know what data about them is being used in a system, the'
 'SAFE AND EFFECTIVE \nSYSTEMS \nWHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS\nThe expectations for automated systems are meant to serve as a blueprint for the development of additional \ntechnical standards and practices that are tailored for particular sectors and contexts. \nIn order to ensure that an automated system is safe and effective, it should include safeguards to protect the \npublic from harm in a proactive and ongoing manner; avoid use of data inappropriate for or irrelevant to the task at hand, including reuse that could cause compounded harm; and demonstrate the safety and effectiveness of the system. These expectations are explained below. \nProtect the public from harm in a proactive and ongoing manner \nConsultation. The public should be consulted in the design, implementation, deployment, acquisition, and \nmaintenance phases of automated system development, with emphasis on early-stage consultation before a system is introduced or a large change implemented. This consultation should directly engage diverse impact\n-\ned communities to consider concerns and risks that may be unique to those communities, or disproportionate -\nly prevalent or severe for them. The extent of this engagement and the form of outreach to relevant stakehold -'
 'should not be used in education, work, housing, or in other contexts where the use of such surveillance \ntechnologies is likely to limit rights, opportunities, or access. Whenever possible, you should have access to \nreporting that confirms your data decisions have been respected and provides an assessment of the \npotential impact of surveillance technologies on your rights, opportunities, or access. \nNOTICE AND EXPLANATION\nYou should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organiza\n-\ntion responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible. \n6'
 'SAFE AND EFFECTIVE \nSYSTEMS \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nWhile technologies are being deployed to solve problems across a wide array of issues, our reliance on technology can \nalso lead to its use in situations where it has not yet been proven to work—either at all or within an acceptable range of error. In other cases, technologies do not work as intended or as promised, causing substantial and unjustified harm. Automated systems sometimes rely on data from other systems, including historical data, allowing irrelevant informa\n-\ntion from past decisions to infect decision-making in unrelated situations.  In some cases, technologies are purposeful -\nly designed to violate the safety of others, such as technologies designed to facilitate stalking; in other cases, intended or unintended uses lead to unintended harms. \nMany of the harms resulting from these technologies are preventable, and actions are already being taken to protect the public. Some companies have put in place safeguards that have prevented harm from occurring by ensuring that key development decisions are vetted by an ethics review; others have identified and mitigated harms found through pre-deployment testing and ongoing monitoring processes. Governments at all levels have existing public consulta\n-\ntion processes that may be applied when considering the use of new automated systems, and existing product develop -\nment and testing practices already protect the American public from many potential harms. \nStill, these kinds of practices are deployed too rarely and unevenly. Expanded, proactive protections could build on these existing practices, increase confidence in the use of automated systems, and protect the American public. Inno\n-']","To ensure behavior transparency in high-risk settings while meeting expectations for automated systems, system design should incorporate several key principles and practices:

1. **Comprehensive Risk Assessment**: Begin with a thorough risk assessment to identify the potential consequences of the automated system's decisions. This assessment should evaluate the specific risks associated with the context in which the system will operate, such as healthcare, criminal justice, or financial services (Source: [document name], p. 5).

2. **Use of Fully Transparent Models**: Design the system using fully transparent models that allow users to understand how decisions are made. This means employing algorithms and methodologies that can be easily interpreted and explained, rather than relying on ""black box"" models that obscure decision-making processes (Source: [document name], p. 5).

3. **Built-in Explanatory Features**: Incorporate built-in explanatory features that provide users with clear and meaningful explanations of the system's behavior. These features should articulate the factors and influences that led to specific decisions, ensuring that users can comprehend the rationale behind outcomes (Source: [document name], p. 5).

4. **User-Friendly Interfaces**: Develop user interfaces that present explanations in an accessible and understandable manner. This includes using plain language and visual aids to help users grasp complex information without requiring specialized knowledge (Source: [document name], p. 5).

5. **Error Reporting and Feedback Mechanisms**: Implement mechanisms for reporting errors and providing feedback on the system's decisions. Users should have the ability to contest decisions and receive explanations that clarify any misunderstandings or mistakes (Source: [document name], p. 5).

6. **Regular Updates and Audits**: Establish processes for regular updates and audits of the explanatory mechanisms to ensure they remain effective and relevant. This includes incorporating user feedback and adapting explanations based on evolving understanding of the system's behavior (Source: [document name], p. 5).

7. **Stakeholder Engagement**: Engage with stakeholders, including affected communities and experts, during the design process to gather input on what constitutes meaningful explanations. This collaborative approach can help ensure that the explanations provided are relevant and useful to those impacted by the system (Source: [document name], p. 5).

By integrating these principles into the system design, organizations can enhance transparency and accountability in high-risk environments, ultimately fostering trust and ensuring ethical use of automated systems.","In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. The extent of explanation provided should be tailored to the risk level.",0.9642857142857143,0.9547549688807347,1.0,0.9999999999857143,0.4556201403606417