<jupyter_start><jupyter_text>Amazon Augmented AI (Amazon A2I) Amazon Augmented AI (Amazon A2I) makes it easy to build the workflows required for human review of ML predictions. Amazon A2I brings human review to all developers, removing the undifferentiated heavy lifting associated with building human review systems or managing large numbers of human reviewers. You can create your own workflows for ML models built on Amazon SageMaker or any other tools. Using Amazon A2I, you can allow human reviewers to step in when a model is unable to make a high confidence prediction or to audit its predictions on an on-going basis. Learn more here: https://aws.amazon.com/augmented-ai/   Integrate Human Reviewers into a Amazon Comprehend Prediction Pipeline To incorporate Amazon A2I into your human review workflows, you need the following resources:* **Workforce** to label your dataset. You can choose either the Amazon Mechanical Turk workforce, a vendor-managed workforce, or you can create your own private workforce for human reviews. Whichever workforce type you choose, Amazon Augmented AI takes care of sending tasks to workers.* **Worker Task Template** to create a Human Task UI for the worker. The worker UI displays your input data, such as documents or images, and instructions to workers. It also provides interactive tools that the worker uses to complete your tasks. For more information, see https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-instructions-overview.html* **Flow Definition** to create a Human Review Workflow.You use the flow definition to configure your human workforce and provide information about how to accomplish the human review task. You can create a flow definition in the Amazon Augmented AI console or with Amazon A2I APIs. To learn more about both of these options, see https://docs.aws.amazon.com/sagemaker/latest/dg/a2i-create-flow-definition.html* **Human Loop** starts your human review workflow. When you use one of the built-in task types, the corresponding AWS service creates and starts a human loop on your behalf when the conditions specified in your flow definition are met or for each object if no conditions were specified. When a human loop is triggered, human review tasks are sent to the workers as specified in the flow definition.When using a custom task type, as this notebook will show, you start a human loop using the Amazon Augmented AI Runtime API. When you call StartHumanLoop in your custom application, a task is sent to human reviewers.  Start a Human Loop When Comprehend Does Not Produce a Confident PredictionThe human loop will engage our workforce for human review if the confidence of the Comprehend prediction is less than the provided confidence.![](img/augmented-ai-comprehend-predictions.png)<jupyter_code>%%javascript
Jupyter.notebook.save_checkpoint()
Jupyter.notebook.session.delete();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.04484304932735426, 'fraction_numerical': 0.0027595722662987236, 'mean_word_length': 5.079664570230608, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '237572', 'license': 'Apache-2.0', 'n_tokens_mistral': 697, 'n_tokens_neox': 659, 'n_words': 410, 'path': '10_pipeline/human_in_the_loop/00_Overview.ipynb', 'repo_name': 'MarcusFra/workshop', 'size': '2899'}