<jupyter_start><jupyter_text>FinanceIn the context of integrating and utilizing NLP in the context of finance frameworks, operations, reporting, and evaluation, we can look at finance from the following three angles:- Organization Perspective: Different organization types have different requirements and perspectives that need to be taken into account.    - Private companies    - Public companies    - Non-profit companies    - Governmental organizations- Actions    - Allocating and reallocating funds    - Accounting and auditing, which includes identifying anomalies and outliers to investigate for both value and risk    - Prioritization and resource planning    - Compliance with legal and policy norms- Financial context    - Forecasting and budgeting    - Retail banking    - Investment banking    - Stock market operations    - Cryptocurrency operations     NLP Applications in Finance- Financial SentimentStock market trading relies on a set of information about specific companies. This knowledge helps create a set of actions that determine whether to buy, hold, or sell off stock. This analysis can be based on companies’ quarterly financial reports or on what analysts are commenting about the companies in their reports. This can also come from social media.The purpose is to guess how the markets will react to a piece of news, as opposed to whether the news is inherently positive or not.FinBERT uses a subset of financial news from Reuters. For sentiment classification, it uses Financial PhraseBank, which has over 4,000 sentences labeled by people with backgrounds in business and finance. Unlike regular sentiment analysis, where positive means that something is of positive emotion, in Financial PhraseBank, a positive sentiment indicates that the stock price of the company will increase based on the news in the sentence.- Risk AssessmentNLP techniques can add a lot more data points that can be used to assess credit risk. For example, in business loans, entrepreneurial ability and attitude can be measured using NLP. This approach is used by Capital Float and Microbnk. Similarly, incoherencies in data provided by the borrower can also be surfaced for more scrutiny. Other more nuanced aspects, such as lenders’ and borrowers’ emotions while applying for a loan, can also be incorporated.Often in personal loan agreements, various information has to be captured from loan documents, which are then fed to credit risk models. The information captured helps in identifying credit risk, and erroneous data extraction from these documents can lead to flawed assessments. Named entity recognition (NER) can improve this.- Accounting and AuditingWhile applying aspects of NLP and ML to areas like contract document reviews and long-term procurement agreements, Deloitte, for example, has evolved its Audit Command Language into a more efficient NLP application.In addition, after decades of long, drawn-out ticking and tying of reams of endless, typical day-to-day transactions and other pieces of paper like invoices, companies have finally realized that NLP and ML has a significant advantage in the audit process. This advantage manifests in the direct identification, focus, visualization, and trend analysis of outliers in transaction types. Time and effort are spent on the investigation of these outliers and their causes. This results in early identification of potentially significant risks and possible fraudulent activity like money laundering along with potentially value-generating activities that can be emulated and extrapolated across a company and customized for various business processes.<jupyter_code># Google Colab for FinBERT
# https://colab.research.google.com/drive/1Ih4P7xq2VyGf4Bg3rGFcnHAycgNpxU-T?usp=sharing<jupyter_output><empty_output><jupyter_text>NLP and the Legal LandscapeSome core tasks where NLP helps legal services include:- Legal researchThis involves finding relevant information for a specific case, including searching both legislatures and case law and regulations. One such service is ROSS Intelligence. It allows matching of facts and relevant cases and also analyzes legal documents.![alt text](https://learning.oreilly.com/library/view/practical-natural-language/9781492054047/assets/pnlp_1018.png)- Contract reviewThis refers to reviewing a contract and making sure it follows a set of norms and regulations. It involves making comments and suggesting edits for different clauses. One example is SpotDraft, which focuses on GPDR-based regulations.- Contract generationThis refers to generating contracts based on a question-and-answer setup. Simple cases may just require a simple form, whereas for more complex cases, an interactive chatbot system may be more suitable. After taking in all the responses, a slot-filling algorithm generates the contract.- Legal DiscoveryThis refers to finding anomalies and patterns in electronically stored information that can be used for the case. In some cases, this discovery is completely unsupervised. In other cases, it can involve more active learning (i.e., providing an initial set of tagged documents). One such product is siren.io, which aids discovery for intelligence, law enforcement, cyber security, and financial crime domains. Legal Entity Extraction with LexNLPLexNLP helps us segment documents into sections and extract facts like recurring contract dates or regulations. LexNLP also provides legal dictionaries and knowledge sets for multiple countries for accounting, financial information, regulators, and legal and medical areas. It also integrates with ContraxSuite, which allows us to deduplicate documents, cluster legal entities according to how they’re mentioned, and so on. When building custom applications, we can also inject code to build on the baseline platform.Clustering of legal entities from a set of documents![alt text](https://learning.oreilly.com/library/view/practical-natural-language/9781492054047/assets/pnlp_1020.png)<jupyter_code># Google Colab for LexNLP
# https://colab.research.google.com/drive/1oPLmrIp0gfSjW2CeGgxBbqoIxDI9DTXj?usp=sharing<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.03557441253263707, 'fraction_numerical': 0.007832898172323759, 'mean_word_length': 5.254081632653061, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '333407', 'license': 'MIT', 'n_tokens_mistral': 1394, 'n_tokens_neox': 1290, 'n_words': 806, 'path': '10_Healthcare, Finance, and Law/02_Finance and Law.ipynb', 'repo_name': 'yasirabd/practical-nlp', 'size': '6128'}