<jupyter_start><jupyter_text>Introduction to Nettlesome Nettlesome is a Python library that lets you use readable English phrases as semantic tags for your data. Once you've created tags, Nettlesome can automate the process of comparing the tags to determine whether they have the *same meaning*, whether one tag *implies* another, whether one tag *contradicts* another, or whether one tag is *consistent with* another.This tutorial will show you how to use some of Nettlesome's basic classes: Predicates, Statements, and Comparisons.  Predicates A Predicate should contain an English-language phrase in the past tense. (The past tense reflect's Nettlesome's original use case of legal analysis, which is usually backward-looking, determining the legal effect of past acts or past conditions.)To create a useful Predicate, you might remember from grade school that in a simple sentence the "subject" and any "objects" are typically nouns, while the "predicate" is a verb phrase that describes an action or relationship of those nouns. In Nettlesome the "subject" and "objects" are simply called "Terms", and they should be marked up using the placeholder syntax for Template Strings, which are a part of the Python standard library. Generally speaking, objects that are part of your data model should be designated as Terms, while other nouns that aren't relevant enough to be part of your data model don't need to be terms.For instance, this example might be a good Predicate for a data model about determining whether an individual is an owner or employee of a company:<jupyter_code>from nettlesome import Predicate

account_for_company = Predicate("$applicant opened a bank account for $company")<jupyter_output><empty_output><jupyter_text>Because `$applicant` and `$company` are marked as placeholders for Terms, you'll be able to add more data about those entities later. But notice that the "bank account" wasn't designated as a Term. That means you won't be able to link any more Statements about the company's specific bank account. That should be okay as long as your data model won't need to include specific details about bank accounts. Predicates also have a `truth` attribute that can be used to establish that one Predicate `contradicts` another.<jupyter_code>no_account_for_company = Predicate("$applicant opened a bank account for $company", truth=False)
str(no_account_for_company)<jupyter_output><empty_output><jupyter_text>The `truth` attribute will become more significant as we build up to create more complex objects.  Formatting Predicates The placeholders you choose always need to `$start_with_a_dollar_sign` and they need to be valid Python identifiers, which means they can't contain spaces. If a placeholder needs to be adjacent to a non-whitespace character, you also need to `${wrap_it_in_curly_braces}`.Don't use capitalization or end punctuation to signal the beginning or end of the phrase in a Predicate, because the phrase may be used in a context where it's only part of a longer sentence.As the example below shows, the use of different placeholders doesn't cause Predicates to be considered to have different meanings.<jupyter_code>account_for_partnership = Predicate("$applicant opened a bank account for $partnership")

account_for_company.means(account_for_partnership)<jupyter_output><empty_output><jupyter_text>If you need to mention the same term more than once in a Predicate,use the same placeholder for that term each time. If you later createa Statement object using the same Predicate, you will only include each uniqueTerm once, in the order they first appear.In this example, a Predicate's template has two placeholders referring to the identical Term.Even though the rest of the text is the same, the reuse of the same Term means that the Predicate has a different meaning.<jupyter_code>account_for_self = Predicate("$applicant opened a bank account for $applicant")

account_for_self.means(account_for_company)<jupyter_output><empty_output><jupyter_text>Statements To put it simply, a Statement is a Predicate plus the Terms that need to be included to make the Statement a complete phrase.<jupyter_code>from nettlesome import Statement, Entity

statement = Statement(predicate=account_for_company, terms=[Entity("Sarah"), Entity("Acme Corporation")])
str(statement)<jupyter_output><empty_output><jupyter_text>An Entity is a Term representing a person or thing. If you're lucky enough to be able to run effective Named Entity Recognition techniques on your dataset, you probably already have good candidates for the Entity objects that should be included in your Statements. The data model of an Entity in Nettlesome includes just a `name` attribute, an attribute indicating whether the Entity should be considered `generic`, and a `plural` attribute mainly used to determine whether the word "was" after the Entity should be replaced with "were".<jupyter_code>not_at_school = Predicate("$group were at school", truth=False)
plural_statement = Statement(not_at_school, terms=Entity("the students", plural=True))
str(plural_statement)
singular_statement = Statement(not_at_school, terms=Entity("Lee", plural=False))
str(singular_statement)<jupyter_output><empty_output><jupyter_text>The `generic` attribute is more subtle than the `plural` attribute. A Term should be marked as `generic` if it's really being used as an example or stand-in for a broader category. For instance, in `singular_statement` above, the fact that `` is generic indicates that the Statement isn't really about a specific incident when Lee was not at school. Instead, it's more about the concept of not being at school. In Nettlesome, when angle brackets appear around the string representation of an object, that's an indication that the object is generic.If two Statements have different generic Terms but they're otherwise the same, they're still considered to have the same meaning as one another. That's the case even if one of the Terms is plural while the other is singular.<jupyter_code>plural_statement.means(singular_statement)<jupyter_output><empty_output><jupyter_text>However, sometimes you need to label an Entity as being somehow sui generis, so that Statements about that Entity aren't really applicable to other, generic Entities. In that case, you can set the Entity's `generic` attribute to False and it'll no longer be found to have the same meaning as generic Entities.<jupyter_code>harry_statement = Statement(not_at_school, terms=Entity("<NAME>", generic=False))
harry_statement.means(singular_statement)<jupyter_output><empty_output><jupyter_text>By default, Entities are generic and Statements are not generic. Both of these defaults can be changed when you create instances of the respective classes.  Comparisons The Comparison class extends the concept of a Predicate.A Comparison still contains a truth value and a template string,but that template should be used to identify a quantity that willbe compared to an expression using a sign such as an equal signor a greater-than sign. This expression must be a constant: eitheran integer, a floating point number, or a physical quantity expressedin units that can be parsed using the pint library.<jupyter_code>from nettlesome import Comparison

weight_in_pounds = Comparison("the weight of ${driver}'s vehicle was", sign=">", expression="26000 pounds")
pounds_statement = Statement(weight_in_pounds, terms=Entity("Alice"))
str(pounds_statement)<jupyter_output><empty_output><jupyter_text>Statements including Comparisons will handle unit conversions when applying operations like `implies` or `contradicts`.<jupyter_code>weight_in_kilos = Comparison("the weight of ${driver}'s vehicle was", sign="<=", expression="3000 kilograms")
kilos_statement = Statement(weight_in_kilos, terms=Entity("Alice"))
str(kilos_statement)
pounds_statement.contradicts(kilos_statement)<jupyter_output><empty_output><jupyter_text>Formatting Comparisons To encourage consistent phrasing, the template string in everyComparison object must end with the word “was”.If you phrase a Comparison with an inequality sign using `truth=False`,Nettlesome will silently modify your statement so it can have`truth=True` with a different sign. In this example, the user’s inputindicates that it’s false that the weight of marijuana possessed by a defendantwas an ounce or more. Nettlesome interprets this to mean it’strue that the weight was less than one ounce.<jupyter_code>drug_comparison_with_upper_bound = Comparison(
   "the weight of marijuana that $defendant possessed was",
    sign=">=",
    expression="1 ounce",
    truth=False)
str(drug_comparison_with_upper_bound)<jupyter_output><empty_output><jupyter_text>When the number needed for a Comparison isn’t a physical quantity that can be describedwith the units in the pint library library, you should phrase the text in the templatestring to explain what the number describes. The template string will still need toend with the word “was”. The value of the expression parameter should be an integeror a floating point number, not a string to be parsed.<jupyter_code>three_children = Comparison(
    "the number of children in ${taxpayer}'s household was",
    sign="=",
    expression=3)
str(three_children)<jupyter_output><empty_output><jupyter_text>Groups of Statements You can also check to see whether a set of Statements, taken as a group, implies another Statement or group of Statements.Here, the use of placeholders that are identical except for a digit on the end indicates to Nettlesome that the positions of those Terms should be considered interchangeable. (In this example, if `site1` is a certain distance away from `site2`, then `site2` must also be the same distance away from `site1`.)<jupyter_code>from nettlesome import FactorGroup

more_than_100_yards = Comparison("the distance between $site1 and $site2 was", sign=">", expression="100 yards")
less_than_1_mile = Comparison("the distance between $site1 and $site2 was", sign="<", expression="1 mile")

protest_facts = FactorGroup(
    [Statement(more_than_100_yards, terms=[Entity("the political convention"), Entity("the police cordon")]),
     Statement(less_than_1_mile, terms=[Entity("the police cordon"), Entity("the political convention")])]
)
str(protest_facts)
more_than_50_meters = Comparison("the distance between $site1 and $site2 was", sign=">", expression="50 meters")
less_than_2_km = Comparison("the distance between $site1 and $site2 was", sign="<=", expression="2 km")

speech_zone_facts = FactorGroup(
    [Statement(more_than_50_meters, terms=[Entity("the free speech zone"), Entity("the courthouse")]),
     Statement(less_than_2_km, terms=[Entity("the free speech zone"), Entity("the courthouse")])]
)
protest_facts.implies(speech_zone_facts)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0640978140051871, 'fraction_numerical': 0.004075583549462764, 'mean_word_length': 6.329938900203666, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 58, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '472297', 'license': 'CNRI-Python', 'n_tokens_mistral': 2853, 'n_tokens_neox': 2730, 'n_words': 1331, 'path': 'notebooks/intro.ipynb', 'repo_name': 'mscarey/nettlesome', 'size': '10802'}