<jupyter_start><jupyter_text>Categorical data validationWhat expression asserts that the distinct Nobel Prize categories catalogued by the "prizes" collection are the same as those catalogued by the "laureates"? Remember to explore example documents in the console via e.g. db.prizes.find_one() and db.laureates.find_one().<jupyter_code># assert set(db.prizes.distinct("category")) == set(db.laureates.distinct("prizes.category"))
from pymongo import MongoClient

client = MongoClient('mongodb://localhost:27017')

# Connect to the "nobel" database
db = client.nobel<jupyter_output><empty_output><jupyter_text>Never from there, but sometimes there at lastThere are some recorded countries of death ("diedCountry") that do not appear as a country of birth ("bornCountry") for laureates. One such country is "East Germany".<jupyter_code># Countries recorded as countries of death but not as countries of birth
countries = set(db.laureates.distinct("diedCountry")) - set(db.laureates.distinct("bornCountry"))
print(countries)<jupyter_output>{'USSR', 'Puerto Rico', 'Jamaica', 'Greece', 'Czechoslovakia', 'Israel', 'Gabon', 'Yugoslavia (now Serbia)', 'Tunisia', 'East Germany', 'Northern Rhodesia (now Zambia)', 'Philippines', 'Barbados'}<jupyter_text>Countries of affiliationWe saw in the last exercise that countries can be associated with a laureate as their country of birth and as their country of death. For each prize a laureate received, they may also have been affiliated with an institution at the time, located in a country.<jupyter_code># The number of distinct countries of laureate affiliation for prizes
count = len(db
    .laureates
    .distinct('prizes.affiliations.country'))
print(count)<jupyter_output>29<jupyter_text>Born here, went thereIn which countries have USA-born laureates had affiliations for their prizes?Possible Answers:Australia, Denmark, United Kingdom, USA<jupyter_code>print(db.laureates.distinct('prizes.affiliations.country', {'bornCountry': 'USA'}))<jupyter_output>['Australia', 'Denmark', 'USA', 'United Kingdom']<jupyter_text>Triple plays (mostly) all aroundPrizes can be shared, even by more than two laureates. In fact, all prize categories but one – literature – have had prizes shared by three or more laureates.<jupyter_code># Save a filter for prize documents with three or more laureates
criteria = {
    'laureates.2': {
        '$exists': True
    }
}

# Save the set of distinct prize categories in documents satisfying the criteria
triple_play_categories = set(db
                            .prizes
                            .distinct(
                                'category', 
                                criteria))

# Confirm literature as the only category not satisfying the criteria.
assert set(db.prizes.distinct("category")) - triple_play_categories == {"literature"}<jupyter_output><empty_output><jupyter_text>Sharing in physics after World War IIWhat is the approximate ratio of the number of laureates who won an unshared ({"share": "1"}) prize in physics after World War II ({"year": {"$gte": "1945"}}) to the number of laureates who won a shared prize in physics after World War II?For reference, the code below determines the number of laureates who won a shared prize in physics before 1945.```db.laureates.count_documents({    "prizes": {"$elemMatch": {        "category": "physics",        "share": {"$ne": "1"},        "year": {"$lt": "1945"}}}})```<jupyter_code>print(db.laureates.count_documents({
    "prizes": {"$elemMatch": {
        "category": "physics",
        "share": {"$ne": "1"},
        "year": {"$lt": "1945"}}}}))
print(db.laureates.count_documents({
    "prizes": {"$elemMatch": {
        "category": "physics",
        "share": {"$ne": "1"},
        "year": {"$gte": "1945"}}}}))
19 / 143<jupyter_output><empty_output><jupyter_text>Meanwhile, in other categories...We learned in the last exercise that there has been significantly more sharing of physics prizes since World War II: the ratio of the number of laureates who won an unshared prize in physics in or after 1945 to the number of laureates who shared a prize in physics in or after 1945 is approximately 0.13. What is this ratio for prize categories other than physics, chemistry, and medicine?<jupyter_code># Save a filter for laureates with unshared prizes
unshared = {
    "prizes": {
        '$elemMatch': {
        "category": {
            "$nin": [
                "physics", 
                "chemistry", 
                "medicine"]
        },
        "share": "1",
        "year": {
            "$gte": "1945"
            },
    }}}

# Save a filter for laureates with shared prizes
shared = {
    "prizes": {"$elemMatch": {
        "category": {
            "$nin": [
                "physics", 
                "chemistry", 
                "medicine"]
        },
        "share": {
            "$ne": "1"
        },
        "year": {
            "$gte": "1945"
        },
    }}}

ratio = db.laureates.count_documents(unshared) / db.laureates.count_documents(shared)
print(ratio)<jupyter_output>1.3653846153846154<jupyter_text>Organizations and prizes over timeHow many organizations won prizes before 1945 versus in or after 1945?<jupyter_code># Save a filter for organization laureates with prizes won before 1945
before = {
    "gender": "org",
    "prizes.year": {
        "$lt": "1945"
        },
    }

# Save a filter for organization laureates with prizes won in or after 1945
in_or_after = {
    'gender': 'org',
    'prizes.year': {
        '$gte': "1945"
        },
    }

n_before = db.laureates.count_documents(before)
n_in_or_after = db.laureates.count_documents(in_or_after)
ratio = n_in_or_after / (n_in_or_after + n_before)
print(ratio)<jupyter_output>0.84<jupyter_text>Glenn, George, and others in the G.B. crewThere are two laureates with Berkeley, California as a prize affiliation city that have the initials G.S. - <NAME> and <NAME>. How many laureates in total have a first name beginning with "G" and a surname beginning with "S"?Evaluate the expression```db.laureates.count_documents({"firstname": Regex(____), "surname": Regex(____)})```in the console, filling in the blanks appropriately.<jupyter_code>from bson.regex import Regex
print(db.laureates.count_documents({
    "firstname": Regex("^G"), 
    "surname": Regex("^S")
    }))<jupyter_output>9<jupyter_text>Germany, then and nowJust as we saw with Poland, there are laureates who were born somewhere that was in Germany at the time but is now not, and others born somewhere that was not in Germany at the time but now is.<jupyter_code>from bson.regex import Regex

# Filter for laureates with "Germany" in their "bornCountry" value
criteria = {
    "bornCountry": Regex("Germany"
    )}
print(set(db.laureates.distinct("bornCountry", criteria)))
from bson.regex import Regex

# Filter for laureates with a "bornCountry" value starting with "Germany"
criteria = {
    "bornCountry": Regex("^Germany", 0)
    }

print(set(db.laureates.distinct("bornCountry", criteria)))
from bson.regex import Regex

# Fill in a string value to be sandwiched between the strings "^Germany " and "now"
criteria = {"bornCountry": Regex("^Germany " + "\(" + "now")}
print(set(db.laureates.distinct("bornCountry", criteria)))
from bson.regex import Regex

#Filter for currently-Germany countries of birth. Fill in a string value to be sandwiched between the strings "now" and "$"
criteria = {"bornCountry": Regex("now" + " Germany\)" + "$")}
print(set(db.laureates.distinct("bornCountry", criteria)))<jupyter_output>{'Mecklenburg (now Germany)', 'East Friesland (now Germany)', 'Bavaria (now Germany)', 'Hesse-Kassel (now Germany)', 'Prussia (now Germany)', 'Schleswig (now Germany)', 'West Germany (now Germany)', 'W&uuml;rttemberg (now Germany)'}<jupyter_text>The prized transistorThree people shared a Nobel prize "for their researches on semiconductors and their discovery of the transistor effect". We can filter on "transistor" as a substring of a laureate's "prizes.motivation" field value to find these laureates.<jupyter_code>from bson.regex import Regex

# Save a filter for laureates with prize motivation values containing "transistor" as a substring
criteria = {'prizes.motivation': Regex('transistor')}

# Save the field names corresponding to a laureate's first name and last name
first, last = 'firstname', 'surname'
print([(laureate[first], laureate[last]) for laureate in db.laureates.find(criteria)])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11605142115815545, 'fraction_numerical': 0.012265597358179031, 'mean_word_length': 4.1145958986731, 'pattern_counts': {'":': 49, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '380668', 'license': 'MIT', 'n_tokens_mistral': 2712, 'n_tokens_neox': 2500, 'n_words': 864, 'path': 'introduction-to-mongodb-in-python/2. Working with Distinct Values and Sets/notebook_section_2.ipynb', 'repo_name': 'nhutnamhcmus/datacamp-playground', 'size': '8492'}