<jupyter_start><jupyter_text>**[Term.ooo](https://term.ooo) "*Random*" Simulations**This notebook performs **1,397** "*random*" simulations for each one of the **1,442** possible valid answers of the game, totalling **2,014,474** simulations. This simulations aren't completely random, the strategy work as follows:+ After each attempt, the words are filtered based on the hits and misses of each element, so, the next attempt is going to be a word randomly selected from the filtered ones.+ The first **2** attempts apply a condition to pick only available words that have the maximum number of unique elements, maximizing the chances of getting good results.+ It simulates "*hard mode*" where the next attempt always uses the correct and misplaced elements from the current attempt.The output file utilizes the following chars to represent the hits and misses:+ **c** : Correctly placed element (🟩 - Green)+ **m** : Misplaced element (🟨 - Purple)+ _ : Incorrect element (⬛ - Black)  Imports and Constants<jupyter_code>from multiprocessing import Pool
from tqdm import tqdm
import pandas as pd


# Game parameters
NUMBER_OF_ATTEMPTS = 6
NUMBER_OF_ELEMENTS = 5

# Elements
ELEMENT_EMPTY = " "
ELEMENT_CORRECT = "c"
ELEMENT_MISPLACED = "m"
ELEMENT_INCORRECT = "_"
ELEMENTS = list("abcdefghijklmnopqrstuvwxyz")

# Attempts
ATTEMPT_CORRECT = ELEMENT_CORRECT * NUMBER_OF_ELEMENTS
ATTEMPT_INCORRECT = ELEMENT_INCORRECT * NUMBER_OF_ELEMENTS
EMPTY_GAME_SIMULATION = [ELEMENT_EMPTY * NUMBER_OF_ELEMENTS] * (NUMBER_OF_ATTEMPTS * 2)


# CSV header
CSV_HEADER = ["solution"]
for i in range(NUMBER_OF_ATTEMPTS):
    CSV_HEADER.append(f"attempt_{i}")
    CSV_HEADER.append(f"hits_{i}")
# Simulation parameters

SIMS_COUNT = 1_000
UNIQUE_ATTEMPTS = 2

FILEPATH_IN = f"./data/1.raw/valid_answers.csv"
FILEPATH_OUT = f"./data/2.simulations/simulations_random.csv"<jupyter_output><empty_output><jupyter_text>Load data*(And setting the index)*<jupyter_code>df_words = pd.read_csv(FILEPATH_IN)
df_words.index = df_words["word"]
df_words<jupyter_output><empty_output><jupyter_text>Pre processing<jupyter_code># Init a dictionary to store all temporary DataFrames
dfs = {k: pd.DataFrame() for k in ["positions", "unique", "elements"]}


# Spliting positions int o columns
for i in range(NUMBER_OF_ELEMENTS):
    dfs["positions"][f"p{i}"] = df_words["word"].str[i].astype("category")

# Counting the number of unique elements per word
dfs["unique"]["count"] = df_words["word"].apply(lambda x: len(set(x))).astype("uint8")

# Counting the amount of each element in each word
for element in ELEMENTS:
    dfs["elements"][element] = df_words["word"].apply(lambda x: x.count(element)).astype("uint8")


# Concatenating the DataFrames into a single multi-indexed one
df_words = pd.concat(dfs, axis=1)
df_words<jupyter_output><empty_output><jupyter_text>**Simulation functions** Compute the hits of the attempt<jupyter_code>def compute_hits(solution: str, attempt: str):

    # Init counter
    counter = {}

    # Count the number of each unique element in the attempt
    for element in set(attempt):
        counter[element] = solution.count(element)

    # Init hits
    hits = list(ATTEMPT_INCORRECT)

    # Compute 'correct' hits
    for i in range(NUMBER_OF_ELEMENTS):
        if attempt[i] == solution[i]:
            counter[attempt[i]] -= 1
            hits[i] = ELEMENT_CORRECT

    # Compute 'misplaced' hits
    for i in range(NUMBER_OF_ELEMENTS):
        if counter[attempt[i]] > 0 and hits[i] == ELEMENT_INCORRECT:
            counter[attempt[i]] -= 1
            hits[i] = ELEMENT_MISPLACED

    # Return hits
    return "".join(hits)
# Perform tests

TESTS = [
    (("abano", "perua"), "____m"),
    (("advir", "haver"), "_mc_c"),
    (("agudo", "lugar"), "_mmm_"),
    (("autor", "gueto"), "_c_mm"),
    (("azedo", "azedo"), "ccccc"),
]

for test, result in TESTS:
    assert compute_hits(*test) == result
else:
    print("All 'compute_hits()' tests passed")<jupyter_output>All 'compute_hits()' tests passed<jupyter_text>Count the number of each element in the attempt based on the attempt hits<jupyter_code>def count_elements(attempt: str, hits: str):

    # Init counter
    counter = {element: {"min": 0, "max": 0} for element in set(attempt)}

    # Count the sum of correct and misplaced elements as well as the number of incorrect elements
    for i in range(NUMBER_OF_ELEMENTS):
        if hits[i] in (ELEMENT_MISPLACED, ELEMENT_CORRECT):
            counter[attempt[i]]["min"] += 1

        else:
            counter[attempt[i]]["max"] += 1

    # Set the counter to the min and max possible elements in the solution
    for element in counter:
        counts = counter[element]

        if counts["min"] == 0:
            counts["max"] = 0

        elif counts["max"] == 0:
            counts["max"] = NUMBER_OF_ELEMENTS

        elif counts["max"] != 0:
            counts["max"] = counts["min"]

    # Return the counter
    return counter
# Perform tests

TESTS = [
    (("abano", "____m"), {'n': {'min': 0, 'max': 0}, 'b': {'min': 0, 'max': 0}, 'a': {'min': 0, 'max': 0}, 'o': {'min': 1, 'max': 5}}),
    (("advir", "_mc_c"), {'d': {'min': 1, 'max': 5}, 'i': {'min': 0, 'max': 0}, 'a': {'min': 0, 'max': 0}, 'v': {'min': 1, 'max': 5}, 'r': {'min': 1, 'max': 5}}),
    (("agudo", "_mmm_"), {'d': {'min': 1, 'max': 5}, 'u': {'min': 1, 'max': 5}, 'a': {'min': 0, 'max': 0}, 'g': {'min': 1, 'max': 5}, 'o': {'min': 0, 'max': 0}}),
    (("autor", "_c_mm"), {'t': {'min': 0, 'max': 0}, 'u': {'min': 1, 'max': 5}, 'a': {'min': 0, 'max': 0}, 'o': {'min': 1, 'max': 5}, 'r': {'min': 1, 'max': 5}}),
    (("azedo", "ccccc"), {'d': {'min': 1, 'max': 5}, 'a': {'min': 1, 'max': 5}, 'z': {'min': 1, 'max': 5}, 'e': {'min': 1, 'max': 5}, 'o': {'min': 1, 'max': 5}}),
]

for test, result in TESTS:
    assert count_elements(*test) == result
else:
    print("All 'count_elements()' tests passed")<jupyter_output>All 'count_elements()' tests passed<jupyter_text>Filter the candidate equations based on the attempt hits and attempt counter<jupyter_code>def filter_words(words: pd.DataFrame, word: str, hits: str, counter: dict):

    # Filter based on the element counts
    for element in counter:
        words = words[words["elements"][element] >= counter[element]["min"]]
        words = words[words["elements"][element] <= counter[element]["max"]]

    # Filter based on the attempt hits positions
    for i in range(NUMBER_OF_ELEMENTS):
        if hits[i] == ELEMENT_CORRECT:
            words = words[words["positions"][f"p{i}"] == word[i]]

        else:
            words = words[words["positions"][f"p{i}"] != word[i]]
    
    # Return the filtered words
    return words<jupyter_output><empty_output><jupyter_text>Simulate all of the attempts of a game<jupyter_code>def play_game(words: pd.DataFrame, solution: str, unique_attempts: int,):

    # Init output simulation data
    simulation = [solution] + EMPTY_GAME_SIMULATION

    # Iterate over the attempts
    for i in range(NUMBER_OF_ATTEMPTS):

        # Get words with the most unique elements based on the number of unique attempts passed
        if i < unique_attempts:
            temp = words[words[("unique", "count")] == words[("unique", "count")].max()]
        else:
            temp = words

        # Get a random word and compute hits
        word = temp.sample(1).iloc[0].name
        hits = compute_hits(solution, word)

        # Store values on the simulation data
        simulation[(2 * i) + 1] = word
        simulation[(2 * i) + 2] = hits

        # Break if the solution is found
        if hits == ATTEMPT_CORRECT:
            break
        
        # Filter words based on the attempt hits
        counter = count_elements(word, hits)
        words = filter_words(words, word, hits, counter)

    # Return simulation data
    return simulation<jupyter_output><empty_output><jupyter_text>Simulate a game for every solution<jupyter_code>def simulate_all_solutions(words: pd.DataFrame):
    
    # Init simulations data
    simulations = []

    # Iterate over every solution
    for solution in words.index:

        # Run and append the simulation data
        simulations.append(play_game(words, solution, UNIQUE_ATTEMPTS))
    
    # Append the simulation data to the CSV file
    with open(FILEPATH_OUT, "a", newline="") as f:
        pd.DataFrame(simulations, columns=CSV_HEADER).to_csv(f, header=False, index=False)<jupyter_output><empty_output><jupyter_text>Run simulations<jupyter_code># Init output file
with open(FILEPATH_OUT, "w", newline="") as f:
    pd.DataFrame(columns=CSV_HEADER).to_csv(f, index=False)

# Starting the multiprocessing Pool
with Pool() as p:
    list(tqdm(p.imap(simulate_all_solutions, [df_words for _ in range(SIMS_COUNT)]), total=SIMS_COUNT))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14643589159644727, 'fraction_numerical': 0.010703712138465042, 'mean_word_length': 3.9677601809954752, 'pattern_counts': {'":': 2, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '621384', 'license': 'MIT', 'n_tokens_mistral': 3046, 'n_tokens_neox': 2887, 'n_words': 871, 'path': '2.simulator_random.ipynb', 'repo_name': 'lfhohmann/wordle-Data-Analysis', 'size': '8782'}