<jupyter_start><jupyter_text>Using the `split` methodAs explained before, `StorageCollection` subtypes have a `split` method that can be used.It is specially useful for distributed storage and parallel computation (you will see during this afternoon!) but we will start to showcase it through a simple sequential exercise with the WordCount application.<jupyter_code>import storage.api
storage.api.init()

from model.words import Words
from model.result import Result<jupyter_output>--- Using global.properties at /gpfs/home/nct00/nct00004/dataclay-patc21/wordcount/cfgfiles/global.properties
No global.properties file found. Using default values
--- Using global.properties at /gpfs/home/nct00/nct00004/.dataClay/13911894/client/cfgfiles/global.properties
Found global.properties variable EE_PERSISTENT_INFO_PATH=/home/nct00/nct00004/.dataClay/13911894/client/storage/ and type <class 'str'>
Found global.properties variable STATE_FILE_PATH=/home/nct00/nct00004/.dataClay/13911894/client/storage/state.txt and type <class 'str'><jupyter_text>Prepare the helper functionsPrepare the following functions. You may --you are encouraged to-- reuse existing code from previous exercises:<jupyter_code>def wordcount(block):
    """Count the occurrences of unique words.
    
    :param block: A StorageDict block, coming from a text.
    :return: A dictionary of <word, occurrences>.
    """
    ...
    
    # ****************************
    # Solution
    # ****************************
    partial_result = dict()
    for word in block.values():
        if word in partial_result:
            partial_result[word] += 1
        else:
            partial_result[word] = 1
    return partial_result
    # ****************************


def merge_results(partial, result):
    """Given a partial dictionary of results, merge it into the final result.
    
    :param partial: A partial wordcount.
    :param result: The final Result instance (modified in-place).
    """
    ...
    
    # ****************************
    # Solution
    # ****************************
    for word, instances in partial.items():
        if word in result:
            result[word] += instances
        else:
            result[word] = instances
    # ****************************<jupyter_output><empty_output><jupyter_text>Prepare the mainYou will need to use the `split` method and also use the previous two functions that you have prepared in the previous cell.<jupyter_code># Retrieve the persistent Words object
words = ...
# Prepare the output
result = ...

for block in words.split():
    ...

# ****************************
# Solution
# ****************************

words = Words.get_by_alias("WordCountWords")
result = Result()

for block in words.split():
    partial = wordcount(block)
    merge_results(partial, result)
    
print("Number of unique words:", len(result))

# ****************************<jupyter_output>Number of unique words: 186
{'fraction_non_alphanumeric': 0.1696763202725724, 'fraction_numerical': 0.02010221465076661, 'mean_word_length': 3.942760942760943, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '121642', 'license': 'Apache-2.0', 'n_tokens_mistral': 840, 'n_tokens_neox': 783, 'n_words': 251, 'path': 'wordcount-solutions/04-split-usage.ipynb', 'repo_name': 'bsc-dom/dataclay-patc21', 'size': '2935'}