<jupyter_start><jupyter_text>Custom Operation To use your custom operation follow these steps:1. Write your own custom operation class by inheriting the Operation base class from Zokyo.2. Use the ArgClass from Zokyo to parse the operation's required arguments (any operation used must have a probability value).3. Import the operation class in the __init__.py file of your package (check the codes inside custom_package folder)4. Add your package path to sys.path.5. Add your package name and your custom operation class name to the config file and you are good to go.<jupyter_code>with open('configs/custom_op_config.json', 'r') as f:
    pprint(json.load(f)) 
# here we can see the operation_module is the custom_package and contains the custom operation class
builder_custom_op = Builder('configs/custom_op_config.json')
gen = builder_custom_op.process_and_generate(batch_size=2) 
# the custom operation is a dummy one used for example purposes and just returns the input as output
i = 0 
while True:
    try:
        res = next(gen) # each item in this batch is an instance of ZokyoData class
        for item in res:
            plt.figure()
            plt.axis('off')
            plt.imshow(item.image)
        i += 1

    except StopIteration:
        break<jupyter_output>2022-01-20 17:20:03,860 [Zokyo Builder] [INFO] sample factor : 3
2022-01-20 17:20:03,865 [Zokyo Builder] [INFO] batch size : 2
2022-01-20 17:20:03,885 [Zokyo Builder] [INFO] Entities num: 6
{'fraction_non_alphanumeric': 0.0727891156462585, 'fraction_numerical': 0.04217687074829932, 'mean_word_length': 3.870860927152318, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '229744', 'license': 'Apache-2.0', 'n_tokens_mistral': 465, 'n_tokens_neox': 429, 'n_words': 181, 'path': 'examples/custom_operation.ipynb', 'repo_name': 'toyotaconnected-India/zokyo', 'size': '1470'}