<jupyter_start><jupyter_text>Otf has its own serialization library.<jupyter_code>otf.dump_text(v)<jupyter_output><empty_output><jupyter_text>We support shared references:<jupyter_code>v = []
otf.dump_text([v, 2, v])<jupyter_output><empty_output><jupyter_text>Shared referencesOut of the box, not many types are supportedIt's pretty easy to add support for new types. You need to write a "reduction" function that returns three values+ A callable *fn*+ A tuple *args* of values that are serialisable+ A dictionary *kwargs* where the keys are valid python identifiers and the values are serialisable<jupyter_code>@otf.register
def _reduce_complex(c: complex):
    return complex, (), {"real": c.real, "imag": c.imag}
otf.dump_text(complex(5))<jupyter_output><empty_output><jupyter_text>Executable format For cases where people might want to edit serialized values we even offer a format that can be ran in python<jupyter_code>c = complex(5, 6)
d = complex(6, 5)
print(otf.dump_text({c: d, d: c}, format=otf.EXECUTABLE))<jupyter_output>_1 = complex(real=5.0, imag=6.0)

_0 = complex(real=6.0, imag=5.0)

{_1: _0, _0: _1}<jupyter_text>Even though this format yields runnable python stubs we do not load it by executing it:<jupyter_code>otf.load_text(
    """
_1 = complex(real=5.0, imag=6.0)

_0 = complex(real=6.0, imag=5.0)

# This will cause an error when evaluated
_2 = 1 / 0.

{_1: _0, _0: _1}

"""
)<jupyter_output><empty_output><jupyter_text>Producers and reducerYou can convert between any of our format without going through an intermediate representation. This has the advange that it lets you edit any syntatically valid value:<jupyter_code>problematic_value = "{'stdin': my_lib.file_descriptor(0), 'stdout': my_lib.file_descriptor(1), 'stderr': ref(3)}"

print(
    otf.pack.reduce_text(
        problematic_value, acc=otf.pack.text.ExecutablePrinter()
    )
)<jupyter_output># There were errors trying to import the following constructors
#
# + 'my_lib.file_descriptor': ImportError Failed to find object

_0 = my_lib.file_descriptor(1)

{'stdin': my_lib.file_descriptor(0), 'stdout': _0, 'stderr': _0}<jupyter_text>Maybe you switch from my_lib to fd_lib and the format of the file descriptor has changed:<jupyter_code>fixed_value = """
# Fixed by till to run with fd_lib

STDIN = fd_lib.fd(1, mode='w')
STDOUT = fd_lib.fd(1, mode='w')
STDERR = STDOUT

{'stdin': STDIN, 'stdout': STDOUT, 'stderr': STDERR}
"""

print(otf.pack.reduce_text(fixed_value, acc=otf.pack.text.CompactPrinter()))<jupyter_output>{'stdin': fd_lib.fd(1, mode='w'), 'stdout': fd_lib.fd(1, mode='w'), 'stderr': ref(4)}
{'fraction_non_alphanumeric': 0.12783981517135157, 'fraction_numerical': 0.019252984212552945, 'mean_word_length': 6.137362637362638, 'pattern_counts': {'":': 2, '<': 26, '<?xml version=': 0, '>': 26, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '432525', 'license': 'CC0-1.0', 'n_tokens_mistral': 955, 'n_tokens_neox': 932, 'n_words': 260, 'path': 'docs/examples/serialisation.ipynb', 'repo_name': 'till-varoquaux/otf', 'size': '2597'}