<commit_before>quickavro examples
==================

Reading an avro file
--------------------

quickavro receives the schema from Avro file header so there is no need to specify the schema when using :class:`quickavro.FileReader`.

.. code-block:: python

    import quickavro

    with quickavro.FileReader("example.avro") as reader:
        for record in reader.records():
            print(record)

Writing an avro file
--------------------

.. code-block:: python

    import quickavro

    records = [
        {"name": "Larry", "age": 21},
        {"name": "Gary", "age": 34},
        {"name": "Barry", "age": 27},
        {"name": "<NAME>", "age": 1134},
        {"name": "<NAME>", "age": None},
    ]

    with quickavro.FileWriter("example.avro") as writer:
        writer.schema = {
          "type": "record",
          "name": "Person",
          "fields": [
            {"name": "name", "type": "string"},
            {"name": "age",  "type": ["int", "null"]}
          ]
        }
        for record in records:
            writer.write_record(record)

Reading an avro file with BinaryEncoder
---------------------------------------

The :class:`quickavro.BinaryEncoder` can also be used directly for reading and writing Avro files directly for extra flexibility.

.. code-block:: python

    import quickavro

    with quickavro.BinaryEncoder() as encoder:
        encoder.schema = {
          "type": "record",
          "name": "Person",
          "fields": [
            {"name": "name", "type": "string"},
            {"name": "age",  "type": ["int", "null"]}
          ]
        }
        with open("example.avro", "r") as f:
            data = f.read()

        header, data = encoder.read_header(data)

        for record in encoder.read_blocks(data):
            print(record)

Writing an avro file with BinaryEncoder
---------------------------------------

.. code-block:: python
<commit_msg>Update docs<commit_after>quickavro examples
==================

Reading an avro file
--------------------

quickavro receives the schema from Avro file header so there is no need to specify the schema when using :class:`quickavro.FileReader`.

.. code-block:: python

    import quickavro

    with quickavro.FileReader("example.avro") as reader:
        for record in reader.records():
            print(record)

Writing an avro file
--------------------

.. code-block:: python

    import quickavro

    records = [
        {"name": "Larry", "age": 21},
        {"name": "Gary", "age": 34},
        {"name": "Barry", "age": 27},
        {"name": "<NAME>", "age": 1134},
        {"name": "<NAME>", "age": None},
    ]

    with quickavro.FileWriter("example.avro") as writer:
        writer.schema = {
          "type": "record",
          "name": "Person",
          "fields": [
            {"name": "name", "type": "string"},
            {"name": "age",  "type": ["int", "null"]}
          ]
        }
        for record in records:
            writer.write_record(record)

Reading an avro file with BinaryEncoder
---------------------------------------

The :class:`quickavro.BinaryEncoder` can also be used directly for reading and writing Avro files directly for extra flexibility.

.. code-block:: python

    import quickavro

    with quickavro.BinaryEncoder() as encoder:
        encoder.schema = {
          "type": "record",
          "name": "Person",
          "fields": [
            {"name": "name", "type": "string"},
            {"name": "age",  "type": ["int", "null"]}
          ]
        }
        with open("example.avro", "rb") as f:
            data = f.read()

        header, data = encoder.read_header(data)

        for record in encoder.read_blocks(data):
            print(record)

Writing an avro file with BinaryEncoder
---------------------------------------

.. code-block:: python

{'fraction_non_alphanumeric': 0.20651890482398957, 'fraction_numerical': 0.005215123859191656, 'mean_word_length': 1.8456973293768546, 'pattern_counts': {'":': 48, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4591828', 'n_tokens_mistral': 1105, 'n_tokens_neox': 1043, 'n_words': 323}