<commit_before># Djangae Model Fields

## ListField / SetField

These allows you to store a list/set of values (strings, floats, integers dates) in a single field.
This is often useful when structuring data for a non-relational database. [See example usage](fields.md#example-usages).

### `ListField(item_field_type, **kwargs)`


* `item_field_type`: An instance of a Django model field which defines the data type and validation for each item in the list.
* `ordering`: A callable which allows the items in the list to be automatically sorted.


### `SetField(item_field_type, **kwargs)`

* `item_field_type`: An instance of a Django model field which defines the data type and validation for each item in the list.


Both fields also accept the following standard Django model field kwargs:

* `default`: works as normal.
* `blank`: works as normal.
* `null`: this cannot be set to `True`.  The App Engine Datastore does not allow us to differentiate between `[]` and `None`, so we stick with the former.
* `validators`: works as normal, but note that it's important where you pass this argument; if you pass it to the `ListField` or `SetField` then the validators operate on the list/set as a whole, but if you pass it to the nested model field then the validators operate on each individual value.
* `choices`: this works as normal, but should be passed to the nested model field, not to the `ListField`/`SetField` itself.  In a form this results in a multiple select field.
* `unique`: works as normal, but note that this relates to the uniqueness across different instances, not uniqueness within the list of values.  If you just want to remove duplicates from the list of values in the field then use `SetField`; setting `unique=True` will check that no 2 objects contain *any* of the same values for this field.
* `editable`: works as normal.
* `help_text`: works as normal.
* `verbose_name`: works as normal.
<commit_msg>Updated documentation regarding IteratorField max_length kwarg<commit_after># Djangae Model Fields

## ListField / SetField

These allows you to store a list/set of values (strings, floats, integers dates) in a single field.
This is often useful when structuring data for a non-relational database. [See example usage](fields.md#example-usages).

### `ListField(item_field_type, **kwargs)`


* `item_field_type`: An instance of a Django model field which defines the data type and validation for each item in the list.
* `ordering`: A callable which allows the items in the list to be automatically sorted.


### `SetField(item_field_type, **kwargs)`

* `item_field_type`: An instance of a Django model field which defines the data type and validation for each item in the list.


Both fields also accept the following standard Django model field kwargs:

* `default`: works as normal.
* `blank`: works as normal.
* `null`: this cannot be set to `True`.  The App Engine Datastore does not allow us to differentiate between `[]` and `None`, so we stick with the former.
* `validators`: works as normal, but note that it's important where you pass this argument; if you pass it to the `ListField` or `SetField` then the validators operate on the list/set as a whole, but if you pass it to the nested model field then the validators operate on each individual value.
* `choices`: this works as normal, but should be passed to the nested model field, not to the `ListField`/`SetField` itself.  In a form this results in a multiple select field.
* `unique`: works as normal, but note that this relates to the uniqueness across different instances, not uniqueness within the list of values.  If you just want to remove duplicates from the list of values in the field then use `SetField`; setting `unique=True` will check that no 2 objects contain *any* of the same values for this field.
* `editable`: works as normal.
* `help_text`: works as normal.
* `verbose_name`: works as normal.
* `max_length`: limits the number of items allowed in the list/set.

{'fraction_non_alphanumeric': 0.07288263382759487, 'fraction_numerical': 0.0005026388539834129, 'mean_word_length': 4.582047685834502, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7169775', 'n_tokens_mistral': 1087, 'n_tokens_neox': 1023, 'n_words': 599}