<commit_before>This is a (very) rough to-do list for this project. It will (hopefully) become more readable, more complete, and better organized soon.

TODO:
- docstrings in sql_database
- how should we store errors?
- once we've done that, how should we present them?
- sort data asc by pk
- consider merging SQLDatabase and RelationalData classes. They started out different, but as of a recent refactoring, they have quite similar structures.

- In what format should we collect the data discrepancies? These are currently stored in self.errors, but we should probably avoid using the term errors, which has specific connations in programming.
- How should we record errors in the matching_row method, which is run on the comparand during compare.
- Once we have collected our discrepancies, how should we present them?
- Can we turn our discrepancies into formats that will allow for updating the comparand to match the original?
- convert to Python 3

Relational Data
# Eventual TODO: account for different data types
# Ex: utf8 vs latin1
# Boolean 0 == False, etc
# Eventual TODO: suggest rows are the same based on similarity of contents.	
# Is there any reason why we would actually want that?<commit_msg>DOC: reorganize todo<commit_after>This is a (very) rough to-do list for this project. It will (hopefully) become more readable, more complete, and better organized soon.

TODO:
- In what format should we collect the data discrepancies? These are currently stored in self.errors, but we should probably avoid using the term errors, which has specific connotations in programming.
- How should we record errors in the matching_row method, which is run on the comparand during compare.
- Once we have collected our discrepancies, how should we present them?
- Can we turn our discrepancies into formats that will allow for updating the comparand to match the original?

- docstrings in sql_database
- how should we store errors?
- once we've done that, how should we present them?
- sort data asc by pk
- consider merging SQLDatabase and RelationalData classes. They started out different, but as of a recent refactoring, they have quite similar structures.

- convert to Python 3

Relational Data
# Eventual TODO: account for different data types
# Ex: utf8 vs latin1
# Boolean 0 == False, etc
# Eventual TODO: suggest rows are the same based on similarity of contents.	
# Is there any reason why we would actually want that?
{'fraction_non_alphanumeric': 0.044745484400656815, 'fraction_numerical': 0.003284072249589491, 'mean_word_length': 4.415555555555556, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2662268', 'n_tokens_mistral': 610, 'n_tokens_neox': 582, 'n_words': 371}