<commit_before>Longer term Roadmap
===================

1. Dataframe is key underlying datatype (as opposed to Matrix{Float64})
2. Performance benchmarking and optimization
3. Robust sampling methods
4. Build out metrics, potentially break out as separate repo
5. Testing against more real world datasets

Short-term TODO's
=================

1. More unit tests
   - likelihood under different observations for bart.jl
   - split.jl
2. Graphical output for experiments
3. Experiments with multiple datasets
4. Support categorical predictors
5. Track & optionally show metadata and state during training 
6. Support minibatch sgd during neural net training
7. Supervised regression support for neural nets

Experiments To Run
==================

1. Impact of the jump distribution on time to convergence
2. Convergence of MCMC results<commit_msg>updated todos<commit_after>Longer term Roadmap
===================

1. Dataframe is key underlying datatype (as opposed to Matrix{Float64})
2. Performance benchmarking and optimization
3. Robust sampling methods
4. Build out metrics, potentially break out as separate repo
5. Testing against more real world datasets

Short-term TODO's
=================

1. Partial/sensitivity refactor + unit tests
2. More unit tests
   - likelihood under different observations for bart.jl
   - split.jl
3. Graphical output for experiments
4. Experiments with multiple datasets
5. Support categorical predictors
6. Track & optionally show metadata and state during training 
7. Support minibatch sgd during neural net training
8. Supervised regression support for neural nets

Experiments To Run
==================

1. Impact of the jump distribution on time to convergence
2. Convergence of MCMC results
{'fraction_non_alphanumeric': 0.09740634005763689, 'fraction_numerical': 0.019020172910662825, 'mean_word_length': 4.825503355704698, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1936396', 'n_tokens_mistral': 433, 'n_tokens_neox': 397, 'n_words': 221}