<jupyter_start><jupyter_text>DataframesData frames are the central object of most data science workflows. This notebook shows some helper function that can assist you in creating them from judgements. The older non-dataframe way of passing data is in most of the example notebooks, so use this code anywhere you see that pattern.<jupyter_code>import ltr.judgments as judge
classic_training_set = [j for j in judge.judgments_from_file(open('data/classic-training.txt'))]
classic_df = judge.judgments_to_dataframe(classic_training_set)
classic_df<jupyter_output><empty_output><jupyter_text>PlottingIs one of the main reasons dataframes are easier to work with.  There are two helper functions to show the distribtion of grade (`plot_grades`) and relationship between features and grades (plot_features).You are encouraged to use whatever python plotting library you are most comformtable with, we have `matplotlib` and `plotnine` installed in the Docker image.<jupyter_code>import ltr.p9_plots as plots
plots.plot_grades(classic_df)
classic_df_long = judge.judgments_dataframe_to_long(classic_df)
classic_df_long
plots.plot_features(classic_df_long)
plots.plot_features<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.055973266499582286, 'fraction_numerical': 0.000835421888053467, 'mean_word_length': 7.2620689655172415, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '176278', 'license': 'Apache-2.0', 'n_tokens_mistral': 363, 'n_tokens_neox': 346, 'n_words': 123, 'path': 'notebooks/elasticsearch/tmdb/Dataframes.ipynb', 'repo_name': 'risdenk/hello-ltr', 'size': '1197'}