<jupyter_start><jupyter_text>Make up some data<jupyter_code>X = np.random.randint(0, 100, (10, 5)) 
df = pd.DataFrame(X, columns=['Type A', 'Type B', 'Type C', 'Type D', 'Type E']) 
df<jupyter_output><empty_output><jupyter_text>Instantiate the ColumnCompleter object, and for the sake of maintaining a valid Python syntax, replace all spaces with an underscore.The value returned will be the original column name though.<jupyter_code>q = ColumnCompleter(df, space_filler='_')  # an underscore is the default value, but it's explicitly put here for pedagogical reasons
q.Type_A
q.Type_D<jupyter_output><empty_output><jupyter_text>It is convinient for indexing, in case Pandas won't autocomplte in some deeply nested calls (usually more nested than this one)<jupyter_code>df[q.Type_B]<jupyter_output><empty_output><jupyter_text>Creating the completion object with a DataFrame in which the columns starts or ends with spaces raises warnings.<jupyter_code>df = pd.DataFrame(X, columns=['type A ', 'type B  ', ' type C ', ' type D', 'type E']) 
q = ColumnCompleter(df)<jupyter_output><empty_output><jupyter_text>Unless `silence_warnings` are set to `True`<jupyter_code>q = ColumnCompleter(df, silence_warnings=True)
q.type_E
q.type_A_<jupyter_output><empty_output><jupyter_text>If the replacing of spaces causes column names to overlap, a ValueError is raised.<jupyter_code>df = pd.DataFrame(X, columns=['type A', 'type_A', 'type B', 'type C', 'type D']) 
q = ColumnCompleter(df)<jupyter_output><empty_output><jupyter_text>Using another value for `space_filler` such as two underscores, will resolve the problem in this particular case.<jupyter_code>q = ColumnCompleter(df, space_filler='__')
q.type__A
q.type_A<jupyter_output><empty_output><jupyter_text>A better approach will often be to rename the columns if the DataFrame, and the `ColumnCompleter` class provides a convinience function for doing just that.Note that the functionality is proviced by the _class_ not the _class instance_ (`q`)!<jupyter_code>df = pd.DataFrame(X, columns=['type A', 'type B', 'type C', 'type D', 'type E']) 
ColumnCompleter.replace_df_column_spaces(df, '_')<jupyter_output><empty_output><jupyter_text>Capatalizing column names can be helpful to distinguish them from the rest of Pandas' methods; capitilization can be achieved by setting `capatilize_first_letter=True`<jupyter_code>ColumnCompleter.replace_df_column_spaces(df, '_', capatilize_first_letter=True)<jupyter_output><empty_output><jupyter_text>A `ValueError` is raised if the rename causes a name collision.<jupyter_code>df = pd.DataFrame(X, columns=['type A', 'type_A', 'type B', 'type C', 'type D']) 
ColumnCompleter.replace_df_column_spaces(df, '_')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10833639368343738, 'fraction_numerical': 0.002570694087403599, 'mean_word_length': 7.647619047619048, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '182196', 'license': 'Unlicense', 'n_tokens_mistral': 934, 'n_tokens_neox': 911, 'n_words': 265, 'path': 'demonstration.ipynb', 'repo_name': 'AllanLRH/column_completer', 'size': '2723'}