<jupyter_start><jupyter_text>How to add columns to a Pandas DataFrame, the basicsThis notebook will give some simple examples of adding columns to DataFrames and explain how to deal with some of the more complex scenarios raised.<jupyter_code>import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Let's start with a very simple DataFrame. This DataFrame has 4 columns of random floating point values. The index of this DataFrame will also be the default, a RangeIndex of the size of the DataFrame.<jupyter_code>df = pd.DataFrame(np.random.rand(6,4), columns=['a', 'b', 'c', 'd'])
display(df)
df.index<jupyter_output><empty_output><jupyter_text>Simple casesLet's start with the simplest way to add a column, such as a single value. This will be applied to all rows in the DataFrame.<jupyter_code>df['e'] = .5
df<jupyter_output><empty_output><jupyter_text>Now, under the hood, pandas is making life easier for you and taking your scalar value (the 0.5) and turning it into an array that matches the index (in this case a ```RangeIndex```) of your DataFrame.This is sort of the equivalent:<jupyter_code>df['e_prime'] = pd.Series(.5, index=pd.RangeIndex(6))
df<jupyter_output><empty_output><jupyter_text>You can also pass in an array yourself without an index, but it must match the dimensions of your DataFrame.<jupyter_code>df['f'] = np.random.rand(6,1)
df<jupyter_output><empty_output><jupyter_text>If you try to do this with a non-matching shape, it won't work. This is because the DataFrame won't know where to put the values.<jupyter_code>try:
    df['g'] = np.random.rand(5,1)
except Exception as ex:
    print(ex)<jupyter_output>Length of values (5) does not match length of index (6)<jupyter_text>Non-matching indicesNow what happens when the data you want to add doesn't match your current DataFrame? Specifically, what if the index is different on the right hand side?<jupyter_code>df['g'] = pd.Series(np.random.rand(50), index=pd.RangeIndex(2,52))
df<jupyter_output><empty_output><jupyter_text>So what happened here? Our column ```g``` only has values at rows 2 through 5, even though we assigned a series with 50 values. Well, these were the rows that matched our index. For the rows that didn't have values, a ```NaN``` was inserted.Another way to think of this is that we could use the ```loc``` method to select the rows we wanted to update, but unless we set the index on the right hand side, we still need to align with the shape of the DataFrame.<jupyter_code>df.loc[2:5, 'g_prime'] = np.random.rand(4)
df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09193173002327386, 'fraction_numerical': 0.01008533747090768, 'mean_word_length': 5.716145833333333, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '262237', 'license': 'CC0-1.0', 'n_tokens_mistral': 813, 'n_tokens_neox': 787, 'n_words': 333, 'path': 'pandas/adding_columns.ipynb', 'repo_name': 'wrighter/python_blogposts', 'size': '2578'}