<jupyter_start><jupyter_text>1.Overview of the dataframe<jupyter_code>import pandas
from datetime import datetime
from pytz import utc
data = pandas.read_csv("reviews.csv", parse_dates=['Timestamp'])
data.head()
data.shape
data.columns
data.hist('Rating')<jupyter_output><empty_output><jupyter_text>2. Selecting data from the dataframe  Select a column<jupyter_code>data['Rating']<jupyter_output><empty_output><jupyter_text>Select multiple columns<jupyter_code>data[['Course Name', 'Rating']]<jupyter_output><empty_output><jupyter_text>Select a row<jupyter_code>data.iloc[3]<jupyter_output><empty_output><jupyter_text>Selecting multiple rows<jupyter_code>data.iloc[1:3]<jupyter_output><empty_output><jupyter_text>Selecting a section<jupyter_code>data[['Course Name', 'Rating']].iloc[1:3]<jupyter_output><empty_output><jupyter_text>Selecting a cell<jupyter_code>data['Rating'].iloc[2]
data.at[2, 'Rating']<jupyter_output><empty_output><jupyter_text>3. Filtering data based on conditions  One condition<jupyter_code>data[data['Rating'] > 4]<jupyter_output><empty_output><jupyter_text>Multiple conditions<jupyter_code>data[(data['Rating'] > 4) & (data['Course Name'] == 'The Complete Python Course: Build 10 Professional OOP Apps')]<jupyter_output><empty_output><jupyter_text>4. Time-based filtering<jupyter_code>data[(data['Timestamp'] >= datetime(2020, 7, 1, tzinfo=utc)) & (data['Timestamp'] < datetime(2020, 12, 31, tzinfo=utc))]<jupyter_output><empty_output><jupyter_text>5. From data to information<jupyter_code>data.head()<jupyter_output><empty_output><jupyter_text>Average rating<jupyter_code>data['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Average rating for a particular course<jupyter_code>data[data['Course Name']=='The Python Mega Course: Build 10 Real World Applications']['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Average rating for a particular period<jupyter_code>data[(data['Timestamp'] >= datetime(2020, 1, 1, tzinfo=utc)) & (data['Timestamp'] < datetime(2020, 12, 31, tzinfo=utc))]['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Average rating for a particular period for a particular course<jupyter_code>data[(data['Timestamp'] >= datetime(2020, 1, 1, tzinfo=utc)) &
     (data['Timestamp'] < datetime(2020, 12, 31, tzinfo=utc)) &
    (data['Course Name']=='The Python Mega Course: Build 10 Real World Applications')
    ]['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Average of uncommented ratings<jupyter_code>data[data['Comment'].isnull()]['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Average of commented ratings<jupyter_code>data[data['Comment'].notnull()]['Rating'].mean()<jupyter_output><empty_output><jupyter_text>Number of uncommented ratings<jupyter_code>data[data['Comment'].isnull()]['Rating'].count()<jupyter_output><empty_output><jupyter_text>Number of commented ratings<jupyter_code>data[data['Comment'].notnull()]['Rating'].count()<jupyter_output><empty_output><jupyter_text>Number of comments containing a certain word<jupyter_code>data[data['Comment'].str.contains('accent', na=False)]['Rating'].count()<jupyter_output><empty_output><jupyter_text>Average of commented ratings with "accent" in comment<jupyter_code>data[data['Comment'].str.contains('accent', na=False)]['Rating'].mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15770501652147792, 'fraction_numerical': 0.018624211474917392, 'mean_word_length': 13.415584415584416, 'pattern_counts': {'":': 0, '<': 88, '<?xml version=': 0, '>': 90, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '149379', 'license': 'MIT', 'n_tokens_mistral': 1243, 'n_tokens_neox': 1150, 'n_words': 165, 'path': 'review_analysis/reviews.ipynb', 'repo_name': 'yingzhuo1994/Data-Analysis-Visualization', 'size': '3329'}