<jupyter_start><jupyter_text>Business UnderstandingTED was founded in 1984 by <NAME> who have observation of a powerful convergencce among three fields: technology, entertainment and design. The first six TED Talks was posted online on Jun 27, 2006 and they had reached more than one million views by September 2006. In 2007, TED talks was so popular that the website was relaunched around them, gving a global audience free access to some of the world's greatest thinkers, leadders and teachers. TED has a mission of spreading ideas. Ted is global community, welcoming people from every discipline and culture who seek a deeper understanding of the world. Since the time I begin watching TED talks in my college, I have learned an incredible amount, from different fields and cultures. I wanted to finding an insight about the world of TED on its speakers and viewers to answer a few question that I had. 1. Which are the most viewed and most favorited Talks of all time?2. What kind of topics attract the maximum discussion and debate(in the form of comments)?3. Who have managed to inspire millions of people through their talks on the TED Platform?  Data Understanding<jupyter_code>#import libraries

%matplotlib inline
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
#reading the data
ted = pd.read_csv("E:/Udacity document/TED talks/ted_main.csv")
ted.head()
ted.info()
#write a function that show rows and columns for the dataframe
def find_DataVolume(dataframe):
    '''Input: data frame
        Output: print the dataframe rows and columns'''
    print('This dataframe has', dataframe.shape[0],'rows and',dataframe.shape[1],'columns')
#Finding the number of rows and columns for each dataframe:
find_DataVolume(ted)<jupyter_output>This dataframe has 2550 rows and 17 columns<jupyter_text>Prepare Data and Clean<jupyter_code>#count the number of missing values in each column
missing_value=ted.isnull().sum()
missing_value<jupyter_output><empty_output><jupyter_text>We can see that there are 6 missing values in speaker_occupation<jupyter_code>#fill the missing values with mean value
ted['speaker_occupation']=ted.speaker_occupation.fillna(ted.speaker_occupation.mode()[0])
#count NaN values
ted.isna().sum()
#count categorical data
ted_cat=ted.select_dtypes('object')
print('There are', ted_cat.shape[1],'categorical columns out of 17 columns')
#see the columns of ted
ted.columns
#check columns has no missing values
no_nulls=set(ted.columns[ted.isnull().mean()==0])
print(no_nulls)
#reorder the columns
ted_arrange=ted[['name','title','description','main_speaker','speaker_occupation','num_speaker','duration','event','film_date','published_date','comments','tags','languages','ratings','related_talks','url','views']]
#fix the datetime
pd.to_datetime(ted_arrange['film_date'])
pd.to_datetime(ted_arrange['published_date'])
ted_arrange.head(5)
ted_arrange.describe()<jupyter_output><empty_output><jupyter_text>Q1 Which are the most viewed and most favorited Talks of all time?<jupyter_code>#top 10 views of Ted talks
popular_talks=ted_arrange[['title','main_speaker','views','film_date']].sort_values('views', ascending=False)
popular_talks.head(10)<jupyter_output><empty_output><jupyter_text>1. <NAME>'s talk on Do Schools Kill Creativity? is the most popular Ted Talk of all time with 4.72million views. 2. Robinson's talk is closely followed by <NAME>'s Your body language may shape who you are. 3. There are only 2 talks that have surpassed 40million mark. Robinson's talk was filmed on 2006 and Cuddy's talk was filmed on 2012. 4. There are 2 talk that havve surpassed 30million mark which is <NAME>'s How great leaders inspire action and <NAME>'s The power of vulnerability.<jupyter_code>speaker=popular_talks['main_speaker'].head(10)
views=popular_talks['views'].head(10)

#figure size
fig = plt.figure(figsize=(10,6))

#Bar plot
plt.bar(speaker, views)
plt.xlabel('Speaker')
plt.ylabel('views(million)')
plt.xticks(rotation=90)
plt.show()
ted_arrange['views'].describe()<jupyter_output><empty_output><jupyter_text>The average number of view on TED Talks in 1.6million and the median number of views is 1.12million. We also noticed that the majority of talks have less than 4million. This will be consider as the cutoff poiint when construction box plots.   Q2 What kind of topics attract the maximum discussion and debate(in the form of comments)?<jupyter_code>ted['comments'].describe()<jupyter_output><empty_output><jupyter_text>1. There are average 191.5 comments on every TED Talks. We can said that the TED Online Community is highly involved in discussion revolving TED Talks. 2. There is huge standard deviation associated witht the comments. It is larger than the mean suggesting that the measures may be sensitive to outliers. 3. The minimum number of comments on TED talks is 2 while the maximum is 6404.<jupyter_code>#plot heatmap to see the relationship between variables
fig, ax=plt.subplots(figsize=(10,6))
sns.heatmap(ted.corr(), center=0, cmap='BrBG', annot=True)<jupyter_output><empty_output><jupyter_text>From the heatmap, the number of views have positive relationship with language and comments. the number of views have negative relationship with published date and number of speaker.<jupyter_code>#plot scatterplot to see the relationship between total number of views and total number of comments
fig, ax =plt.subplots(figsize=(10,6))
sns.regplot(data=ted,x='views',y='comments',ax=ax)
#check number of views and comments on the 10most commented TED talks of all time.
ted[['title','main_speaker','views','comments']].sort_values('comments', ascending=False).head(10)<jupyter_output><empty_output><jupyter_text>1. <NAME>'s talk on Militant Atheism generated the greatest amount of discussion and opions but having significantly lesser views than <NAME>'s talk. Which talks tend to attract the largest amount of discussion?<jupyter_code>#creating a new column 'comments_per_view'
ted['comments_per_view']=ted.comments/ted.views

ted[['title','main_speaker','views','comments','comments_per_view']].sort_values('comments_per_view',ascending=False).head(10)<jupyter_output><empty_output><jupyter_text>This analysis raised interesting insight. Half of the talks in the top 10 are on the lines of Faith and Region. Faith and Region might be a very hot debated topic in 21st century. The most discussed talk is The case for same-sex marriage by <NAME> which has religious undertones.   Q3 Who have managed to inspire millions of people through their talks on the TED Platform?<jupyter_code>#most popular speakers give the most talks 
speaker_df=ted.groupby('main_speaker').count().reset_index()[['main_speaker','comments']]
speaker_df.columns=['main_speaker','appearances']
speaker_df=speaker_df.sort_values('appearances',ascending=False)
speaker_df.head(10)<jupyter_output><empty_output><jupyter_text><NAME> is the most popular TED speaker with 9 appearances on the TED Talks. <NAME> has 7 appearance. Rives and March Tempest has 6 appearance on TED Talks.<jupyter_code>#create new dataframe for speaker_occupation and appearances
occupation_df=ted.groupby('speaker_occupation').count().reset_index()[['speaker_occupation','comments']]
occupation_df.columns=['occupation','appearances']
occupation_df=occupation_df.sort_values('appearances', ascending=False)
occupation_df.head()
#plot bar chart for occupation and appearances
x=occupation_df['occupation'].head(10)
y=occupation_df['appearances'].head(10)
plt.figure(figsize=(10,6))
plt.bar(x,y)
plt.xlabel('occupation')
plt.ylabel('appearances')
plt.xticks(rotation=45)
plt.show()<jupyter_output><empty_output><jupyter_text>1.Writers are the most popular with 50 speakers identifying themselves.2. Artist and Desginer are the second most popular with 34 speakers in each category. Do some professions tend to attract a larger number of views?<jupyter_code>#plot boxplot
occupation=['Writer','Artist','Designer','Journalist','Entrepreneur','Architect','Inventor','Psychologist','Photographer','Filmmaker']
vclasses=pd.api.types.CategoricalDtype(ordered=True, categories=occupation)

ted['speaker_occupation']=ted['speaker_occupation'].astype(vclasses)

fig, ax= plt.subplots(nrows =1, ncols=1,figsize= [20,15])
bx= sns.boxplot(data=ted[ted['speaker_occupation'].isin(occupation_df.head(10)['occupation'])], x='speaker_occupation',y='views', palette="muted")
plt.xticks(rotation=15)
bx.set(ylim=[0,  0.9e7])
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09988207547169811, 'fraction_numerical': 0.01686320754716981, 'mean_word_length': 7.186293436293436, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 58, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '104896', 'license': 'CNRI-Python', 'n_tokens_mistral': 2662, 'n_tokens_neox': 2488, 'n_words': 870, 'path': 'TED Talks.ipynb', 'repo_name': 'carmondc/Ted-Talks', 'size': '8544'}