<jupyter_start><jupyter_text>Understanding Descriptive StatisticsImport the necessary libraries here:<jupyter_code># Libraries


import numpy as np
import pandas as pd
import random
from random import randint
import matplotlib<jupyter_output><empty_output><jupyter_text>Challenge 1 1.- Define a function that simulates rolling a dice 10 times. Save the information in a dataframe.**Hint**: you can use the *choices* function from module *random* to help you with the simulation.<jupyter_code># your code here

# ANSWER 1/2

random.seed(29)
a = np.random.randint(1,6,10)
a
import numpy as np
import random # random number generators
# RNG = pseudo random numbers (generators).
# the thing that fixes the numbers are random.seed(_) # the seed fixes the numbers. the same sequence of numbers
# example:

# np.random.rand(10,3)

#df = pd.DataFrame(a, columns=list('R'))
#df
# ANSWER 2/2

df = pd.DataFrame(a, columns=['Rolled_Outcome'])
df.index = pd.RangeIndex(start=1, stop=11, step=1)
df
# ANSWER 3/3
# named new column 'index'

indexes = [('Number Roll' + str(x)) for x in range(1,11)]
df['Index'] = indexes
df.set_index('Index')

# delete column: del df['column_name']
# another way 

#import random

#print([random.randint(1,6) for i in range(10)])
# df = pd.DataFrame(np.random.randint(1,6,10), columns=list('R')) # REPLACED np.random.randint(1,6,10) with the variable i gave it 'a'
# df
#df = pd.DataFrame(np.random.randint(1,6,10), columns=['Rolled_Outcome'])
#df<jupyter_output><empty_output><jupyter_text>2.- Plot the results sorted by value.<jupyter_code># your code here
# ANSWER

df.sort_values(by=list(df.columns),axis=0)<jupyter_output><empty_output><jupyter_text>3.- Calculate the frequency distribution and plot it. What is the relation between this plot and the plot above? Describe it with words.<jupyter_code># your code here

# ANSWER 

import matplotlib
%matplotlib inline

df.boxplot()

# ANSWER 

df.hist()
"""
your comments here:


It displays the outliers the same in both. showing the middle values as the more usual/common results.


"""<jupyter_output><empty_output><jupyter_text>Challenge 2Now, using the dice results obtained in *challenge 1*, your are going to define some functions that will help you calculate the mean of your data in two different ways, the median and the four quartiles.  1.- Define a function that computes the mean by summing all the observations and dividing by the total number of observations. You are not allowed to use any methods or functions that directly calculate the mean value.<jupyter_code># your code here

#df.describe()

df.mean()
# not allowed to use any methods or functions that directly calculate the mean value
#sum the column values and divide by length of it


# ANSWER 1/3

sum = df['Rolled_Outcome'].sum()
sum
# ANSWER 2/3

length = df['Rolled_Outcome'].count()
length
# ANSWER 3/3

mean = (sum / length)
mean<jupyter_output><empty_output><jupyter_text>2.- First, calculate the frequency distribution. Then, calculate the mean using the values of the frequency distribution you've just computed. You are not allowed to use any methods or functions that directly calculate the mean value.<jupyter_code># your code here

df.describe()
# ANSWER 1/2

frequency_dist = df.apply(pd.value_counts)
frequency_dist
sum2 = frequency_dist['Rolled_Outcome'].sum()
sum2
length2 = frequency_dist['Rolled_Outcome'].count()
length2
mean2 = (sum / length)
mean2<jupyter_output><empty_output><jupyter_text>3.- Define a function to calculate the median. You are not allowed to use any methods or functions that directly calculate the median value. **Hint**: you might need to define two computation cases depending on the number of observations used to calculate the median.<jupyter_code># your code here

sort = df.sort_values(by=list(df.columns),axis=0)
sort
def median(sort):
    srt = sorted(sort)
    mid = len(sort)//2
    if len(sort) % 2: # if list length mid 2 has a remainder the list is an odd length
            return srt[mid]
    else:
        median = (srt[mid] + srt[mid-1]) / 2  # in a list [1,2,3,4] srt[mid]-> 2, srt[mid-1] -> 3
        return median
median<jupyter_output><empty_output><jupyter_text>4.- Define a function to calculate the four quartiles. You can use the function you defined above to compute the median but you are not allowed to use any methods or functions that directly calculate the quartiles.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>Challenge 3Read the csv `roll_the_dice_hundred.csv` from the `data` folder. 1.- Sort the values and plot them. What do you see?<jupyter_code># your code here

import numpy as np
import pandas as pd

roll = pd.read_csv('/Users/gracemartinez/ironhack/daft-miami-1019-labs/module-2/Descriptive-Stats/data/roll_the_dice_hundred.csv')
roll.dtypes # so the data is all numerical
roll.describe()
roll.shape
list(roll.columns) 
df = pd.DataFrame(roll, columns=['Unnamed: 0', 'roll', 'value'])
df
sort_values = roll.sort_values(by=['value'])
sort_values

import matplotlib
%matplotlib inline

roll.boxplot()
"""
your comments here:


I see a big discrepancy of the 'value' column versus the other two.
"""<jupyter_output><empty_output><jupyter_text>2.- Using the functions you defined in *challenge 2*, calculate the mean value of the hundred dice rolls.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>3.- Now, calculate the frequency distribution.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>4.- Plot the histogram. What do you see (shape, values...) ? How can you connect the mean value to the histogram?<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>5.- Read the `roll_the_dice_thousand.csv` from the `data` folder. Plot the frequency distribution as you did before. Has anything changed? Why do you think it changed?<jupyter_code># your code here

import numpy as np
import pandas as pd

roll2 = pd.read_csv('/Users/gracemartinez/ironhack/daft-miami-1019-labs/module-2/Descriptive-Stats/data/roll_the_dice_thousand.csv')
roll2
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>Challenge 4In the `data` folder of this repository you will find three different files with the prefix `ages_population`. These files contain information about a poll answered by a thousand people regarding their age. Each file corresponds to the poll answers in different neighbourhoods of Barcelona. 1.- Read the file `ages_population.csv`. Calculate the frequency distribution and plot it as we did during the lesson. Try to guess the range in which the mean and the standard deviation will be by looking at the plot.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>2.- Calculate the exact mean and standard deviation and compare them with your guesses. Do they fall inside the ranges you guessed?<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>3.- Now read the file `ages_population2.csv` . Calculate the frequency distribution and plot it.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>4.- What do you see? Is there any difference with the frequency distribution in step 1?<jupyter_code>"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>5.- Calculate the mean and standard deviation. Compare the results with the mean and standard deviation in step 2. What do you think?<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>Challenge 5Now is the turn of `ages_population3.csv`. 1.- Read the file `ages_population3.csv`. Calculate the frequency distribution and plot it.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>2.- Calculate the mean and standard deviation. Compare the results with the plot in step 1. What is happening?<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>3.- Calculate the four quartiles. Use the results to explain your reasoning for question in step 2. How much of a difference is there between the median and the mean?<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>4.- Calculate other percentiles that might be useful to give more arguments to your reasoning.<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output><jupyter_text>Bonus challengeCompare the information about the three neighbourhoods. Prepare a report about the three of them. Remember to find out which are their similarities and their differences backing your arguments in basic statistics.<jupyter_code># your code here
"""
your comments here
"""<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09312663855009688, 'fraction_numerical': 0.013222386868802007, 'mean_word_length': 5.432551319648094, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 95, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '19307', 'license': 'MIT', 'n_tokens_mistral': 2780, 'n_tokens_neox': 2664, 'n_words': 1015, 'path': 'module-2/Descriptive-Stats/your-code/main.ipynb', 'repo_name': 'gracegm/daft-miami-1019-labs', 'size': '8773'}