<jupyter_start><jupyter_text>DataWe are going to be using the same data set as last Thursday, the data is from the Incident Record-Type File of the NCVS. See page 31 of the codebook (on bCourses) for a description of the survey.<jupyter_code>incidents = Table.read_table('../data/incident/36828-0004-Data.tsv', delimiter='\t')
incidents<jupyter_output><empty_output><jupyter_text>Proportion of Incidents that are Hate Crimes Use the code below to find the proportion of hate crimes in the dataset.<jupyter_code>sum(incidents.column('V4526AA') == 1) / incidents.num_rows<jupyter_output><empty_output><jupyter_text>Value of Property Lost Now find the incidents where there were property losses (more than $0), and save the result in a variable "nonzero_loss"<jupyter_code>loss = incidents.where('V4364', are.below(99997))
nonzero_loss = loss.where("V4364", are.above(0))
np.mean(nonzero_loss.column('V4364'))<jupyter_output><empty_output><jupyter_text>Histograms Histograms are a nifty way to display quantitative information. The x-axis is typically a quantitative variable of interest, and the y-axis is generally a frequency. Plot a histogram of the losses, and then experiment with the bin sizes.<jupyter_code>loss.hist('V4364')
loss.hist('V4364', bins=range(0,25100,100))<jupyter_output><empty_output><jupyter_text>Question 1: Histograms What happens as you increase the number of bins to 100? Does having too many bins hinder interpretation?  Scatter Plots Scatter plots are generally used to relate two variables to one another. They can be useful when trying to infer relationships between variables, visualize simple regressions, and get a general sense of the "spread" of your data. Run the following code to plot the value of the property after it was recovered ("V4385") vs. the value of the property before it was taken ("V43634").<jupyter_code>recover_loss = loss.where("V4385", are.below(99997))
recover_loss.scatter("V4364", "V4385")<jupyter_output><empty_output><jupyter_text>Question 2: Scatterplots Looking at the scatterplot above, what can you infer about property values before and after a robbery? What would you expect this plot to look like if property never lost value after a robbery?  Practice with Plots Practice on your own! Try plotting a histogram and a scatterplot of some variables that you find interesting in the data, then interpret them. Optionally, also try to make a barplot and line graph!  Probability and Randomness Next, we'll turn to exercises with probability and randomness in Python. First, let's review how to simulate a coin flip. The following code flips a fair coin 5 times, and saves the results.<jupyter_code># Create an object that contains the outcome space, "Heads," and "Tails"
coin = make_array('Heads', 'Tails')

# Make an array that will hold the results
tosses = make_array()

# Use a for loop to flip the coin, store its results, then do it again 5 times
for i in np.arange(5):
    tosses = np.append(tosses, np.random.choice(coin))

#Return the results
tosses<jupyter_output><empty_output><jupyter_text>Exercise 3: Simulate a Dice Roll Write your own code that simulates a fair dice, and roll the dice 100 times. Plot your results in a histogram. Do the results match what you thought would happen?<jupyter_code>dice = make_array(...)

rolls = make_array()

for i in np.arrage(100):
    rolls = np.append(...)

# Histogram code here<jupyter_output><empty_output><jupyter_text>Probability Sampling In the last lab, we learned how to take deterministic samples with the "take" and "where" functions. Now we're going to learn how to do take random samples from the data.<jupyter_code>"""Choose a random start among rows 0 through 9;
then take every 10th row."""

start = np.random.choice(np.arange(10))
incidents.take(np.arange(start, incidents.num_rows, 10))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0803014553014553, 'fraction_numerical': 0.02624740124740125, 'mean_word_length': 5.659169550173011, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '177116', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1208, 'n_tokens_neox': 1116, 'n_words': 480, 'path': 'labs/Archived Labs/1-23/1-23.ipynb', 'repo_name': 'jdmarshl/Legal-123-Sp20', 'size': '3848'}