<jupyter_start><jupyter_text>Reading in Data in Python This is a tutorial to get you started with reading in data in Python. Reading in data seems straightforward but can get pretty complicated depending on how your data is formatted. In this class each project group will be dealing with different data so the way that you end up reading in data might be different. Hopefully, this tutorial gives you a starting point as to how simple data is read so that you understand what you're doing when you touch more complicated formats.Some other online tutorials can be found below. These tutorials tend to use slightly different methods but all are valid ways to read in data. Some of these tutorials focus on reading in 'strings' (words) to Python which is also useful to know about:https://newcircle.com/s/post/1572/python_for_beginners_reading_and_manipulating_csv_filesopening-a-csv-filehttp://www.pythonforbeginners.com/files/reading-and-writing-files-in-pythonhttp://www.pythonforbeginners.com/systems-programming/using-the-csv-module-in-python/http://opentechschool.github.io/python-data-intro/core/text-files.htmlhigher-level, more astronomy specific: https://python4astronomers.github.io/files/asciifiles.html For this tutorial I will be reading data in from the "example.txt" file. The data in this file looks like:<jupyter_code>0   15
1   30
2   60
3   120
4   240
5   480<jupyter_output><empty_output><jupyter_text>When reading in data I find the numpy library to be particularly useful. So let's first load the numpy library.<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>I use numpy to read in data in two different ways using: loadtxt and genfromtxt. These are both great at reading in data when it's not too complicatedThe documentation for these two commands can be found here: https://docs.scipy.org/doc/numpy/reference/generated/numpy.loadtxt.htmlhttps://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html Let's first use loadtxt to read in our data<jupyter_code>data = np.loadtxt('example.txt')

print(data)<jupyter_output>[[   0.   15.]
 [   1.   30.]
 [   2.   60.]
 [   3.  120.]
 [   4.  240.]
 [   5.  480.]]<jupyter_text>Yay! We have successfully read in our data into Python! However, are data is not in a very usable form right now. For example, if I wanted to plot the second column versus the first column I wouldn't be able to do that right away. So now let's change the keywords that we use to read in data:<jupyter_code>data_x, data_y = np.loadtxt('example.txt', unpack = True)

print(data_x, data_y)<jupyter_output>[ 0.  1.  2.  3.  4.  5.] [  15.   30.   60.  120.  240.  480.]<jupyter_text>Here, I've used the keyward "unpack" which puts each of the columns in their own array. Using this keyword means that I cannot store all of my data in one variable anymore and I now have to have a variable for each of the columns given. Take a look at some of the other loadtxt keywords that you can use in the documentation above. Let's now use genfromtxt to read in the same data. genfromtxt is a very similar package to loadtxt, however, you can use it to read in more complicated data because it has more keywords!<jupyter_code>data_x, data_y = np.genfromtxt('example.txt', unpack = True)

print(data_x, data_y)<jupyter_output>[ 0.  1.  2.  3.  4.  5.] [  15.   30.   60.  120.  240.  480.]<jupyter_text>So far reading in the data is the same. However, genfromtxt can do some things that loadtxt cannot. For example, you can tell python how many rows you want to read in from your data. Here, I tell python to just read the first two rows<jupyter_code>data_x, data_y = np.genfromtxt('example.txt', unpack = True, max_rows=2)

print(data_x, data_y)<jupyter_output>[ 0.  1.] [ 15.  30.]
{'fraction_non_alphanumeric': 0.07856191744340879, 'fraction_numerical': 0.025565912117177098, 'mean_word_length': 4.195020746887967, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '643424', 'license': 'MIT', 'n_tokens_mistral': 1199, 'n_tokens_neox': 1121, 'n_words': 497, 'path': 'tutorials/Reading in Data in Python.ipynb', 'repo_name': 'adwasser/ucsc-astro-jupyterhub', 'size': '3755'}