<jupyter_start><jupyter_text>Python Introduction Setting up pythonYour machine should come preloaded with python.  You can check whether your machine has python with the following command:```$ which python/usr/bin/python$ python --versionPython 2.7.12```However, if it does not, you can install python (on an ubuntu machine) with the following command:```sudo apt-get install -y python2-dev```or on mac following [this guide](http://docs.python-guide.org/en/latest/starting/install/osx/). Setting up PipPip allows for the easy installation of python packages. To check if your machine has pip, run the following commands:```$ which pip/home/eric/.local/bin/pip$ pip --versionpip 9.0.1```If you did not see an installed version of pip with the previous commands, you can install pip as follows:```$ wget https://bootstrap.pypa.io/get-pip.py$ python get-pip.py   installs pip``` Installing some Basic PackagesIt is often useful to set up some default packages to python that all of our environments will likely need. In a minute, we will see why it is bad to install ALL packages here, but it is a good idea to get packages that most software will use on the base python installation:```$ pip install [packages]``` Setting up VirtualenvVirtual-env allows you to set up virtual environments within python seamlessly. With a language like python with thousands of available packages, it is disadvantageous to install all the packages at once in one environment, as python gets "heavy" (slow) when there are too many packages installed. To combat this issue, python provides virtual environments. Basically, a virtual environment adds to the base python environment an extension of packages. This means that you get all the packages installed to the base python environment, and also those installed only to your virtual environment. This allows you to have different python environments for different pieces of software. For example, one package might require numpy version 0.1.12, but another might require version 0.1.13. With virtual environments, you can have both on your system fairly seamlessly. Install virtualenv with the following command on ubuntu:```sudo apt-get install virtualenv```or the following command on mac:```pip install virtualenv``` Creating a new VirtualEnvMake a directory describing the name of your project. For example, create a new directory somewhere for this tutorial:```$ mkdir python-tutorial$ cd python-tutorial$ virtualenv env-tutorial -p python   make a new python env called env-tutorial with the default python interpreter$ source env-tutorial/bin/activate   set the environment to active(env-tutorial) $   note that the environment is active to the left```Note that you can specify any p|ython environment here, so if you instead for instance wanted a python3 environment, you could specify that with:```virtualenv env-tutorial -p python3   specify python3virtualenv env-tutorial -p /path/to/python/interpreter   specify an interpreter of your choice``` Installing packages to the virtualenvOnce the environment is sourced and set to active, your default "python" and "pip" commands point to the appropriate python/pip for the environment. We can install some basic packages as follows:```(env-tutorial) $ pip install numpy scipy jupyter ipython plotly colorlover``` Exiting the virtualenvTo inactivate the virtual environment, run the following command:```(env-tutorial)$ deactivate$   note that the environment is gone``` Python TutorialIn the below tutorial, we will discuss how to make basic sample data using scipy and numpy, and show some basic plots one might want to construct using plotly. First, we will write a basic function to create normally distributed timeseries data:<jupyter_code>import numpy as np
import plotly as py
import colorlover as cl
from IPython.display import HTML
from collections import OrderedDict
from plotly.offline import download_plotlyjs, init_notebook_mode, iplot, plot
from plotly import tools
import plotly.plotly as py
from plotly.graph_objs import *

init_notebook_mode(connected=True)


def make_timeseries(nf, no):
    """
    A function to create random normally distributed timeseries, of dimensions [nf x no].
    
    Params:
        - nf: the number of features.
        - no: the number of observations of each feature.
    """
    ar = np.zeros((nf, no)) # initialize the empty array
    # looping over the features
    for i in range(0, nf):
        ar[i,:] = np.random.normal(size=no, loc=i)  # generate data with no observations of normally distributed data with $i$ mean and unit variance
    return ar

nf = 5
no = 20
ts = make_timeseries(nf, no)

cols = cl.scales['9']['seq']['BuPu']
cols2 = OrderedDict()
for idx in range(0, nf):
    cols2[idx] = cols[idx]
cols = cols2
HTML(cl.to_html(cols.values()))
data = []

labs = ['']
for i in range(0, nf):
    trace = Scatter(
        x = np.linspace(1, no, no),
        y = ts[i,:],
        line = Line(color=cols[i]),
        name = str(i),
        showlegend=True,
        legendgroup=str(i)
    )
    data.append(trace)
iplot(data)<jupyter_output><empty_output><jupyter_text>As you can see, the figure above is very bare-bones. To prettify, we might want to add x and y legends, titles, and more descripitive labels for our timeseries lines. We can do this as follows:<jupyter_code>labs = ['Frontal Lobe', 'Parietal Lobe', 'Occipital Lobe', 'Temporal Lobe', 'Brainstem']

data = []
for i in range(0, nf):
    trace = Scatter(
        x = np.linspace(1, no, no),
        y = ts[i,:],
        line = Line(color=cols[i]),
        name = labs[i]
    )
    data.append(trace)
    
layout = Layout(
    title = 'EEG Timeseries Plot',
    xaxis = dict(title="Time (s)"),
    yaxis=dict(title="Intensity (V)")
)
    
fig = Figure(data=data, layout=layout)
iplot(fig)<jupyter_output><empty_output><jupyter_text>Plotting a heat mapHere, we will take the pairwise correlation of our timeseries, and then plot it as a heatmap:<jupyter_code>cmtx = np.corrcoef(ts)
print(cmtx)
trace = Heatmap(z = cmtx, x=labs, y=labs)
data = [trace]
iplot(data)<jupyter_output>[[ 1.          0.04872539  0.01320359  0.30540732 -0.09015031]
 [ 0.04872539  1.         -0.40615845  0.24884386  0.21241744]
 [ 0.01320359 -0.40615845  1.          0.0348265  -0.52062781]
 [ 0.30540732  0.24884386  0.0348265   1.          0.07611111]
 [-0.09015031  0.21241744 -0.52062781  0.07611111  1.        ]]
{'fraction_non_alphanumeric': 0.08637925648234926, 'fraction_numerical': 0.03358325523273977, 'mean_word_length': 4.10199203187251, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '435224', 'license': 'Apache-2.0', 'n_tokens_mistral': 2002, 'n_tokens_neox': 1821, 'n_words': 795, 'path': 'python_tutorial.ipynb', 'repo_name': 'NeuroDataDesign/jupyter_tutorial', 'size': '6402'}