<jupyter_start><jupyter_text><NAME>COMPUTE course2020-03-09, Lund University  LAYOUT- What is a Jupyter notebook?- How can it improve my workflow?- How to get started- How to share my notebook to help other scientists to reproduce my analysis- Practical details about this course  What is a Jupyter notebook?Interactive document that integrates:- code:    - A long list of available programming languages:        - Python, Java, R, Julia, Matlab, Octave, Scala, Spark, PHP, C, C++, etc.- command-line tools:    - copying / deleting / moving files with `cp` / `rm` / `mv`    - navigate in the directory tree with `cd`    - create a new folder with `mkdir` - narrative text:    - equations    - tables    - links- visualizations  CodeDocumentation accessible within the notebook.- How can I call this function?- Which arguments does it have?- What attributes does this object have?<jupyter_code>names = ['marie_curie','amedeo_avogadro','rosalind_franklin']
print(type(names), names[0], type(names[0]))
for name in names:
    first_last = name.split('_')
    print('first_last is ',first_last)
    first = first_last[0]
    last = first_last[1]
    print(first.capitalize()+' '+last.swapcase())<jupyter_output>first_last is  ['marie', 'curie']
<NAME>
first_last is  ['amedeo', 'avogadro']
<NAME>
first_last is  ['rosalind', 'franklin']
<NAME><jupyter_text>Command-Line Tools:No need to use the terminal or file managers.- copy / delete / move files or folders with `cp` / `rm` / `mv`- navigate in the directory tree with `cd`- create a new folder with `mkdir` - check the pth of the current directory with `pwd`<jupyter_code>%pwd
%mkdir data
%ls
%rm -r data
%ls<jupyter_output>[34maux[m[m/                 jupyter.ipynb        [34mreveal.js[m[m/
compute.ipynb        jupyter.slides.html
custom.css           jupyter_slides.pdf<jupyter_text>Narrative Text[Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) markup language:- equations- tables- links ``` Free Induction DecayThe oscillating voltage, $V(t)$, has an initial amplitude $V(0)$ which freely decays in time, $t$. The dampened signal can be modeled as a sine function of frequency $\nu$, decaying exponentially with decay constant $T_2$:\begin{equation}V(t)=V(0)\,\exp{(-t/T_2)}\,\sin{(2 \pi \nu t)}.\end{equation}| Variable      | Description    | Unit  ||---------------| ---------------|-------|| $t$      | time | ms || $V$      | voltage | V || $\nu$    | frequency | Hz || $T_2$ | decay constant | ms | References1. [Wikipedia](http://tiny.cc/r9r0ez)2. [Merriam-Webster](http://tiny.cc/uas0ez)```  Free Induction DecayThe oscillating voltage, $V(t)$, has an initial amplitude $V(0)$ which freely decays in time, $t$. The dampened signal can be modeled as a sine function of frequency $\nu$, decaying exponentially with decay constant $T_2$:\begin{equation}V(t)=V(0)\,\exp{(-t/T_2)}\,\sin{(2 \pi \nu t)}.\end{equation}| Variable      | Description    | Unit  ||---------------| ---------------|-------|| $t$      | time | ms || $V$      | voltage | V || $\nu$    | frequency | Hz || $T_2$ | decay constant | ms | References1. [Wikipedia](http://tiny.cc/r9r0ez)2. [Merriam-Webster](http://tiny.cc/uas0ez)  How can it improve my workflow?All the steps of your data analysis and visualization in a single document. - Interactive data exploration and analysis- Immediate access to documentation: learn coding, readily use new libraries!- Facilitates iteration:     - once the notebook is set up, the analysis can be repeated effortlessly with new variables / data sets- A large set of freely available tools:    - Python libraries for linear algebra, fitting data, plotting, handling tabular data, image analysis, bioinformatics, spectroscopy, molecular visualization- [Examples](https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks) ``````<jupyter_code>import numpy as np
x = np.linspace(0,np.pi/2.,10)
x
np.mean(x) # np.std() to compute the standard deviation
y = np.cos(x) # np.sin(), np.tan(), np.log(), np.exp() etc.
y<jupyter_output><empty_output><jupyter_text><jupyter_code>!head -n 1 aux/pmf.dat
# Load data file
x,y,z = np.loadtxt('aux/pmf.dat',unpack=True)
# Integrate along the given axis using the composite trapezoidal rule.
np.trapz(y,x)
# Return the derivative of an array.
dy = np.gradient(y,x)
# Save the gradient to a text file
np.savetxt('aux/gradient.dat',y)<jupyter_output><empty_output><jupyter_text><jupyter_code>import matplotlib.pyplot as plt
plt.rcParams.update({'figure.dpi': 70})
import matplotlib.pyplot as plt
x = np.linspace(0,10*np.pi,200)
y = np.cos(x)
plt.plot(x,y)
plt.ylabel('Cosine')
plt.xlabel('Angle  /  rad')
plt.show()<jupyter_output><empty_output><jupyter_text>$$P(x) = \frac{1}{\sqrt{2 \pi \sigma}} \exp{\left ( \frac{-(x-\mu)^2}{2\sigma^2} \right)}$$<jupyter_code>x = np.linspace(0,9,1000)
for u in range(1,8):
    y = np.exp(-(u-x)**2/(2*0.2**2)) / np.sqrt(2*np.pi*0.2**2)
    plt.plot(x,y,label=str(u))
plt.legend(frameon=False, title='$\mu$ / nm')
plt.xlim(0,9)
plt.xlabel('Distance, $x$ / nm'); plt.ylabel('Probabilty, $P(x)$')
plt.savefig('aux/normal.pdf') # png, jpg, eps
plt.show()
plt.rcParams.update({'figure.dpi': 300})
import matplotlib.image as mpimg
img = mpimg.imread('aux/protein.png')
print(img.shape)
fig = plt.figure(figsize=(1.2, 1.2))
plt.imshow(img, interpolation='bilinear')
plt.axis('off')
plt.show()
plt.rcParams.update({'figure.dpi': 75})
from matplotlib.collections import LineCollection
x = np.linspace(0,10,1000)
y = np.exp(-(2.6-x)**2) / np.sqrt(4*np.pi) + np.exp(-(7-x)**2/2) / np.sqrt(8*np.pi)
points = np.array([x, y]).T.reshape(-1, 1, 2)
segments = np.concatenate([points[:-1], points[1:]], axis=1)
norm = plt.Normalize(x.min(), x.max())
lc = LineCollection(segments, cmap='plasma', norm=norm)
lc.set_array(x); lc.set_linewidth(4); plt.gca().add_collection(lc)
plt.ylabel(r'Probability, $P(x)$'); plt.xlabel(r'Distance, $x$  /  nm')
plt.xlim(0,10); plt.ylim(0,.3)
plt.show()<jupyter_output><empty_output><jupyter_text>Q: How can I plot a gradient-colored line?A: Google ["matplotlib gradient color line"](https://matplotlib.org/3.1.1/gallery/lines_bars_and_markers/multicolored_line.html)You can google very specific questions and quickly find excellent answers, generally on matplolib.org or stackoverflow.com  Multiple Subplots<jupyter_code>fig, axes = plt.subplots(nrows=3,ncols=2,figsize=(7, 6))
x = np.arange(10)
axes[0,0].plot(x, x**2, 'bo')
axes[2,1].plot(x, -x**2, 'r^')
axes[2,1].yaxis.set_ticks_position('right') # yticks on the right side
def plotQCM():    
    fig, ((ax1,ax2),(ax3,ax4)) = plt.subplots(nrows=2,ncols=2)

    colors = plt.rcParams['axes.prop_cycle'].by_key()['color']

    ax1 = plt.subplot2grid((11, 5), (5, 0), rowspan=6, colspan=3)
    ax3 = plt.subplot2grid((11, 5), (0, 0), rowspan=5, colspan=3)
    ax2 = plt.subplot2grid((11, 5), (5, 3), rowspan=6, colspan=2)
    ax4 = plt.subplot2grid((11, 5), (0, 3), rowspan=5, colspan=2)

    a = np.linspace(0,np.pi,100)

    for ax in [ax2,ax4]:
        for i,c in zip(range(1,13,2),colors):
            ax.plot(np.cos(a*i)+i*2,a,lw=2,color=c)
            ax.plot(-np.cos(a*i)+i*2,a,lw=2,ls=':',color=c)

        ax.set_xticks(np.arange(1,13,2)*2)
        ax.set_xticklabels(np.arange(1,13,2))
        ax.tick_params(axis='both',which='both',bottom=False,right=False,labelbottom=True,labelright=False,
                        left=False,labelleft=False,pad=-.5)
        ax.set_frame_on(False)

    ax2.set_ylim(0-0.06/1.3*np.pi,np.pi+0.06/1.3*np.pi)
    ax4.set_ylim(0-0.06*np.pi,np.pi+0.06*np.pi)

    x0 = 2; y0 = 0; x1 = 1.7;

    c=7

    ax1.fill([x1,x1+2,x1+2.6,x1+.6], [y0,y0,y0+1,y0+1], colors[c], alpha=0.3, 
             edgecolor=colors[c],ls='--',lw=0)
    ax1.fill([x1+.6,x1+2.6,x1+2.6,x1+.6], [y0+1,y0+1,y0+1.3,y0+1.3], colors[c], alpha=0.6, 
             edgecolor=colors[c],ls='--',lw=0)

    ax1.fill([x0,x0+2,x0+2,x0], [y0+1,y0+1,y0+1.3,y0+1.3], colors[9], alpha=0.6, edgecolor=colors[9],ls='-',lw=0)
    ax1.fill([x0,x0+2,x0+2,x0], [y0,y0,y0+1,y0+1], colors[9], alpha=0.3, edgecolor=colors[9],ls='-',lw=0)

    ax3.fill([x1,x1+2,x1+2.6,x1+.6], [y0,y0,y0+1,y0+1], colors[c], alpha=0.3, 
             edgecolor=colors[c],ls='--',lw=0)
    ax3.fill([x0,x0+2,x0+2,x0], [y0,y0,y0+1,y0+1], colors[9], alpha=0.3, edgecolor=colors[9],ls='-',lw=0)

    for ax in [ax1,ax3]:    
        ax.axis('off')
        ax.plot([2.3,2.3],[0,1],marker='o',lw=0,color='k')
        ax.hlines(y=[0,1],xmin=[.9925,.9925],xmax=[2.3,2.3],lw=1,color='k')
        ax.annotate(r'$\bigcirc$',xy=(1,0.5),fontsize=24,color='k',horizontalalignment='center',
                    verticalalignment='center')
        ax.annotate(u'\u223F',xy=(1,0.5),fontsize=16,color='k',horizontalalignment='center',
                    verticalalignment='center')
        ax.vlines(x=[1,1],ymin=[0,0.61],ymax=[.4,1],lw=1,color='k')

    ax1.set_xlim(.7,4.5)
    ax3.set_xlim(.7,4.5)
    ax3.set_ylim(-.05,1.05)
    ax1.set_ylim(-.05,1.35)

    ax3.annotate('QCR', xy=(3,0.5),fontsize=14,color='k',horizontalalignment='center', verticalalignment='center')
    ax1.annotate('QCR', xy=(3,0.5),fontsize=14,color='k',horizontalalignment='center', verticalalignment='center')
    ax1.annotate('Film', xy=(3,1.15),fontsize=14,color='k',horizontalalignment='center', verticalalignment='center')

    plt.gcf().text(.59, 0.57, '$n$  = ', fontsize=12)
    plt.gcf().text(.59, 0.045, '$n$  = ', fontsize=12)

    plt.tight_layout(w_pad=2.5,h_pad=1)
    plt.show()
def plotFourier():    
    fig, (ax1,ax2) = plt.subplots(nrows=1,ncols=2,figsize=(7, 3.5))
    
    colors = plt.rcParams['axes.prop_cycle'].by_key()['color']

    gamma = 2e-1
    fr = 5
    t = np.arange(0,10,.001)
    cos = np.cos(2*np.pi*fr*t)
    exp = np.exp(-t*2*np.pi*gamma)
    func = exp*cos
    ax1.plot(t,func,color=colors[0])
    ax1.set_xlim(0,4)

    fs = np.linspace(0,10,1000)
    curr = []
    for f in fs:
        curr.append( np.trapz( func*np.cos(2*np.pi*f*t),t) )
    curr = np.array(curr)
    ax2.plot(fs,curr,lw=2,color=colors[0]) 
    ax2.yaxis.set_label_position('right'); ax2.yaxis.set_ticks_position('right')
    ax2.set_xlim(0,7)
    ax2.set_ylim(-.05,.55)

    ax1.hlines(y=-np.exp(-1),xmin=0,xmax=1/(2*np.pi*gamma))
    ax1.hlines(y=.7,xmin=4/5,xmax=1)
    ax1.hlines(y=.7,xmin=1,xmax=1.5,lw=1,linestyle=':')
    ax1.vlines(x=4/5,ymin=np.exp(-4/5*2*np.pi*gamma),ymax=.7,lw=1,linestyle=':')
    ax1.vlines(x=1,ymin=np.exp(-2*np.pi*gamma),ymax=.7,lw=1,linestyle=':')
    ax1.vlines(x=1/(2*np.pi*gamma)-.01,ymin=-np.exp(-1),ymax=-.6,lw=1,linestyle=':')
    ax1.annotate('1 / ( 2 $\pi$ $\Gamma_r$ )',xy=(.67,-.8),fontsize=16)
    ax2.hlines(y=curr.max()/2.,xmin=5,xmax=5+gamma)
    ax2.hlines(y=curr.max()/2.,xmin=5,xmax=5+gamma+.7,lw=1,linestyle=':')
    ax2.vlines(x=fr,ymin=curr.max()/2.,ymax=curr.max()+.05,lw=1,linestyle=':')
    ax2.annotate('$\Gamma_r$',xy=(6,.185),fontsize=16)
    ax2.annotate("$f_r$",xy=(fr-.2,curr.max()+.07),fontsize=16)
    ax1.annotate('1 / $f_r$',xy=(1.6,.65),fontsize=16)
    ax1.text(x=4.16,y=.1,s='FT',fontsize=16)
    ax1.text(x=4.13,y=-.05,s='⟶',fontsize=16)

    gamma = 4e-1
    fr = 2
    t = np.arange(0,10,.001)
    cos = np.cos(2*np.pi*fr*t)
    exp = np.exp(-t*2*np.pi*gamma)
    func = exp*cos
    ax1.plot(t,func,color=colors[3],lw=1)

    fs = np.linspace(0,10,1000)
    curr = []
    for f in fs:
        curr.append( np.trapz( func*np.cos(2*np.pi*f*t),t) )
    curr = np.array(curr)
    ax2.plot(fs,curr,lw=1,color=colors[3]) 
    ax2.yaxis.set_label_position('right'); ax2.yaxis.set_ticks_position('right')

    ax2.hlines(y=curr.max()/2.,xmin=fr,xmax=fr+gamma)
    ax2.hlines(y=curr.max()/2.,xmin=fr,xmax=fr+gamma+.7,lw=1,linestyle=':')
    ax2.vlines(x=fr,ymin=curr.max()/2.,ymax=curr.max()+.05,lw=1,linestyle=':')
    ax2.annotate("$\Gamma$",xy=(3.2,.08),fontsize=16)
    ax2.annotate("$f$",xy=(fr-.2,curr.max()+.07),fontsize=16)
    ax1.tick_params(axis='both',which='both',left=False,bottom=False,labelbottom=False,labelleft=False)
    ax2.tick_params(axis='both',which='both',bottom=False,right=False,labelbottom=False,labelright=False)
    ax1.set_xlabel('Time',labelpad=6)
    ax2.set_xlabel('Frequency',labelpad=6)
    ax1.set_ylabel('Current,  $I(t)$',labelpad=6)
    ax2.set_ylabel('Current,  $I(t)$',labelpad=8)

    fig.tight_layout(w_pad=3)
    plt.show()
plt.rcParams.update({'figure.dpi': 60})
plotQCM()
plotFourier()<jupyter_output><empty_output><jupyter_text>[Jupyter Widgets](https://ipywidgets.readthedocs.io/en/latest/)Gain control and visualize changes in the data!<jupyter_code>from mpl_toolkits.axes_grid1.inset_locator import inset_axes
from ipywidgets import interactive

def plot_cos_decay_FT(freq=1,gamma=.2):

    def cos_decay(time,freq,gamma):
        cos = np.cos(2*np.pi*freq*time)
        exp = np.exp(-time*2*np.pi*gamma)
        return exp*cos
    
    def FT(time,freq,gamma):
        fourier = []
        for f in np.linspace(0,freq*2,1000):
            cos = np.cos(2*np.pi*f*time)
            fourier.append( np.trapz( cos_decay(time,freq,gamma)*cos,time) )
        return np.array(fourier)

    time = np.arange(0,10,.001)

    fig = plt.figure(figsize=(3.5,4))
    ax = plt.axes()
    ax.plot(time,cos_decay(time,freq,gamma),color=plt.get_cmap('tab10')(3), lw=1)
    
    axins = inset_axes(ax, width='60%', height='30%', loc='upper right', borderpad=1.1)
    axins.plot(np.linspace(0,freq*2,1000),FT(time,freq,gamma),color=plt.get_cmap('tab10')(0), lw=1)
    
    axins.set_xlabel(r'Frequency, $f$',color=plt.get_cmap('tab10')(0),fontsize=10,labelpad=1)
    axins.set_ylabel(r'$I(f)$',fontsize=10,labelpad=1)
    
    ax.set_ylabel('$I(t)$',fontsize=12)
    ax.set_xlabel(r'Time, $t$',color=plt.get_cmap('tab10')(3),fontsize=12)
    ax.set_xlim(0,10)
interactive_plot = interactive(plot_cos_decay_FT, freq=(1, 5, .1), gamma=(.08,.14,.01) )
interactive_plot.children[0].description=r'$f$' # slide bar
interactive_plot.children[1].description=r'$\Gamma$' # slide bar
interactive_plot<jupyter_output><empty_output><jupyter_text><jupyter_code>import pandas as pd
from IPython.display import display<jupyter_output><empty_output><jupyter_text>Library to handle tabular data: a convenient alternative to Excel!  Data Scraping: Importing an HTML Table from [Sigma Aldrich](https://www.sigmaaldrich.com/life-science/metabolomics/learning-center/amino-acid-reference-chart.html)<jupyter_code>url = "https://www.sigmaaldrich.com/life-science/metabolomics/learning-center/amino-acid-reference-chart.html"
df = pd.read_html(url, header=0, index_col=0, na_values='–')[0]
df = df['Alanine':'Valine'] # select rows we are interested in
df = df.apply(pd.to_numeric,errors='ignore') # convert numbers from strings to numeric values
display( df.iloc[::3] ) # show every third amino acid
display( df['Arginine':'Glutamic acid'][['pKa1','pKb2','pKx3']] )
df['Arginine':'Glutamine']['Molecular  Weight'].values
df['Arginine':'Glutamine']['Molecular  Weight'].values.mean()
display(df[df['pl4']>7])
df[df['pl4']>7]['Molecular  Weight'].values.mean()
np.mean(df['Molecular  Weight']-df['Residue Weight  (-H2O)'])<jupyter_output><empty_output><jupyter_text>[](https://jakevdp.github.io/PythonDataScienceHandbook/)https://jakevdp.github.io/PythonDataScienceHandbook/ [](http://www.rdkit.org/)<jupyter_code>from rdkit import Chem
from rdkit.Chem.Draw import IPythonConsole
m1 = Chem.MolFromSmiles('n1c2C(=O)NC(N)=Nc2ncc1CNc3ccc(cc3)C(=O)N[C@H](C(O)=O)CCC(O)=O')
m1
from rdkit.Chem import Draw
Draw.MolToFile(m1,'aux/folate.svg')
m1.GetNumAtoms()
Chem.MolToSmiles(m1)
m2 = Chem.AddHs(m1) # add hydrogens
m2
from rdkit.Chem import AllChem
Chem.AllChem.EmbedMolecule(m2) # make it 3D using ETKDG method
m2
import nglview as nv
view = nv.show_rdkit(m2)
view
print(Chem.MolToMolBlock(m2),file=open('aux/folate.mol','w+'))

view = nv.show_file('aux/folate.mol')
view<jupyter_output><empty_output><jupyter_text><jupyter_code>import mdtraj as md
s = md.load('aux/4mqj.pdb')
print('Number of atoms:', s.n_atoms)
print('Number of residues:', s.n_residues)
chains = [chain for chain in s.top.chains]
n_chains = len(chains)
print('Number of chains:', n_chains)
import nglview as nv
view = nv.show_pdbid('4mqj')
view
import matplotlib as mpl
view = nv.show_mdtraj(s)
view.clear_representations(component=0)
for i in range(4):
    chain = [a.index for a in s.top.chain(i).atoms]
    view.add_representation('spacefill', selection=chain, color=mpl.colors.to_hex(plt.cm.tab20(i)))
view
def viewColorScheme(molecule,dataframe):
    dataframe = dataframe.copy()
    dataframe['Abbr.'] = dataframe['Abbr.'].str.upper()
    dataframe.set_index('Abbr.', drop=True, inplace=True)
    dd = dataframe.dropna()
    dataframe = dataframe.fillna(0)
    preg = (dd['pKx3']-dd['pKx3'].min()) / (dd['pKx3'].max() - dd['pKx3'].min())
    colorscheme = pd.Series([mpl.colors.to_hex(c) for c in plt.cm.rainbow_r(preg)],index=preg.index)
    view = nv.show_mdtraj(molecule)
    view.clear_representations(component=0)
    for res in [res for chain in chains[:4] for res in chain.residues ]:
        atoms = [a.index for a in res.atoms]
        if dataframe.loc[res.name]['pKx3'] == 0:
            view.add_spacefill(selection=atoms, color='#ffffff')
        else:
            view.add_spacefill(selection=atoms, color=colorscheme[res.name])
    view.camera = 'orthographic'
    return view
viewColorScheme(s,df)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19645834531133213, 'fraction_numerical': 0.04392571724256885, 'mean_word_length': 4.435625, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 53, 'https://': 8, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '546651', 'license': 'MIT', 'n_tokens_mistral': 7484, 'n_tokens_neox': 7081, 'n_words': 1142, 'path': 'qdetailss/compute.ipynb', 'repo_name': 'urania277/jupyter-course', 'size': '17424'}