<jupyter_start><jupyter_text>Introduction to Programming for Engineers Python 3  03 Plotting and Analysing SUPPLEMENTARY MATERIAL 1. Matplotlib 2. Scipy  3. Line and Scatter Plots  4. Curve Fitting   1. MatplotlibMatplotlib is the most widely used Python package for plotting.It's functionality is huge.Today's seminar will introduce the core functionality of Matplotlib.<jupyter_code>import numpy as np

# plotting
import matplotlib.pyplot as plt

# colours
import matplotlib.cm as cm

# Plotting in 3D
from mpl_toolkits.mplot3d import axes3d<jupyter_output><empty_output><jupyter_text>2. SciPyBoth Numpy and Matplotlib are part of SciPy; a Python package for mathematics, science, and engineering related computing.Today we will also be importing functions from the SciPy package. To display plots in Jupyter Notebook, we must also have the following line in the notebook *before* generating the plot:<jupyter_code>%matplotlib inline<jupyter_output><empty_output><jupyter_text>To display plots when running a .py file (e.g. in Spyder, we must also have the following line in the notebook *after* generating the plot:<jupyter_code>plt.show()<jupyter_output><empty_output><jupyter_text>Online resources for more Matplotlib functionality: - Matplotlib Gallery (http://matplotlib.org/gallery.html) - Github (http://gree2.github.io/python/2015/04/10/python-matplotlib-plotting-examples-and-exercises)    3. Line and Scatter PlotsMatplotlib can be used to produce basic line and scatter plots.You can manipulate the appearance of these plots in code. For a sample data set of values of $x$ and corresponding values of $f$:<jupyter_code>x = [-1, 3, 4, 8 , 10]
f = [-1, -2, 7, 13 , 1]<jupyter_output><empty_output><jupyter_text>We can produce a line plot..<jupyter_code>plt.plot(x, f)<jupyter_output><empty_output><jupyter_text>Notice the statement `[]` appears each time (the numbers on your computer may look different). This is a handle to the line that is created with the last command in the code block (in this case `plt.plot(x, y)`). You can avoid printing this by putting a semicolon after the last command in the code block (so type `plot(x, y);`).  The `plot` function can take many arguments.Summary so far: `plot` can take: - One argument `plot(y)`, which plots `y` values along the vertical axis and enumerates the horizontal axis starting at 0. - Two arguments, `plot(x, y)`, which plots `x` vs `y`. - Three arguments `plot(x, y, formatstring)`, which plots `x` vs `y` using colors and markers defined in `formatstring`.   The optional __format string__ is a convenient way for defining basic formatting:Using the `formatstring` argument we can chnage: - the colour of the plot (e.g. `r` = red, `k`= black) - the style of the markers (e.g. `o` = points, `*` = stars) - the style of the line (e.g. `--` = dashes, `.` = dots) In the example below: - `r` makes the plot red - `x` makes the markers crosses - `-` makes the line solid<jupyter_code>plt.plot(x, f, '-xr');<jupyter_output><empty_output><jupyter_text>Note the semi-colon when calling the `plot` function.<jupyter_code>plt.plot(x, f, '--og')  
# plt.plot(x, f, 'og--')  
# plt.plot(x, f, 'or--') 
# plt.plot(x, f, 'k.')<jupyter_output><empty_output><jupyter_text>Available colours, maker styles and line styles can be found in the matplotlib documentation.  - Colour: https://matplotlib.org/2.0.2/api/colors_api.html - Marker style: https://matplotlib.org/api/markers_api.html - Line style: https://matplotlib.org/devdocs/gallery/lines_bars_and_markers/line_styles_reference.html The `plot` function also takes a large number of *keyword arguments*. For example, to plot a line with width 6 (the default is 1):<jupyter_code>plt.plot(x, f, '-xr', linewidth=6, markersize=10);<jupyter_output><empty_output><jupyter_text>Remeber keyword arguments must always appear after regular arguments when calling the function.Documentation of all available keyword arguments can be found here: https://matplotlib.org/api/_as_gen/matplotlib.pyplot.plot.htmlmatplotlib.pyplot.plot Other methods such as `legend`, `xlabel`, `ylabel` and `title` can be used to add useful information to a plot.  Adding Labels, Legend and Title. The location of the legend can be set the the 'best' in terms of not obscuring data. The position of the legend can also be set expllicitly instead. The location codes acan be found here: https://matplotlib.org/api/legend_api.htmlUsing the $ sign can be used to italicise a string.The font size of all strings in the figure can be selected.<jupyter_code># Use the label kwarg to create a name to use in the figure legend. 
plt.plot(x, f, '-xr', label="data 1")

# Legend
plt.legend(loc='best', fontsize=12)

# Axes labels
plt.xlabel('$x$', fontsize=20)
plt.ylabel('$f$', fontsize=20)

# Title
plt.title("Simple plot of $f$ against $x$", fontsize=18);<jupyter_output><empty_output><jupyter_text>Note the semi-colon when calling the `title` function.  Setting the Axis LimitsExample: plot the funstion $sin(x)$  for 100 equally spaced values of x between $0$ and $4 \pi$ (inclusive).<jupyter_code>num_points = 100
x = np.linspace(0, 4*np.pi, num_points)
f = np.sin(x)

# Plot graph
plt.plot(x, f);

# Label axis
plt.xlabel('$x$')
plt.ylabel('$\sin(x)$')<jupyter_output><empty_output><jupyter_text>The zero on the x axis is offset by an unknown amount. We can improve the appearance of the plot by specifying the $x$ limits:<jupyter_code># Plot graph
plt.plot(x, f);

# Label axis
plt.xlabel('$x$')
plt.ylabel('$\sin(x)$')

# Use the start and end values in x as x limits 
plt.xlim(x[0], x[-1])

# ...or set the lower x limit only
#plt.xlim(xmin=0)
#plt.xlim(xmin=x[0])<jupyter_output><empty_output><jupyter_text>Horizontal LinesWe can add a line to the graph to indicate the y=0 axes.<jupyter_code># Plot graph
plt.plot(x, f);

# Use the start and end values in x as x limits 
plt.xlim(x[0], x[-1])

# Add a line at y=0 
plt.axhline(0, color='black', lw=0.5) # y=0 axis<jupyter_output><empty_output><jupyter_text>Multiple PlotsPlots can be overlaid by called `plt.plot(...` multiple times.<jupyter_code># Plot sin(x) and cos(x), and add label for each
plt.plot(x, np.sin(x), label="$\sin(x)$");
plt.plot(x, np.cos(x), label="$\cos(x)$");

# Label the x-axis
plt.xlabel('$x$');

# Add legend
plt.legend();

# Use the start and end values in x as x limits
plt.xlim(x[0], x[-1]);<jupyter_output><empty_output><jupyter_text>Saving a PlotA plot can be saved as a file for your to use e.g. in a report.A file name should be given.The file type is specified using the file extension.`/` can be used to seperate the names of nested folders. In the example below, the graph is saved as a pdf and a png file in the `img` folder in your directory. Run the code in the cell below then open the folder `img` using the file browser on your computer to verify that the files were generated by noting the time stamp shown for each file.<jupyter_code># Save plot to file
plt.savefig("img/my-plot.pdf")
plt.savefig("img/my-plot.png")<jupyter_output><empty_output><jupyter_text>__Try it yourself__In the cell below:- create a green line plot of the function $sin^2(x)$ (or $(sin(x))^2$) using the range of $x$ from the last example. - add a blue triangular marker at $x=\frac{\pi}{2}, \pi, \frac{3\pi}{2}, 2\pi, \frac{5\pi}{2}, 3\pi, \frac{7\pi}{2}$ and $4\pi$.- label the axes- save the figure as a .jpg file in the `img` folder in your repository.  4. Curve FittingIt can be useful to define a relationship between two variables e.g. x and y values. For example, we often want to 'fit' a function to a set of data points such as experimental data.Python has several tools for finding relationships in a set of data.  Linear RegressionLinear regression calculates a __linear function__ that minimizes the distance between the fitted line and all of the data points. A __linear function__ has the form:$$y = mx + c$$Such a linear function can be fitted to a data set using the `linregress` function from `scipy.stats`.  $$y = mx + c$$`linregress(x, y)` returns (in the following order): - gradient $m$ of the fitted linear function- y-intercept $c$ of the fitted linear function- statistical parameters for evaluating the correlation between the x and y values:    - r value    - p value    - standard error<jupyter_code>from scipy.stats import linregress

# imported data
x = students[:, 0]
y = students[:, 1]


# gradient is first returned value
# y-intercept is second returned value
m, c, r, p, se = linregress(x, y)<jupyter_output><empty_output><jupyter_text>The coefficient of correlation, $R$, and determination $R^2$ The correlation coefficient $R$ is used to statistically analyse the strength of a linear relationship between two variables.$R$ values range from -1 to 1.1 indicates that the two variables have perfect positive correlation i.e. they rise and fall together.-1 indicates that the two variables have perfect negative correlation i.e. as one goes up the other goes down. The weaker the correlation, the closer the  $R$ value will be to 0. Correlation can be easily understood for simple linear regression –  two variables: x and y. For multiple linear regression (>2 variables), positive and neagative correlation are more difficult to interpret. $R^2$ values range from 0 to 1 (i.e. it is always positive).$R^2$ is therefore often used as it can be used to evaluate the the strength of a linear relationship for simple linear regressions and also for multiple linear regressions.  The coefficient of correlation$$R = \frac{n(\sum_{i=1}^{n} x_i y_i)-(\sum_{i=1}^{n} x)(\sum_{i=1}^{n} y_i)}{\sqrt{(n\sum_{i=1}^{n} x_i^2 - (\sum_{i=1}^{n} x_i)^2) (n\sum_{i=1}^{n} y_i^2 - (\sum_{i=1}^{n} y_i)^2) }}$$$n$ = number of data points $x_i$ = independent variable $y_i$ = dependent variable The coefficient of determination$$R^2 = \frac{explained\, variation}{total\, variation} = R \times R = \frac{\sum_{i=1}^{n}(\hat{y_i} - \overline{y_i})^2}{\sum_{i=1}^{n}(y_i - \overline{y_i})^2}$$ where:$y_i=$ actual dependent variable $\hat{y_i}=$ estimated dependent variable $\overline{y_i}=$ average dependent variable  The p-valueThe p-value for each independent variable (e.g  x) indicates the degree to which each relationship with y is statistically significant.The p-value for each independent variable (e.g  x) tests the *null hypothesis* that x has __no correlation__ with the dependent variable, y.The p-value is often used to determine which independent variables to use in a regression model. We will not be working with this statistical measure.  The Standard Error The standard error of the regression slope, $s$,  represents the average distance that the data points deviate from the regression line. The smaller the $s$ value, the closer the values are to the regression line.$$s = \frac{\sqrt {\frac{\sum_{i=1}^{n}(y_i - \hat{y_i})^2}  {n - 2}}} {\sqrt {\sum_{i=1}^{n}(x_i - \overline{x_i})^2 }}$$where:$y_i=$ actual dependent variable $\hat{y_i}=$ estimated dependent variable$x_i=$ actual independent variable $\overline{x_i}=$ average independent variable$n=$ number of data points<jupyter_code>x = [0.000000000000000000e+00, 1.052631578947368363e+00, 2.105263157894736725e+00, 3.157894736842105310e+00,4.210526315789473450e+00, 5.263157894736841591e+00, 6.315789473684210620e+00,7.368421052631578760e+00,8.421052631578946901e+00,9.473684210526315042e+00,1.052631578947368318e+01,1.157894736842105132e+01,1.263157894736842124e+01,1.368421052631578938e+01,1.473684210526315752e+01, 1.578947368421052566e+01,1.684210526315789380e+01,1.789473684210526372e+01,1.894736842105263008e+01,2.000000000000000000e+01]
y = [7.445192947240600745e+01, 4.834835792411828947e+01, 6.873305436340778840e+01, 5.979576407972768948e+01,6.404530772390434379e+01,6.090548420541189500e+01, 7.157546008677115879e+01, 8.620253336570679892e+01, 1.138154622045899913e+02, 8.493639813028174501e+01, 9.783457330550828601e+01, 1.082064229481453594e+02, 1.063876210674365979e+02, 1.001971993955305038e+02, 1.061496321788094832e+02, 1.279575585921491836e+02, 1.556956405962417875e+02, 1.584164804859289859e+02, 1.753888794716459358e+02, 1.980941276403034124e+02]
# convert to numpy array to allow elementwise operations
x = np.array(x)
y = np.array(y)


# gradient and y-intercept
m, c = linregress(x, y)[0], linregress(x, y)[1]


# generate the fitted y values
yfit = float(m) * x + c


# plot a scatter plot of raw data
plt.plot(x, y, 'o', label='experiment data')


# plot the fitted linear function 
# the argument 'label' is used to print the equation of the line on the plot 
plt.plot(x, yfit, label=f'y = {round(m, 3)}x + {round(c, 3)}');


# display the label
plt.legend()

# label the axes
plt.xlabel('x')
plt.ylabel('y')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11417105987882603, 'fraction_numerical': 0.07608781178692266, 'mean_word_length': 5.140096618357488, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 66, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '56487', 'license': 'MIT', 'n_tokens_mistral': 4857, 'n_tokens_neox': 4201, 'n_words': 1526, 'path': '03_PlottingAnalysing__SupplementaryMaterial.ipynb', 'repo_name': 'hphilamore/UoB_PythonForEngineers_2020', 'size': '12709'}