<jupyter_start><jupyter_text>Sveučilište u ZagrebuFakultet elektrotehnike i računarstva Strojno učenjehttp://www.fer.unizg.hr/predmet/suAk. god. 2015./2016. Bilježnica 0: Uvod u SciPy(c) 2015 <NAME>Verzija: 0.5 (2015-10-15) NEPOTPUNO  1. SciPy stack Glavni paketi (*core packages*):* **Python*** **NumPy*** **biblioteka SciPy*** **IPython*** **Matplotlib*** SymPy * **pandas*** noseDodatni paketi:* Cython* SciKits paketi: **scikit-learn**, scikit-multilearn, scikit-image, ...  2. IPython notebook Ćelije se evaluiraju sa SHIFT+ENTER Markdown tekst s **posebnim formatiranjem** i kodom u $\LaTeX$-u: $f(\mathbf{x}) = \sum_{i=1}^n \ln \frac{P(x)P(y)}{P(x, y)}$<jupyter_code>10
_
?
%quickref<jupyter_output><empty_output><jupyter_text>Više: https://ipython.org/ipython-doc/3/interactive/tutorial.html  3. Python  3.1. Varijable i vrijednosti<jupyter_code>x = 5
x
print(x)
print x
type(x)
(x + 1) ** 2
x += 1; x
?x
del x
x
X=7; varijabla_s_vrlo_dugackim_imenom = 747
x=1; y=-2
x==y
(x==y)==False
x!=y
x==y or (x>0 and not y>0)
z = 42 if x==y else 66
z
moj_string = 'ringe ringe'
'hopa' + ' ' + "cupa"
moj_string += ' raja'; moj_string
len(moj_string)
print "X=%0.2f y=%d, s='%s'" % (x, y, moj_string)
1/2
1/2.0
1/float(2)
round(0.5)<jupyter_output><empty_output><jupyter_text>3.2. Matematičke funkcije<jupyter_code>import math
math.sqrt(68)
math.exp(1)
math.log(_)
math.log(100, 2)<jupyter_output><empty_output><jupyter_text>Više: https://docs.python.org/2/library/math.html  3.3. Lista<jupyter_code>xs = [5, 6, 2, 3]   # Stvara listu
xs
xs[0]  # Zero-based indeksiranje
xs[-1]  # Negativni indeksi broje od kraja liste
xs[1] = 100  # Ažuriranje liste
xs
xs[1] = 'foo'  # Liste mogu biti heterogene
xs
xs[3] = [1,2]
xs
xs.append(x)  # Dodaje na kraj
xs
xs + [77, 88]
xs.extend([77, 88]); xs
xs.pop()  # Skida zadnji element liste
xs
xs[0:2]
xs[1:]
xs[:3]
xs[:]
xs[:-2]  # Sve osim zadnja dva
xs[0:2] = [1,2]
xs
range(10)
range(1, 10)
range(0, 51, 5)
for x in range(5):
    print x
for x in xs: print x
for ix, x in enumerate(range(0, 51, 5)):
  print ix, x
xs = []
for x in range(10):
    xs.append(x ** 2)
xs
[x ** 2 for x in range(10)]
[x ** 2 for x in range(10) if x % 2 == 0]
[(x, x ** 2) for x in range(10)]
zip([1, 2, 3], [4, 5, 6])
zip(*[(1, 4), (2, 5), (3, 6)])
xs, ys = zip(*[(1, 4), (2, 5), (3, 6)])
xs
map(lambda x : x + 1, xs)
[ x + 1 for x in xs ]
ys = []
for x in xs :
    ys.append(x + 1)
ys
sum(ys)<jupyter_output><empty_output><jupyter_text>3.4. Rječnik (mapa)<jupyter_code>d = {'zagreb' : 790017, 'split' : 178102, 'rijeka' : 128624}
d['split']
d['osijek']
d.get('osijek', 0)
d['osijek'] = 108048; d
'rijeka' in d
d['zagreb'] = 790200; d
del d['rijeka']; d<jupyter_output><empty_output><jupyter_text>Iteriranje po rječniku:<jupyter_code>for grad in d:
    print 'Grad %s ima %d stanovnika' % (grad, d[grad])<jupyter_output>Grad osijek ima 108048 stanovnika
Grad split ima 178102 stanovnika
Grad zagreb ima 790200 stanovnika<jupyter_text>Iteriranje po ključevima i po vrijednostima:<jupyter_code>for grad, stanovnici in d.iteritems():
    print 'Grad %s ima %d stanovnika' % (grad, stanovnici)<jupyter_output>Grad osijek ima 108048 stanovnika
Grad split ima 178102 stanovnika
Grad zagreb ima 790200 stanovnika<jupyter_text>Ugniježđeni rječnici:<jupyter_code>d2 = {'zagreb' : {'trešnjevka' : 120240, 'centar' : 145302}}
d2 ['zagreb']['trešnjevka']<jupyter_output><empty_output><jupyter_text>3.5. Funkcije<jupyter_code>def inc(x): return x + 1
def sign(x):
    if x > 0:
        return 'pozitivno'
    elif x < 0:
        return 'negativno'
    else:
        return 'nula'

for x in [-1, 0, 1]:
    print sign(x)<jupyter_output>negativno
nula
pozitivno<jupyter_text>Podrazumijevani argumenti:<jupyter_code>def broj_stanovnika(grad, godina=2015):
    if grad in d:
        return d[grad] + round((godina - 2015) * 10000 * (-1.2))
    else: 
        raise ValueError('Nepoznat neki grad')
broj_stanovnika('zagreb')
broj_stanovnika('zagreb', godina=2020)
broj_stanovnika('zadar')<jupyter_output><empty_output><jupyter_text>3.6. Klase<jupyter_code>class RegistarStanovnika:

    # Konstruktor
    def __init__(self, drzava, d):
        self.drzava = drzava  # Varijabla instance (drugačija za svaku instancu)
        self.d = d  
        
    prirast = -1.2  # Varijabla klase (dijele ju sve instance)

    # Metoda
    def broj_stanovnika(self, grad, godina=2015):
        if grad in self.d:
            return self.d[grad] + round((godina - 2015) * 10000 * self.prirast)
        else: 
            raise ValueError('Nepoznat neki grad')
    
    def ukupno_stanovnika(self):
        return sum(self.d.values())
reg = RegistarStanovnika('Hrvatska', {'zagreb' : 790017, 'split' : 178102, 'rijeka' : 128624})
reg.broj_stanovnika('split')
reg.ukupno_stanovnika()<jupyter_output><empty_output><jupyter_text>4. Numpy<jupyter_code>import numpy as np
?np
np.__version__<jupyter_output><empty_output><jupyter_text>4.1. Polja Jednodimenzijsko polje (polje ranga 1):<jupyter_code>a = np.array([1, 2, 3])
a
print a
type(a)
a = np.array([1, 2, 3], dtype=np.float64)
a
a[0]
a[0] = 100; a
a.shape
len(a)
np.array([1,'a',2])<jupyter_output><empty_output><jupyter_text>Matrica (dvodimenzijsko polje, polje ranga 2):<jupyter_code>m = np.array([[1,2,3],[4,5,6]])
print m
m[1]
m[1,1]
m[1][1]
m.shape
m2 = np.array([[1,2,3],[4,5]])
print m2<jupyter_output>[[1, 2, 3] [4, 5]]<jupyter_text>Izrezivanje (engl. slicing):<jupyter_code>print m
m[:,1]
m[0,1:3]
m[1,:2] = [77, 78]
m<jupyter_output><empty_output><jupyter_text>Uočiti razliku:<jupyter_code>m[:,0]  # daje polje ranga 1
m[:,0:1]  # daje polje ranga 2<jupyter_output><empty_output><jupyter_text>Trodimenzijsko polje (tenzor ranga 3):<jupyter_code>t = np.array([[[1,2],[3,4]],[[4,5],[6,7]]])
t.shape
t[0,1,1]
t[0]
t[0,:,1]<jupyter_output><empty_output><jupyter_text>4.2. Stvaranje polja<jupyter_code>np.zeros((5,5))
np.ones((3,1))
np.full((5,5), 55)
np.eye(6)
np.random.random((4,4))
np.arange(1, 10)
np.arange(1, 10, 2)
np.linspace(1, 10, 5)
np.linspace(1, 10)<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html  4.3. Napredno indeksiranje Indeksiranje poljem brojeva:<jupyter_code>a = np.array([[1,2], [3, 4], [5, 6]]); a
a[0,1]
a[[0,2]]   # Nije isto kao a[0,2] !
a[[0,1,2], [0,1,0]]   # Isto kao: np.array([a[0,0], a[1,1], a[2,0]])<jupyter_output><empty_output><jupyter_text>Indeksiranje Booleovim poljem:<jupyter_code>a
bool_ix = a > 2
bool_ix
a[bool_ix]
a[a > 2]<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/numpy/reference/arrays.indexing.html  4.4. Širenje i naslagivanje Širenje (eng. broadcasting):<jupyter_code>x = np.array([[1, 2], [3, 4]])
v = np.array([1, 2])
print x
x + v
np.ones((2,2,3)) * 5<jupyter_output><empty_output><jupyter_text>Naslagivanje (engl. stacking):<jupyter_code>v
np.vstack([v, v])
np.vstack([x, x])
np.vstack((v, x))
np.hstack((v, v))
np.hstack((x, x))
np.hstack((v, x))
np.column_stack((v, x))
x
np.dstack((x, x))
np.shape(_)<jupyter_output><empty_output><jupyter_text>Preoblikovanje polja:<jupyter_code>m = np.array([[ 1,  2,  3], [77, 78,  6]])
m.reshape(3, 2)<jupyter_output><empty_output><jupyter_text>4.4. Operacije s poljima (vektorske i matrične operacije)<jupyter_code>x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
print x; print y<jupyter_output>[[1 2]
 [3 4]]
[[5 6]
 [7 8]]<jupyter_text>Operacije "po elementima" (element-wise):<jupyter_code>x + y
x - y
x / 2.0
x.dtype
(x/2.0).dtype
x * y
x.dtype='float64'
y.dtype='float64'
x / y
np.sqrt(x)<jupyter_output><empty_output><jupyter_text>Vektorske/matrične operacije:<jupyter_code>x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
v = np.array([1,2])
w = np.array([5,3])<jupyter_output><empty_output><jupyter_text>Skalarni (unutarnji, dot) umnožak vektora:$\begin{pmatrix}1 & 2 \\\end{pmatrix}\cdot\begin{pmatrix}5\\3\\\end{pmatrix}= 11$<jupyter_code>print v.dot(w)
print w.dot(v)
print np.dot(v, w)<jupyter_output>11
11
11<jupyter_text>Umnožak matrice i vektora:$\begin{pmatrix}1 & 2 \\3 & 4 \\\end{pmatrix}\cdot\begin{pmatrix}1\\2\\\end{pmatrix}=\begin{pmatrix}5\\11\\\end{pmatrix}$<jupyter_code>x.dot(v)
np.dot(x, v)<jupyter_output><empty_output><jupyter_text>Umnožak vektora i matrice:$\begin{pmatrix}1 & 2\\\end{pmatrix}\cdot\begin{pmatrix}1 & 2 \\3 & 4 \\\end{pmatrix}=\begin{pmatrix}7 & 10\\\end{pmatrix}$<jupyter_code>v.dot(x)
np.dot(v,x)<jupyter_output><empty_output><jupyter_text>Primijetite da nema razlike između vektor-stupca i vektor-retka. Umnožak matrice i matrice:$\begin{pmatrix}1 & 2\\3 & 4\\\end{pmatrix}\cdot\begin{pmatrix}5 & 6 \\7 & 7 \\\end{pmatrix}=\begin{pmatrix}19 & 22\\43 & 50\\\end{pmatrix}$<jupyter_code>x.dot(y)
np.dot(x, y)<jupyter_output><empty_output><jupyter_text>Vanjski umnožak vektora: $\begin{pmatrix}1\\2\\\end{pmatrix}\times\begin{pmatrix}5 \\ 3 \\\end{pmatrix}=\begin{pmatrix}1\\2\\\end{pmatrix}\cdot\begin{pmatrix}5 & 3\\ \end{pmatrix}=\begin{pmatrix}5 & 3 \\10 & 6 \\\end{pmatrix}$<jupyter_code>np.outer(v, w)<jupyter_output><empty_output><jupyter_text>Ostale operacije:<jupyter_code>x = np.array([0, 2, 4, 1])
np.max(x)
np.argmax(x)<jupyter_output><empty_output><jupyter_text>4.5. Statističke funkcije<jupyter_code>x = np.random.random(10); x
np.mean(x)
np.median(x)
np.var(x)
np.std(x)
x = np.array([1, 2, np.nan])
np.mean(x)
np.nanmean(x)
np.ptp(x)
X = np.array([[1,2],[3,4]])
print X
np.mean(X)
np.mean(X, axis=0)
np.cov(X)
x = np.random.random(10000); x
np.histogram(x)<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/numpy/reference/routines.statistics.html  4.6. Druge često korištene funkcije<jupyter_code>x = np.array([[1,2],[3,4]]); x
np.sum(x)
np.sum(x, axis=0)
np.sum(x, axis=1)
x.T
v
v.T
x.diagonal()
x.trace()  # == x.sum(x.diagonal())<jupyter_output><empty_output><jupyter_text>Aplikacija funkcije na polje:<jupyter_code>x
np.apply_along_axis(sum, 1, x)
np.apply_along_axis(len, 1, x)<jupyter_output><empty_output><jupyter_text>Većina ugrađenih funkcija su *vektorizirane*, tj. moguće ih je primijeniti na cijelo polje tako da provode operaciju nad pojedinačnim elementima polja. Npr.:<jupyter_code>np.sign(x)
np.log(x)<jupyter_output><empty_output><jupyter_text>Isto vrijedi i za korisnički definirane funkcije koje su definirane pomoći vektoriziranih ugrađenih funkcija:<jupyter_code>def inc(x) : return x + 1
inc(x)<jupyter_output><empty_output><jupyter_text>Složenije funkcije treba eksplicitno vektorizirati pomoću [`numpy.vectorize`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html) (ili jednostavno aplicirati funkciju u `for` petlji, što funkcija `vectorize` zapravo i radi). Permutacije:<jupyter_code>x = np.arange(0,10); x
np.random.permutation(x)
x
np.random.shuffle(x); x
x<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/numpy/reference/routines.sort.html  4.7. Konverzija lista  polje<jupyter_code>l = [1, 2, 3]
a = np.array(l); a
list(a)
a.tolist()
l = [[1, 2, 3], [4,5,6]]
a = np.array(l); a
list(a)
a.tolist()<jupyter_output><empty_output><jupyter_text>5. SciPy<jupyter_code>import scipy as sp
sp.__version__<jupyter_output><empty_output><jupyter_text>SciPy importa NumPy. Npr.:<jupyter_code>x = sp.array([1,2,3])<jupyter_output><empty_output><jupyter_text>Iz biblioteke SciPy interesantni su nam moduli [`scipy.linalg`](http://docs.scipy.org/doc/scipy/reference/tutorial/stats.html) i [`scipy.stats`](http://docs.scipy.org/doc/scipy/reference/tutorial/stats.html).  5.1. SciPy.linalg<jupyter_code>from scipy import linalg<jupyter_output><empty_output><jupyter_text>Inverz matrice:<jupyter_code>y
y_inv = linalg.inv(y); y_inv
sp.dot(y, y_inv)<jupyter_output><empty_output><jupyter_text>Determinanta:<jupyter_code>linalg.det(y)<jupyter_output><empty_output><jupyter_text>Euklidska norma ($l_2$-norma) vektora: $\|\mathbf{x}\|_2 = \sqrt{\sum_i x_i^2}$<jupyter_code>w
linalg.norm(w)<jupyter_output><empty_output><jupyter_text>Općenita $p$-norma: $\|\mathbf{x}\|_p = \big(\sum_i |x_i|^p\big)^{1/p}$<jupyter_code>linalg.norm(w, ord=1)
linalg.norm(w, ord=sp.inf)<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html  5.2. SciPy.stats<jupyter_code>from scipy import stats
stats.norm
stats.norm.pdf(0)
xs = sp.linspace(-2, 2, 10);
stats.norm.pdf(xs)
stats.norm.pdf(xs, loc=1, scale=2)<jupyter_output><empty_output><jupyter_text>Uzorkovanje iz normalne distribucije:<jupyter_code>stats.norm.rvs(loc=1, scale=2, size=10)<jupyter_output><empty_output><jupyter_text>"Zamrzavanje" distribucije:<jupyter_code>normal = stats.norm(1, 2)
normal.pdf(xs)
normal.rvs(size=5)<jupyter_output><empty_output><jupyter_text>Multivarijatna Gaussova distribucija:<jupyter_code>?stats.multivariate_normal
mean    = sp.array([1.0, 3.0])
cov     = sp.array([[2.0, 0.3], [0.5, 0.7]])
mnormal = stats.multivariate_normal(mean, cov)
mnormal.pdf([1, 0])
np.random.seed(42)   # Radi reproducibilnosti rezultata
mnormal.rvs(size=5)<jupyter_output><empty_output><jupyter_text>Koeficijent korelacije:<jupyter_code>x, y = np.random.random((2, 10))
y
stats.pearsonr(x, y)<jupyter_output><empty_output><jupyter_text>Više: http://docs.scipy.org/doc/scipy/reference/tutorial/stats.html  6. Matplotlib `matplotlib` sadrži više modula: [`pyplot`](http://matplotlib.org/api/pyplot_api.htmlmodule-matplotlib.pyplot), `image`, `matplot3d`, ...<jupyter_code>import matplotlib.pyplot as plt
import matplotlib
matplotlib.__version__
%pylab inline<jupyter_output>Populating the interactive namespace from numpy and matplotlib<jupyter_text>`pylab` kombinira `pyplot` i `numpy`. Gornja naredba (ipython magic) osigurava da pplotovi budu renderirani direktno u bilježnicu, umjesto da otvoaraju zaseban prozor.  6.1. Funkcija `plot`<jupyter_code>plt.plot([1,2,3,4,5], [4,5,5,7,3])
plt.show()
plt.plot([4,5,5,7,3]);
plt.plot([4,5,5,7,3], 'ro');
def f(x) : return x**2
xs = linspace(0,100); xs
f(xs)
plt.plot(xs, f(xs));
plt.plot(xs, f(xs), 'bo');
plt.plot(xs, f(xs), 'r+');
plt.plot(xs, 1 - f(xs), 'b', xs, f(xs)/2 - 1000, 'r--');
plt.plot(xs, f(xs), label='f(x)')
plt.plot(xs, 1 - f(xs), label='1-f(x)')
plt.legend()
plt.show()
xs = linspace(-5,5)
plt.plot(xs, stats.norm.pdf(xs), 'g--');
plt.plot(xs, stats.norm.pdf(xs, loc=1, scale=2), 'r', linewidth=3);<jupyter_output><empty_output><jupyter_text>6.2. Funkcija `scatter`<jupyter_code>plt.scatter([0, 1, 2, 0], [4, 5, 2, 1])
plt.show()
plt.scatter([0,1,2,0], [4, 5, 2, 1], s=200, marker='s');
np.random.random(10)
for c in 'rgb':
  plt.scatter(sp.random.random(100), sp.random.random(100), s=200, alpha=0.5, marker='o', c=c)<jupyter_output><empty_output><jupyter_text>6.3. Grafikon konture i gustoće<jupyter_code>x = np.linspace(1,5,5); x
X, Y = np.meshgrid(x, x)
X
Y
Z = 10 * X + Y
Z
plt.pcolormesh(X, Y, Z, cmap='gray')
plt.show()<jupyter_output><empty_output><jupyter_text>Više: http://matplotlib.org/api/pyplot_api.htmlmatplotlib.pyplot.pcolormesh, http://matplotlib.org/users/colormaps.html<jupyter_code>mnormal = stats.multivariate_normal([0, 1], [[1, 1], [0.2, 3]])
mnormal.pdf([1,1])
x = np.linspace(-1, 1)
y = np.linspace(-2, 2)
X, Y = np.meshgrid(x, y)
shape(X)
shape(XY)
mnormal.pdf(XY)
plt.pcolormesh(X, Y, mnormal.pdf(XY))
plt.show()
plt.contourf(X, Y, mnormal.pdf(XY));
plt.contourf(X, Y, mnormal.pdf(XY), levels=[0,0.06, 0.07]);
plt.contour(X, Y, mnormal.pdf(XY));
x = linspace(-10,10)
X, Y = np.meshgrid(x, x)
Z = X*3 + Y
plt.contour(X, Y, Z);
plt.contour(X, Y, Z, levels=[0]);<jupyter_output><empty_output><jupyter_text>Kombinacija više grafikona:<jupyter_code>plt.contour(X, Y, Z, levels=[0])
plt.scatter([-5,-3,2,5], [4, 5, 2, 1])
plt.show()<jupyter_output><empty_output><jupyter_text>6.4. Histogram<jupyter_code>np.random.seed(42)
x = stats.norm.rvs(size=1000)
plt.hist(x);<jupyter_output><empty_output><jupyter_text>Više-manje istovjetno s:<jupyter_code>hist, bins = np.histogram(x)
centers = (bins[:-1] + bins[1:]) / 2
plt.bar(centers, hist);<jupyter_output><empty_output><jupyter_text>6.5. Podgrafikoni TODO  7. Pandas<jupyter_code>import pandas as pd
pd.__version__<jupyter_output><empty_output><jupyter_text>TODO  8. Sklearn<jupyter_code>import sklearn
sklearn.__version__<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1865233771848558, 'fraction_numerical': 0.0474337594960163, 'mean_word_length': 5.958315427589171, 'pattern_counts': {'":': 0, '<': 240, '<?xml version=': 0, '>': 244, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '61', 'dir': 'jupyter-structured-clean-dedup', 'id': '363532', 'license': 'CC0-1.0', 'n_tokens_mistral': 7966, 'n_tokens_neox': 7542, 'n_words': 1286, 'path': 'notebooks/SU-2015-0-SciPy.ipynb', 'repo_name': 'jsnajder/StrojnoUcenje', 'size': '16196'}