<jupyter_start><jupyter_text>심볼변수 스칼라, 벡터, 행렬 정의- `dscalar`, `dvector`, `dmatrix`<jupyter_code>x1 = T.dscalar('x1')
y1 = T.dscalar('y1')

x2 = T.dvector('x2')
y2 = T.dvector('y2')

x3 = T.dmatrix('x3')
y3 = T.dmatrix('y3')

type(x1), type(x2), type(x3)<jupyter_output><empty_output><jupyter_text>그래프 정의- 사칙연산과 Theano 수학함수로 다른 심볼변수 만들 수 있음- graph: new & existing variable 연산관계<jupyter_code>z1 = x1 + y1
type(z1)
u1 = T.exp(z1)
type(u1)
z2 = T.dot(x2, y2)
type(z2)
z3 = T.sum(x3) + T.mean(y3)
type(z3)<jupyter_output><empty_output><jupyter_text>심볼 프린트 및 그래프 시각화- `theano.printing.pprint`, `theano.printing.pydotprint`<jupyter_code>theano.printing.pprint(x1)
theano.printing.pprint(z1)
theano.printing.pprint(u1)
from IPython.display import SVG
SVG(theano.printing.pydotprint(z1, return_image=True, format='svg'))
SVG(theano.printing.pydotprint(u1, return_image=True, format='svg'))
SVG(theano.printing.pydotprint(z2, return_image=True, format='svg'))<jupyter_output><empty_output><jupyter_text>심볼 함수- `theano.function`- input & output 심볼변수 지정 - 일반함수<jupyter_code># x1+y1 = z1
f1 = theano.function(inputs=[x1, y1], outputs=z1)
f1(1, 2)<jupyter_output><empty_output><jupyter_text>- 벡터 & 행렬 함수<jupyter_code># np.dot
f2 = theano.function([x2, y2], z2)
f2([1, 2], [3, 4])
# np.sum + np.mean
f3 = theano.function([x3, y3], z3)
f3([[1], [2]], [[3], [4]])<jupyter_output><empty_output><jupyter_text>- 로지스틱함수<jupyter_code>s1 = 1 / (1 + T.exp(-x1))
logistic = theano.function([x1], s1)
logistic(3)
s2 = 1 / (1 + T.exp(-x2))
logistic2 = theano.function([x2], s2)
logistic2([0, 1])<jupyter_output><empty_output><jupyter_text>- 난수<jupyter_code>from theano.tensor.shared_randomstreams import RandomStreams

srng = RandomStreams(99)
rv_u = srng.uniform()
f_rv_u = theano.function([], rv_u)

f_rv_u(), f_rv_u(), f_rv_u(), f_rv_u(), f_rv_u()<jupyter_output><empty_output><jupyter_text>변수 갱신- 공유 메모리에 저장 by `shared`- 심볼그래프 사용X- `function` and `updates` eg. $w_{k+1} = w_k + x$<jupyter_code># 갱신되는 값: w
x = T.dscalar('x')
y = x
w = theano.shared(0.0, name='w')

update = theano.function([x], y, updates=[(w, w + x)])

w.set_value(f_rv_u())
w.get_value()
SVG(theano.printing.pydotprint(update, return_image=True, format='svg'))
update(1)
w.get_value()
update(2)
w.get_value()<jupyter_output><empty_output><jupyter_text>그래프 최적화<jupyter_code># x1을 10번 곱하지 않음
x1 = T.dvector('x1')
y1 = x1 ** 10
f1 = theano.function([x1], y1)

SVG(theano.printing.pydotprint(f1, return_image=True, format='svg'))
import numpy as np
x1 = np.ones(100000)
# 바보
%time x1 ** 10
# smart
%time f1(x1)<jupyter_output>CPU times: user 549 µs, sys: 0 ns, total: 549 µs
Wall time: 556 µs<jupyter_text>미분- `grad`<jupyter_code>x1 = T.dscalar('x1')
y1 = x1 ** 2
fy1 = theano.function([x1], y1)

SVG(theano.printing.pydotprint(fy1.maker.fgraph.outputs[0], return_image=True, format='svg'))
gy1 = T.grad(y1, x1)
fgy1 = theano.function([x1], gy1)

SVG(theano.printing.pydotprint(fgy1.maker.fgraph.outputs[0], return_image=True, format='svg'))<jupyter_output><empty_output><jupyter_text>- sigmoid<jupyter_code>x1 = T.dscalar('x1')
s1 = 1 / (1 + T.exp(-x1))
logistic = theano.function([x1], s1)

SVG(theano.printing.pydotprint(logistic, return_image=True, format='svg'))
gs1 = T.grad(s1, x1)
dlogistic = theano.function([x1], gs1)

SVG(theano.printing.pydotprint(dlogistic, return_image=True, format='svg'))
xx = np.linspace(-5, 5, 100)
y1 = np.hstack([logistic(xi) for xi in xx])
y2 = np.hstack([dlogistic(xi) for xi in xx])

plt.plot(xx, y1, label='logistic func')
plt.plot(xx, y2, label='derivative of logistic func')
plt.title('logistic func and derivative of it')

plt.show()<jupyter_output><empty_output><jupyter_text>Perceptron 구현<jupyter_code>y
from sklearn.datasets import load_iris
iris = load_iris()

idx = np.in1d(iris.target, [0, 2])
x = iris.data[idx, 0:2]
y = iris.target[idx] - 1 #y = 1 or -1

x1 = x[:, 0]
x2 = x[:, 1]

plt.scatter(x1[y==-1], x2[y==-1], label='setosa', c='r', marker='o',\
           s=100, edgecolor='k')
plt.scatter(x1[y==1], x2[y==1], label='virginia', c='b', marker='x',\
           s=100, edgecolor='k')
plt.xlabel('petal length')
plt.ylabel('petal width')
plt.legend()
plt.title('iris data')
plt.show()
X = T.dmatrix('X')
Y = T.dvector('Y')
np.random.seed(2)

w = theano.shared(0.001 * np.random.randn(2), name='w')
b = theano.shared(0.001 * np.random.randn(1)[0], name='b')

z = T.tanh(T.dot(X, w) + b)
cost = T.sum(T.maximum(0, -Y * z))

gw, gb = T.grad(cost, [w, b])
mu = 1e-6
train = theano.function([X, Y], cost, updates=[(w, w - mu * gw), (b, b - mu * gb)])
cost_history = []
for _ in range(100):
    cost_history.append(train(x, y))
    
plt.plot(cost_history)
plt.xlabel('epoch')
plt.ylabel('cost func')
plt.title('cost fluctuation according to train')
plt.show()
y_pred = T.sgn(z)
y_pred_func = theano.function([X], y_pred)

y_pred = y_pred_func(x)
y_pred
from sklearn.metrics import confusion_matrix
confusion_matrix(y, y_pred)<jupyter_output><empty_output><jupyter_text>연습1. y값은 0과 1이라는 값만 가진다.2. 활성화 함수는 로지스틱 함수를 사용한다.3. 코스트 함수는 다음과 같은 오차 제곱 함수를 사용한다.<jupyter_code>np.random.seed(2)
y = (y+1) / 2
X = T.dmatrix('X')
Y = T.dvector('Y')

w = theano.shared(0.001 * np.random.randn(2), name='w')
b = theano.shared(0.001 * np.random.randn(1)[0], name='b')

z = 1 / (1 + T.exp(-(T.dot(X, w) + b)))
cost = T.sum((Y - z)**2)

gw, gb = T.grad(cost, [w, b])
mu = 1e-6

train = theano.function([X, Y], cost, updates=[(w, w - mu * gw), (b, b - mu * gb)])
cost_history = []
for _ in range(50):
    cost_history.append(train(x, y))
    
plt.plot(cost_history)
plt.xlabel('epoch')
plt.ylabel('cost func')
plt.title('cost fluctuation according to train')
plt.show()
theano.printing.pprint(T.round(1.2))
y_pred = T.round(z)
y_pred_func = theano.function([X], y_pred)

y_pred = y_pred_func(x)
y_pred<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19548353732115153, 'fraction_numerical': 0.04051025685226685, 'mean_word_length': 5.504484304932736, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '620453', 'license': 'MIT', 'n_tokens_mistral': 2927, 'n_tokens_neox': 2965, 'n_words': 503, 'path': '07.DeepLearning/20.1 DeepLearning_theano.ipynb', 'repo_name': 'HenryPaik1/Study', 'size': '5801'}