<jupyter_start><jupyter_text>Lab 3 : Convex OptimisationGradient Descent **Write the code following the instructions to obtain the desired results** Import all the required libraries<jupyter_code>import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Find the value of $x$ at which $f(x)$ is minimum : 1. Find $x$ analytically2. Write the update equation of gradient descent3. Find $x$ using gradient descent method  **Example 1** : $f(x)$ = $x^{2}$ + $x$ + $2$**Analytical** : $$\frac{d}{dx} f(x)=2x+1=0$$$$\frac{d^{2}}{dx^{2}} f(x)=2~(Minima)$$$$x=-\frac{1}{2}~(analytical~solution)$$**Gradient Descent Update equation** : $$x_{init}=4$$$$x_{updt}=x_{old}-\lambda (\frac{d}{dx} f(x)|x=x_{old})$$$$x_{updt}=x_{old}-\lambda (2x_{old}+1)$$**Gradient Descent Method** :Follow the below steps and write your code in the block below1. Generate $x$, 1000 data points from -10 to 102. Generate and Plot the function $f(x)$ = $x^{2}$ + $x$ + $2$3. Initialize the starting point ($x_{init}$) and learning rate ($\lambda$)4. Use Gradient descent algorithm to compute value of $x$ at which the function $f(x)$ is minimum5. Also vary the learning rate and initialisation point and plot your observations<jupyter_code>plt.figure();
x=np.linspace(-10,10,1000)
f=np.power(x,2)+x+2;
plt.plot(x,f);
x_val=list()
vals=[-8,4,-6,3,-3,1,-0.9,0.8,-0.1,-0.5]
for _ in range(10):
  x_init=vals[_];
  lr=0.01
  x_val.append(x_init);
  epoch=500;
  X=0;
  for i in range(epoch):
    X=x_init-lr*(2*x_init+1);
    if(X==x_init):
      break;
    x_init=X;
x_val=np.array(x_val);
f=np.power(x_val,2)+x_val+2;
print("Value of x at which f(x) is minimum=",X,"f(x)=",f[-1]);
plt.plot(x_val,f,color='k');<jupyter_output>Value of x at which f(x) is minimum= -0.5 f(x)= 1.75<jupyter_text>**Example 2** : $f(x)=x sinx$**Analytical** : Find solution analytically**Gradient Descent Update equation** : Write Gradient descent update equations**Gradient Descent Method** :Follow the below steps and write your code in the block below1. Generate $x$, 1000 data points from -10 to 102. Generate and Plot the function $f(x)$ = $x^{2}$ + $x$ + $2$3. Initialize the starting point ($x_{init}$) and learning rate ($\lambda$)4. Use Gradient descent algorithm to compute value of $x$ at which the function $f(x)$ is minimum5. Also vary the learning rate and initialisation point and plot your observations<jupyter_code>## Write your code here
x=np.linspace(-10,10,1000);
f=x*np.sin(x);
plt.plot(x,f);
x_init=6;
lr=0.001;
x_val=list();
epoch=1000;
for i in range(epoch):
    x_val.append(x_init);
    X=x_init-lr*(x_init*np.cos(x_init)+np.sin(x_init));
    if(x_init==X):
      break;
    x_init=X;
x_val=np.array(x_val);
f=x_val*np.sin(x_val);
plt.plot(x_val,f,'-o', color='k', mfc='r');
#print(x_val)<jupyter_output><empty_output><jupyter_text>Find the value of $x$ and $y$ at which $f(x,y)$ is minimum : **Example 1** : $f(x,y)$ = $x^{2}+y^{2}+2x+2y$**Gradient Descent Method** :Follow the below steps and write your code in the block below1. Generate $x$ and $y$, 1000 data points from -10 to 102. Generate and Plot the function $f(x,y)$ = $x^{2}+y^{2}+2x+2y$3. Initialize the starting point ($x_{init}$,$y_{init}$) and learning rate ($\lambda$)4. Use Gradient descent algorithm to compute value of $x$ and $y$ at which the function $f(x,y)$ is minimum5. Also vary the learning rate and initialisation point and plot your observations<jupyter_code>x=np.linspace(-10,10,1000);
y=np.linspace(-10,10,1000);

xx,yy=np.meshgrid(x,y);
f=np.power(xx,2)+np.power(yy,2)+2*(xx+yy);
fig=plt.figure()
ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(xx, yy, f, alpha=0.9)
fig=plt.figure();
xinit=0
yinit=0
epoch=1000;
xval,yval=list(),list()
for _ in range(epoch):
  xval.append(xinit);
  yval.append(yinit);
  xinit=xinit-0.01*(2*xinit+2);
  yinit=yinit-0.01*(2*yinit+2);
print("Value of x,y at which f(x,y) is minimum is at:(",xinit,",",yinit,")")

'''xval=np.array(xval);
yval=np.array(yval);
xx1,yy1=np.meshgrid(xval,yval);
f1=np.power(xx1,2)+np.power(yy1,2)+2*(xx1+yy1);'''
f1=np.power(xinit,2)+np.power(yinit,2)+2*(xinit+yinit);
anglesx = np.array(xval)[1:] - np.array(xval)[:-1]
anglesy = np.array(yval)[1:] - np.array(yval)[:-1]
#cp = plt.contour(xx, yy, f,  linewidths=1)

fig=plt.figure()
ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(xx, yy, f, alpha=0.6)
cp = ax.contour(xx, yy, f, linewidths=1)
plt.figure();
plt.clabel(cp, inline=1, fontsize=10)
cp = plt.contourf(xx, yy, f )
#plt.scatter(xinit,yinit,f);
plt.quiver(4, 4, xinit, yinit,scale=7)<jupyter_output>Value of x,y at which f(x,y) is minimum is at:( -0.9999999983170326 , -0.9999999983170326 )<jupyter_text>**Example 2** : $f(x,y)$ = $xsin(x)+ysin(y)$**Gradient Descent Method** :Follow the below steps and write your code in the block below1. Generate $x$ and $y$, 1000 data points from -10 to 102. Generate and Plot the function $f(x,y)$ = $xsin(x)+ysin(y)$3. Initialize the starting point ($x_{init}$,$y_{init}$) and learning rate ($\lambda$)4. Use Gradient descent algorithm to compute value of $x$ and $y$ at which the function $f(x,y)$ is minimum5. Also vary the learning rate and initialisation point and plot your observations<jupyter_code>x=np.linspace(-10,10,1000);
y=np.linspace(-10,10,1000);

xx,yy=np.meshgrid(x,y);
f=xx*np.sin(xx)+yy*np.sin(yy)
fig=plt.figure()
ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(xx, yy, f, alpha=0.9)
xinit=4
yinit=4
epoch=1000;
xval,yval=list(),list()
for _ in range(epoch):
  xval.append(xinit);
  yval.append(yinit);
  xinit=xinit-0.001*(xinit*np.cos(xinit)+np.sin(xinit));
  yinit=yinit-0.001*(yinit*np.cos(yinit)+np.sin(yinit));
print("Value of x,y at which f(x,y) is minimum is at:(",xinit,",",yinit,")")


f1=np.power(xinit,2)+np.power(yinit,2)+2*(xinit+yinit);
#cp = plt.contour(xx, yy, f,  linewidths=1)

fig=plt.figure()
ax = fig.add_subplot(111,projection='3d')
ax.plot_surface(xx, yy, f, alpha=0.3)
cp = ax.contour(xx, yy, f, linewidths=1)
plt.figure();
plt.clabel(cp, inline=1, fontsize=10)
cp = plt.contourf(xx, yy, f )<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1926485067279291, 'fraction_numerical': 0.049721037085658025, 'mean_word_length': 5.981672394043528, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '614173', 'license': 'CC0-1.0', 'n_tokens_mistral': 2606, 'n_tokens_neox': 2461, 'n_words': 593, 'path': 'Lab_3_Convex_Optimisation.ipynb', 'repo_name': 'Sourav1429/ANN_from_Scratch', 'size': '6094'}