<jupyter_start><jupyter_text>COURSE: A deep understanding of deep learning SECTION: Gradient descent LECTURE: CodeChallenge: Unfortunate starting value TEACHER: <NAME>, sincxpress.com COURSE URL: udemy.com/course/dudl/?couponCode=202201<jupyter_code># import all necessary modules
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Gradient descent in 1D<jupyter_code># function (as a function)
def fx(x):
  return np.cos(2*np.pi*x) + x**2

# derivative function
def deriv(x):
  return -2*np.pi*np.sin(2*np.pi*x) + 2*x
# plot the function and its derivative

# define a range for x
x = np.linspace(-2,2,2001)

# plotting
plt.plot(x,fx(x), x,deriv(x))
plt.xlim(x[[0,-1]])
plt.grid()
plt.xlabel('x')
plt.ylabel('f(x)')
plt.legend(['y','dy'])
plt.show()

# random starting point
localmin = np.random.choice(x,1) #np.array([0])#

# learning parameters
learning_rate = .01
training_epochs = 100

# run through training
for i in range(training_epochs):
  grad = deriv(localmin)
  localmin = localmin - learning_rate*grad

localmin
# plot the results

plt.plot(x,fx(x), x,deriv(x))
plt.plot(localmin,deriv(localmin),'ro')
plt.plot(localmin,fx(localmin),'ro')

plt.xlim(x[[0,-1]])
plt.grid()
plt.xlabel('x')
plt.ylabel('f(x)')
plt.legend(['f(x)','df','f(x) min'])
plt.title('Empirical local minimum: %s'%localmin[0])
plt.show()<jupyter_output><empty_output><jupyter_text>Additional explorations<jupyter_code># 1) The derivative has a multiplicative factor of 2 in it. Is that constant necessary for the accuracy of the g.d. result?
#    Try removing that '2' from the derivative and see whether the model can still find the minimum. Before running the
#    code, think about what you expect to happen. Does reality match your expectations? Why is (or isn't) that factor necessary?
# 
# 2) What about the factor of '2' inside the np.sin() function? Is that important? Can you get an accurate result if you
#    remove it?
# 
# 3) Try setting the initial value to a small but non-zero number, e.g., .0001 or -.0001. Does that help the solution?
#<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13608345187292556, 'fraction_numerical': 0.020862968231389285, 'mean_word_length': 5.413373860182371, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '599230', 'license': 'MIT', 'n_tokens_mistral': 795, 'n_tokens_neox': 748, 'n_words': 226, 'path': '02_gradientDescent/DUDL_GradientDescent_CodeChallengeStartValue.ipynb', 'repo_name': 'amitmeel/DUDL', 'size': '2115'}