<jupyter_start><jupyter_text>First-Order ODEsNote that a first-order ODE must be able to be written in the form \begin{align}\frac{d}{dx}\Psi + p(x)\Psi = g(x)\end{align}Where both $p(x)$ and $g(x)$ are continuous functions.  Example IConsider the first-order ODE:\begin{align}\frac{d}{dx}\Psi + (x+\frac{1 + 3 x^2}{1 + x+ x^3})\Psi = x^3 + 2x +x^2\frac{1 + 3x^2}{1 + x + x^3},\quad \Psi(0) = 1 \quad and \quad x\in [0,1]\end{align}Solve for $\Psi(x)$<jupyter_code># Define Differential Equation
# Initial condition
f_0 = 1

# differential equation
def g(x):
    return x**3 + 2*x + (x**2)*((1+3*x**2)/(1+x+x**3))

def p(x):
    return x + ((1+3*(x**2))/(1+x+x**3))
learning_rate = 0.003
epochs = 1000
display_step = 100

# Define NN
neural_network = m.MLP()

# Create optimizer
optimizer = optim.Adam(neural_network.parameters(), lr=learning_rate)

for epoch in range(epochs):
    optimizer.zero_grad()
    loss = utils.first_order_loss_with_ic(neural_network, p, g, f_0)
    loss.backward()
    optimizer.step()
    # print(loss.item())
    if epoch % display_step == 0:
        print(loss.item())
x = torch.linspace(0, 1, 50, requires_grad=True).unsqueeze(1)
y_approx = (f_0 + x * neural_network(x)).squeeze(0).detach().numpy().transpose()[0]
y_exact = ((torch.exp(-(x**2)/2)/(1+x+x**3))+x**2).squeeze(0).detach().numpy().transpose()[0]
x = x.squeeze(0).detach().numpy().transpose()[0]
plt.plot(x, y_approx, label='NN Approximation')
plt.plot(x, y_exact, label='Exact Solution')
plt.legend()<jupyter_output><empty_output><jupyter_text>Example II: More ComplicatedConsider the first-order ODE:\begin{align}\frac{d}{dx}\Psi + \frac{1}{5}\Psi = e^{-\frac{x}{5}}cos(x),\quad \Psi(0) = 0 \quad and \quad x\in [0,10]\end{align}Solve for $\Psi(x)$<jupyter_code># Define sample problem
# Initial condition
f_0 = 0

# differential equation
def g(x):
    return torch.exp(-x/5)*torch.cos(x)

def p(x):
    return 1/5

# Domain
domain = (0, 6)
learning_rate = 0.003
epochs = 5000
display_step = 500

# Define NN
neural_network = m.MLP()

# Create optimizer
optimizer = optim.Adam(neural_network.parameters(), lr=learning_rate)

for epoch in range(epochs):
    optimizer.zero_grad()
    loss = utils.first_order_loss_with_ic(neural_network, p, g, f_0, domain[0], domain[1], 20)
    loss.backward()
    optimizer.step()
    # print(loss.item())
    if epoch % display_step == 0:
        print(loss.item())
x = torch.linspace(domain[0], domain[1], 100, requires_grad=True).unsqueeze(1)
y_approx = (f_0 + x * neural_network(x)).squeeze(0).detach().numpy().transpose()[0]
y_exact = (torch.exp(-x/5)*torch.sin(x)).squeeze(0).detach().numpy().transpose()[0]
x = x.squeeze(0).detach().numpy().transpose()[0]
plt.plot(x, y_approx, label='NN Approximation')
plt.plot(x, y_exact, label='Exact Solution')
plt.legend()<jupyter_output><empty_output><jupyter_text>Second-Order ODEsNote that a second-order ODE must be able to be written in the form \begin{align}\frac{d^2}{dx^2}\Psi + a(x)\frac{d}{dx}\Psi + b(x)\Psi = g(x)\end{align}Where $a(x)$, $b(x)$, and $g(x)$ are continuous functions.  Second-Order Example 1.The method described will be used to approximate the solution $\Psi(x)$ to the following second-order ODE:\begin{align}\frac{d^2}{dx^2}\Psi + \frac{1}{5}\frac{d}{dx}\Psi + \Psi= -\frac{1}{5}e^{-\frac{x}{5}}cos(x), \quad \Psi(0) = 0, \; \frac{d}{dx}\Psi(0)=1 \quad over \quad x\in [0,2]\end{align}<jupyter_code># Define sample problem
# Initial condition
f_0 = 0
f_prime_0 = 1

# differential equation
def g(x):
    result = -(1.0/5.0)*torch.ones_like(x, requires_grad=True)*torch.exp(-x/5.0)*torch.cos(x)
#     print(result.requires_grad)
    return result

def a(x):
    result =(1.0/5.0)*torch.ones_like(x, requires_grad=True)
#     print(result.requires_grad)
    return result

def b(x):
    return torch.ones_like(x, requires_grad=True)

domain = (0, 2)
learning_rate = 0.0005
epochs = 2000
display_step = 100

# Define NN
neural_network = m.MLP()

# Create optimizer
optimizer = optim.Adam(neural_network.parameters(), lr=learning_rate)

for epoch in range(epochs):
    optimizer.zero_grad()
    loss = utils.second_order_loss_with_ic(neural_network, a, b, g, f_0, f_prime_0, domain[0], domain[1], 9)
    loss.backward()
    optimizer.step()
    
    if epoch % display_step == 0:
        print(loss.item())
x = torch.linspace(domain[0], domain[1], 100, requires_grad=True).unsqueeze(1)
y_approx = (f_0 + x*f_prime_0 + x**2 * neural_network(x)).squeeze(0).detach().numpy().transpose()[0]
y_exact = (torch.exp(-x/5)*torch.sin(x)).squeeze(0).detach().numpy().transpose()[0]
x = x.squeeze(0).detach().numpy().transpose()[0]
plt.plot(x, y_approx, label='NN Approximation')
plt.plot(x, y_exact, label='Exact Solution')
plt.legend()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.19091673675357443, 'fraction_numerical': 0.036585365853658534, 'mean_word_length': 4.87283950617284, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '253610', 'license': 'MIT', 'n_tokens_mistral': 1967, 'n_tokens_neox': 1888, 'n_words': 371, 'path': 'notebooks/.ipynb_checkpoints/Examples-checkpoint.ipynb', 'repo_name': 'shankal17/NNs-for-Differential-Equations', 'size': '4756'}