<jupyter_start><jupyter_text>[A Regret Minimization Approach to Iterative Learning Control](https://arxiv.org/pdf/2102.13478.pdf )<jupyter_code>%load_ext autoreload
%autoreload 2
import jax
import jax.numpy as jnp
from deluca.igpc.ilqr import iLQR
from deluca.envs import PlanarQuadrotor<jupyter_output>WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)<jupyter_text>System - Planar Quadrotor with wind ([deluca implementation](https://github.com/google/deluca/blob/main/deluca/envs/classic/_planar_quadrotor.py)) Algorithms Compared* **Closed Loop ILQG** - [Li,Todorov](https://homes.cs.washington.edu/~todorov/papers/LiICINCO04.pdf) ([deluca implementation](https://github.com/MinRegret/deluca-igpc/blob/main/deluca/igpc/ilqr.py))* **iLC - Iterative Learning Control** [Abeel et al.](https://ai.stanford.edu/~ang/papers/icml06-usinginaccuratemodelsinrl.pdf) ([deluca implementation](https://github.com/MinRegret/deluca-igpc/blob/main/deluca/igpc/ilc.py))* **iGPC - Iterative GPC** [Agarwal et. al.](https://arxiv.org/abs/2102.13478) ([deluca implementation](https://github.com/MinRegret/deluca-igpc/blob/main/deluca/igpc/igpc.py)<jupyter_code>def cost(x, u, sim):
    return 0.1 * (u - sim.goal_action) @ (u - sim.goal_action) + (x.arr - sim.goal_state) @ (x.arr - sim.goal_state)

env_true, env_sim = PlanarQuadrotor.create(wind=0.4), PlanarQuadrotor.create()
#Initial actions
U0 = jnp.tile(env_sim.goal_action, (env_sim.H, 1))
# Warm up with either file or iLQR
warmup_steps = 8
X, U, k, K, c = iLQR(env_sim, cost, U0, warmup_steps, verbose=False)
%%capture iLC_output
## Run ILC
from deluca.igpc.ilc import iLC_closed
T = 40
iLC_result = iLC_closed(env_true, env_sim, cost, U, T, k, K, X, ref_alpha=0.2)
iLC_output()
%%capture iGPC_output
## Run IGPC
from deluca.igpc.igpc import iGPC_closed
T = 40
iGPC_result = iGPC_closed(env_true, env_sim, cost, U, T, k, K, X, 'de', ref_alpha=0.2, lr=0.001)
#Cell takes ~8 mins to run
iGPC_output()
%%capture iLQRO_output
## Running the ILQR(Oracle) - This is the unachievable baseline
iLQRO_result = iLQR(env_true, cost, U, 10, k, K, X, verbose=True)
iLQRO_output()
import matplotlib.pyplot as plt
from deluca.igpc.plot_utils import process_for_plot, zero_cost

iLC_perf = process_for_plot(str(iLC_output.stdout), 60)
iGPC_perf = process_for_plot(str(iGPC_output.stdout), 60)
iLQRO_perf = process_for_plot(str(iLQRO_output.stdout), 60)
iLQG_cost = zero_cost(str(iLC_output.stdout))

plt.plot(list(range(60)), [iLQG_cost]*60, label='iLQG')
plt.plot(iLC_perf[0], iLC_perf[1], 'green', label='iLC')
plt.plot(iGPC_perf[0], iGPC_perf[1], 'red', label='iGPC')
plt.plot(iLQRO_perf[0], iLQRO_perf[1], 'black', label='iLQR(oracle)')
plt.legend()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14972875226039783, 'fraction_numerical': 0.02278481012658228, 'mean_word_length': 6.184415584415585, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '72420', 'license': 'CC0-1.0', 'n_tokens_mistral': 1228, 'n_tokens_neox': 1116, 'n_words': 218, 'path': 'PlanarQuadrotorExperiment.ipynb', 'repo_name': 'MinRegret/deluca-igpc', 'size': '2765'}