Our advantage does not come for free, but rather we assume that we can observe what a dynamical system does - a systems core geometry and its motions -, which, however, is often apparent, as mentioned in the abstract. We speculate that this additional knowledge will give us the advantage in comparison to learning dynamical models without knowledge of the systems geometry. 
A rigorous research of this hypothesis is a promising new research direction. We believe GEM provides a step towards structured-RL and opens up future research on the connection between the manifold's structure and perception action cycle [1]}, [2]}.

Supplementary Materials
In the supplementary materials, we provide a) additional details of the Lie groups used in our work, b) a working example of performing calculations with these groups, c) hyperparameters used in our experiments. The supplementary materials are not mandatory for understanding the main ideas in the paper.The line numbers, the equation references, and the citations continue/refer those in the main paper.

Special Orthogonal Group \(SO\)
A relevant example for applications in our work is the group of rotations in 3D space—Special Orthogonal group, \(SO(3)\) . This section provides a detailed overview of \(SO(3)\) . Derived from representation theory [3]}, the elements of \(SO(3)\)  are orthogonal matrices with unit determinant. The corresponding Lie algebra, \(\mathfrak {g}=so(3)\) , is a 3-dimensional vector space. A commonly used basis for \(so(3)\)  is:
\(\!\!\!\!E_1\!\!=\!\!\begin{bmatrix}0&0&0\\0&0&-1\\0&1&0\end{bmatrix}\!\!,E_2\!\!=\!\!\begin{bmatrix}0&0&1\\0&0&0\\-1&0&0\end{bmatrix}\!\!,E_3\!\!=\!\!\begin{bmatrix}0&-1&0\\1&0&0\\0&0&0\end{bmatrix}.\) 


From (REF ) in the main text, any element in \(g_{\alpha }\in so(3)\)  can be represented by some coefficient vector \(\alpha {=}(\alpha _1, \alpha _2, \alpha _3)\) , i.e. \(g_{\alpha }=\alpha _1 E_1 + \alpha _2 E_2 + \alpha _3 E_e\) .
The matrix representation of \(SO(3)\)  allows us to conveniently use the standard matrix multiplication for the composition operator \(\circ \)  and the standard matrix exponential for \(\exp \)  in equation (REF ) from the main text.
Alternatively, the elements of \(SO(3)\)  can be constructed from the axis-angle representation, given by a rotation axis vector, \(u\) , and a corresponding angle, \(\theta \) :
\(G \!= \!\cos (\theta )\!\!\cdot \!\!\mathbf {I}\!+\! \sin (\theta )\!\!\cdot \!\!(u\times u) \!+\! (1\!-\!\cos (\theta ))\!\!\cdot \!\!(u\otimes u),\) 

where \(\mathbf {I}\) , \(^{\prime }\times ^{\prime }\) , and \(^{\prime }\otimes ^{\prime }\)  are the identity matrix with dimension 3 by 3, the cross product, and and the outer product, respectively. Then, the angular components of a dynamical system state can be represented by rotation matrices, provided one knows the primitive geometric motions such as axes of rotations. The later can be revealed either from the hardware specification [4]}, or by computer vision techniques [5]}, [6]}. .
\(SO3\)  and \(so3\)  in GEM: given a state, we represent it by rotation \(G_t\in \mathfrak {G}\)and/or translation matrices, \(SE(3)\) ., and predict the Lie algebra coefficient vector, \(\hat{\alpha }_{\psi }\) . Thus, the predicted rotation matrix, \(\hat{G}_{t+1}\) , is given by Eq. (REF ): \(\hat{G}_{t+1}=\exp (\hat{\alpha }_{\psi })\circ G_t\) . The coefficient vector, \(\hat{\alpha }_{\psi }\) , is represented by deep neural network with parameters \(\psi \) .

Special Euclidean Group \(SE\)
For completeness, we overview the Special Euclidean group, \(SE(3)\)  in Supplementary Material. These two groups characterize the primitive geometric motion of rigid body. Derived from representation theory [3]}, the elements of \(SE(3)\)  are essentially \(SO(3)\)  matrices that contain coordinates for transformations. The corresponding Lie algebra, \(\mathfrak {g}=se(3)\) , is a 6-dimensional vector space. A commonly used basis for \(se(3)\)  is:
\(\!\!\!\!E_1\!\!=\!\!\begin{bmatrix}0&0&0&0\\0&0&-1&0\\0&1&0&0\\0&0&0&0\end{bmatrix}\!\!,E_2\!\!=\!\!\begin{bmatrix}0&0&1&0\\0&0&0&0\\-1&0&0&0\\0&0&0&0\end{bmatrix}\!\!,E_3\!\!=\!\!\begin{bmatrix}0&-1&0&0\\1&0&0&0\\0&0&0&0\\0&0&0&0\end{bmatrix}.\) 

\(\!\!\!\!E_4\!\!=\!\!\begin{bmatrix}0&0&0&1\\0&0&0&0\\0&0&0&0\\0&0&0&0\end{bmatrix}\!\!,E_5\!\!=\!\!\begin{bmatrix}0&0&0&0\\0&0&0&1\\0&0&0&0\\0&0&0&0\end{bmatrix}\!\!,E_6\!\!=\!\!\begin{bmatrix}0&0&0&0\\0&0&0&0\\0&0&0&1\\0&0&0&0\end{bmatrix}.\) 


From (REF ) in the main text, any element in \(g_{\alpha }\in se(3)\)  can be represented by some coefficient vector \(\alpha {=}(\alpha _1, \alpha _2, \alpha _3, \alpha _4, \alpha _5, \alpha _6)\) , i.e. \(g_{\alpha }=\alpha _1 E_1 + \alpha _2 E_2 + \alpha _3 E_3 + \alpha _4 E_4 + \alpha _5 E_5 + \alpha _6 E_6\) .
The matrix representation of \(SE(3)\)  allows us to conveniently use the standard matrix multiplication for the composition operator \(\circ \)  and the standard matrix exponential for \(\exp \)  in equation (REF ) from the main text.

An example: Reacher
We provide an example of an environment as it goes through the GEM process.
System description: Reacher is a task of guiding a robotic arm to a randomized goal point. This arm is built from a limb that rotates relative to a fixed center point, the second limb rotates relative to the end of the first limb. Reacher will be built from two \(SO2\)  rotational groups (\(SO2\)  is a subset of \(SO3\)  discussed in  where angles are just in two dimensions) - one group will represent the angle of the primary limb position relative to its center pivot (\(\theta _1\) ), and the second group will represent the angle of the second limb position relative to the joint between the two limbs (\(\theta _2\) ). The angles form the static state of reacher (\(\rho = \theta = \begin{bmatrix} \theta _1 \\ \theta _2 \end{bmatrix} \) ). There are also two angular velocities for each limb respectively that form the dynamic state (\(\upsilon = \begin{bmatrix} \upsilon _1 \\ \upsilon _2 \end{bmatrix}\) ) and two applied torques that make up the action (\({a}\) ).
Reacher Initial Angles converted to Lie Group representation \(SO(2)\) , \(\theta \)  contains both angles and we apply each angle sequentially:
\(  \rho \rightarrow \mathbf {\theta } \rightarrow \begin{bmatrix} cos(\mathbf {\theta }) & -sin(\mathbf {\theta }) \\ sin(\mathbf {\theta }) & cos(\mathbf {\theta })\end{bmatrix} = G \in SO2 \) 
\(  \mathcal {L}_{\phi }^{\alpha }\bigg {(}\begin{bmatrix} G \\ \upsilon \\ a\end{bmatrix}\bigg {)} = \hat{\alpha } = \begin{bmatrix} \hat{\alpha _1} \\ \hat{\alpha _2} \end{bmatrix} \) 

The coefficients are applied to the basis matrix of the algebra for SO2, and an exponential map is taken to produce the relevant rotational matrix described in REF . Then a matrix multiplication is performed for each group to predict the future position of the limb as described in equation REF .
\(  g_{\hat{\alpha }} = \begin{bmatrix} 0 & \hat{\alpha } \\ -\hat{\alpha } & 0\end{bmatrix} \xrightarrow{} \begin{bmatrix} cos(\hat{\alpha }) & -sin(\hat{\alpha }) \\ sin(\hat{\alpha }) & cos(\hat{\alpha })\end{bmatrix} = G_{\hat{\alpha }}\) 
\( \hat{G^{\prime }} = \begin{bmatrix} cos(\theta ) & -sin(\theta ) \\ sin(\theta ) & cos(\theta )\end{bmatrix} \begin{bmatrix} cos(\hat{\alpha }) & -sin(\hat{\alpha }) \\ sin(\hat{\alpha }) & cos(\hat{\alpha })\end{bmatrix} \) 
These coefficients are first order derivatives of position, if one were to apply the coefficient \(\frac{\pi }{2}\)  to the first limb, it would rotate the limb 90 degrees counterclockwise. This essentially means we apply a rotational matrix of magnitude \(\alpha \)  radians to our group.
After predicting coefficients, the velocity model predicts future velocities.
\(  \mathcal {L}_{\psi }^{\upsilon } \bigg {(}\begin{bmatrix} G \\ \upsilon \\ a \\ \hat{\alpha } \end{bmatrix} \bigg {)} = \Delta \hat{\upsilon ^{\prime }} = \begin{bmatrix} \Delta \hat{\upsilon _1^{\prime }} \\ \Delta \hat{\upsilon _2^{\prime }} \end{bmatrix}\) 
GEMs forward prediction is complete. The loss is takes as sum of a frobenius norm difference between true delta resultant \( G^{\prime }\)  state and predicted delta resultant state \( \hat{G}^{\prime }\)  as well as a two norm difference of the true resultant \(\Delta \upsilon ^{\prime }\)  and the predicted \(\Delta \hat{\upsilon }\) .
\(  l_{\text{reacher}} = \Vert  \hat{G^{\prime }} - G^{\prime }\Vert _F^2 + \Vert \Delta \hat{\upsilon ^{\prime }} - \Delta \upsilon ^{\prime }\Vert _2^2\) 

Experiment details
The environments utilized in our experiments are from the MuJoCo control suite [4]}. All experiments are run over 5 random seeds and we report the average and standard deviation. The code to run the experiments can found at https://tinyurl.com/GEMMBRL.
Hyperparameters
Our neural networks are multi-layer perceptions (MLPs) built using pytorch. For each task, we search over multiple hidden sizes for each model.
For all tasks but ant and humanoid, the hidden sizes searched over are \([100], [100, 100], [100, 100, 100]\) . For ant and humanoid they are \([512], [512, 512], [512, 512, 512]\) . The best performing ones are used in our experiments.
For optimizer we use Adam with learning rate \(5\cdot 10^{-4}\) . A batch size of 10 is used for all experiments.
Offline data samples are collected using Soft Actor Critic.
For the models trained in the offline setting, we train until convergence which take 10000 training iterations or 100000 training samples. Validation set contain 10000 samples and testing set 20000.

Ensemble Details
For the ensemble experiments, we took 5 base models and outputted their average. They were trained in unison via total loss from all networks.

Planning Algorithm
We used MPPI optimizer for planning (Link to codebase: https://github.com/kzl/lifelong_rl). Actions were taken by planning via MPC on trained dynamics model.

Trajectory Prediction Experiments
The models in this experiment are trained in an offline setting. We compare prediction power on trajectories gathered from Soft Actor Critic. In the trajectory, we pick a random starting position and take the actions from the ground truth trajectory, and move the dynamics forward based on our trained dynamics model, while keeping track of the deviation from the ground truth state.
Then average per step error is reported for each prediction horizon in figure  and . Images were sampled from these trajectories as well.

Planning Experiments
The models in this experiment were trained in an offline setting. Planning was done in an open loop setting: once an environments is initialized, the planner and model never receive an update from the environment besides the first initial state. Rewards are then tracked from each step in the trajectory. Both GEM and the baselines are given the reward function, which is a function of the state.

Model Based Reinforcement Learning Experiments
The models in this experiment were trained in an online setting. The models iterate between planning out trajectories while collecting data and training on the collected data, this was done for 300 iterations. Though, we plot samples trained on to maintain fair comparison between models. In the planning and data collection portion, the model is only updated from the environment every fifth step. Average final rewards over five random seeds are tracked relative to number of samples collected. This code is under MBRL experiments in ExperimentScripts at https://tinyurl.com/GEMMBRL.

PETS Experiments
We take the original PETS [9]} code (pytorch implementation used: https://github.com/quanvuong/handful-of-trials-pytorch) and modify the input and objective of the dynamics model which is done by adding the observation to group transformation for the input, and predicting coefficients for output. The input takes in group state and velocity (similar to PETS alone). Instead of predicting direct delta state, the probabilistic ensemble is trained to predict in coefficient (algebra) space as well as delta velocity. From there, we apply (REF ) to get the next predicted state from the coefficients. Though there is no two step network, just one ensemble network. This was done to not change the other structures in the original PETs code. The rest of the code is the same. GEM+PET experiment appears under the PETS folder in https://tinyurl.com/GEMMBRL. Figure  shows the comparison results for different environments.
