We consider three more complex, dynamic domains. First, we consider Harvest [1]}, a common resource pool management problem, with engineered features and without a cleaning beam. We describe our experiment in detail in apx:domain. In this domain, we contract by imposing a fine \(\theta \in [0, 10]\) , paid evenly to the other agents, on consuming an apple in a region of low density (i.e. less than 4 apples within a radius of 5). We also consider Cleanup [1]}, an asymmetric social dilemma, which we also describe in apx:domain. In this domain, we contract by providing a reward \(\theta \in [0, 0.2]\)  per garbage piece cleaned, paid for evently by the other agents. Implementations for Harvest and Cleanup are based on code from the GitHub repository of [3]}. Third, we consider an asymmetric social dilemma inspired by utonomous mobility. A set of \(N-1\)  cars approaches a merge, an ambulance behind them. The ambulance incurs a penalty of 100 per timestep that it has not reached the end of a road segment past the merge. The cars in front also want to get to the end of the road segment, but incur a penalty of only 1 per timestep. They are limited to one-fourth of the velocity that the ambulance can go. We assume access to controllers preventing cars from colliding (stopping cars short of crashing into another car) and managing merging, and so the actions \(a_i \in [-0.1, 0.1]\)  only control the forward acceleration of each vehicle. A dilemma arises as cars prefer to drive to the merge fast, not internalizing the strong negative effect this has on the ambulance. To mitigate this dilemma, we give the ambulance the option to propose a per-unit subsidy of \(\theta \in [0, 100]\)  to the cars at the time of ambulance crossing, where each car is transferred \(\theta \)  times its distance behind the ambulance it is at time of merge by the ambulance. If a car is ahead of the ambulance at time of reward, it pays the ambulance \(\theta \)  times its distance it is ahead of the ambulance.
