<jupyter_start><jupyter_text>--- Project Microeconometrics | Summer 2020, M.Sc. Economics, University of Bonn | [<NAME>](https://github.com/VincentSelz) Replication of “A Structural Analysis of Disappointment Aversion in a Real Effort Competition” by <NAME> and <NAME> (2012)  ---  Miscellaneous In this notebook, I replicate the findings of > Gill, David, and <NAME>. 2012. A Structural Analysis of Disappointment Aversion in a Real Effort Competition. *American Economic Review*, 102 (1): 469-503.The original paper and the corresponding data and code can be found [here](https://www.aeaweb.org/articles?id=10.1257/aer.102.1.469). For their analysis, the authors use Stata and Matlab. To view this notebook:* Recommended, is to clone the repository and to locally view it via jupyter notebook.* Unfortunately, the notebook cannot be inspected with mybinder or nbviewer, since rendering it returns an error.Aside, from the accompanying text, the results are labelled into three categories:    Successful replication: All the results are replicated.    Extension: The results of the paper are expanded.    Notes: Pointers to the workflow.    Generally, most code, with the exception of R code, is moved to the auxiliary files to enhance readability.  Table of Contents1&nbsp;&nbsp;Introduction2&nbsp;&nbsp;Experimental Design2.1&nbsp;&nbsp;Slider Task2.2&nbsp;&nbsp;Experimental Procedure3&nbsp;&nbsp;Theoretical Predictions3.1&nbsp;&nbsp;Canonical Model3.2&nbsp;&nbsp;Disappointment Aversion4&nbsp;&nbsp;Identification Strategy5&nbsp;&nbsp;Empirical Results5.1&nbsp;&nbsp;Data Exploration5.2&nbsp;&nbsp;Reduced Form Analysis5.2.1&nbsp;&nbsp;Model5.2.2&nbsp;&nbsp;Results5.2.3&nbsp;&nbsp;Fixed vs. Random Effects5.2.4&nbsp;&nbsp;Further Investigation5.3&nbsp;&nbsp;Structural Model5.3.1&nbsp;&nbsp;Empirical Specification5.3.2&nbsp;&nbsp;Method of Simulated Moments5.3.3&nbsp;&nbsp;Empirical Moments5.3.4&nbsp;&nbsp;Weighting Matrix5.3.5&nbsp;&nbsp;Simulation5.3.6&nbsp;&nbsp;Further Steps6&nbsp;&nbsp;Critical Assessment7&nbsp;&nbsp;Appendix7.1&nbsp;&nbsp;Effort Distribution Per Round7.2&nbsp;&nbsp;Prize Distribution7.3&nbsp;&nbsp;Moment Computation Specifications8&nbsp;&nbsp;References<jupyter_code>%load_ext rpy2.ipython
%load_ext autoreload
%autoreload 2
%matplotlib inline


from datetime import datetime
from IPython.display import HTML
from IPython.display import display
from linearmodels.panel import PanelOLS
from scipy.stats import chisquare
import numpy as np
import pandas as pd
import pandas.io.formats.style
import respy as rp
import scipy.stats as stats
import seaborn as sns
import matplotlib as plt
import matplotlib.pyplot as pyplt

from auxiliary.helper import *
from auxiliary.msm import *
from auxiliary.overview import *
from auxiliary.simulation import *

# Displays floats only to the third decimal 
pd.set_option("display.precision", 3)
pd.set_option('display.max_colwidth', -1)
%%R

library(plm)
library(haven)
library(msm)
library(stargazer)
library(car)<jupyter_output><empty_output><jupyter_text>--- Introduction ---Gill & Prowse (2012) use a real effort tournament in a laboratory experiment to test experimentally whether agents are disappointment averse. Pairs of subjects need to complete a real effort task sequentially. The payoff is determined by a lottery where the probabilities of winning are linear in the difference in the agent's efforts. Hence, in the standard framework, the effort of the *First Mover* should not impact the choice of effort for the *Second Mover*. Yet, as predicted by Gill & Prowse's model of disappointment aversion, subjects exhibit a discouragement effect, where high effort by the *First Mover*, impacts the effort by the *Second Mover* negatively. Their effort choices are strategic substitutes.Gill & Prowse (2012) use a reduced form **random effects panel regression** to estimate potential effects. Given the substantial variation induced by their experimental design, they subsequently use the **Method of Simulated Moments (MSM)** to estimate the parameters of a structural model of disappointment aversion. This structural model allows them to estimate the strength of disappointment aversion on average but also to estimate the extent of heterogeneity in disappointment aversion across the population. This notebook will attempt to replicate the results of Gill & Prowse (2012) and will further add extensions in data exploration and the reduced form analysis. If possible Python is used to replicate the findings, yet if not possible R is used. For instance, the random effects regression with the common Python packages did yield close results but did not reproduce the same numbers reported in the paper. Unfortunately, I was not capable to reproduce the results of the structural estimation. Nonetheless, the steps taken towards the estimation of structural model are outlined. For this implementation, the documentation by [respy](https://respy.readthedocs.io/en/latest/) is closely followed. In the following section, the experimental design is shown. Next, the identification strategy is discussed, followed by the theoretical predictions. Then, the empirical analysis and extension thereof are shown. Lastly, the paper will be critically assessed. ---  Experimental Design---The six experimental session took place from late February to early March 2009 at the Nuffield Centre for Experimental Social Sciences (CESS) in Oxford. Per session twenty subjects participated in the study, leading to a subject pool of 120 participants in total. Each of them were paid a show-up fee of 4 pounds and went on to earn a further 10 pounds on average.   Slider Task The slider task consists of a single screen displaying numerous sliders. The number and position of sliders is not varied across subjects or across repetition of the task. At first, the sliders who range from 0 to 100 are placed at 0. It is possible to place the slider at every integer position between 0 and 100. The integer position is displaced next to the slider. The sliders can be adjusted and readjusted an unlimited number of times. The task for the subjects is to position as many sliders as possible at 50 during the time allocated to the subject. The number of sliders positioned at 50 constitutes the **points score**. The points score and the time remaining is displayed to the subject. In this experiment, 48 sliders were displayed and the subjects had 120 seconds to position them. Below, the schematic representation illustrates the task at hand.  Experimental ProcedureEach session consists of 10 *First Movers* and 10 *Second Movers*, who stay in their role for the duration of their session. After two practice rounds, the subjects played 10 paying rounds. The prize for each pair was drawn randomly from {0.10,0.20,...3.90} and then revealed to the pair members. The task was completed sequentially, where the *Second Mover* observes the points score of the First Mover. This information is also displayed next to the alloted time and the own points score during the task. The probability to win the ensuing lottery is determined by\begin{equation*}P_i(e_i,e_j) = \frac{e_i - e_j + 50}{100}\end{equation*}were $e_i$ in this setting corresponds to the points score of player $i$, and respectively $e_j$ for player $j$. Henceforth, effort is used synonymously to the points score. This use of the term is unlike the traditional literature, where effort is the associated cost of action. Here, it directly corresponds to the behavior of the subjects.After each round the subjects were paired with another subjects. To ensure that subjects never meet each other, the *'no contagion'* matching algorithm by Cooper et al. (1996) is used. This prevents strategic action by the subjects, where they collude by decreasing their efforts. In fact, zero effort by both subjects yields the social optimum.  --- Theoretical Predictions---The paired subjects compete with each other to win a fixed prize $v$. First, the first movers works at the slider task and subsequently the second mover chooses his effort. Together the above stated probability to winning prize $v$ is determined. The probabilities are linear in the difference of points scores. For their analysis, the authors only focus on the Second Mover. This leaves them with a pure optimization problem.     Canonical ModelFor the canonical model to work, the separability assumption common in tournament literature is used. The assumption states that the utility $U_{2}$ can be separated into the *material utility* $u_{2}(y_{2})$ and the *cost of effort* $C(e_{2})$. This implies cost of effort does not depend on whether the agent wins the prize and the valuation of the prize does not change with increasing effort. This leaves the agent with the expected utility:\begin{equation*}EU_2(e_2,e_1) = P_2(e_2,e_1)u_2(v) + \left(1 - P_2(e_1,e_2)\right) u_2(0) - C_2(e_2)\end{equation*}Given the linearity of the winning probabilities in the difference of efforts, the effort $e_1$ of the First Mover has no impact on the Second Mover's effort $e_2$. This leads to the proposition:**PROPOSITION 1:** *In the canonical model without disappointment aversion the Second Mover’s optimal effort $e_2$ (or set of optimal efforts) does not depend on the First Mover’s effort $e_1$.*The relative lack of further structural assumptions makes this model malleable. For instance, the results still holds when risk aversion, inequity aversion (Fehr and Schmidt 1999) or a fixed reference point are webbed into the utility function.  Disappointment AversionThe basis of the model of disappointment aversion is that agents are sensitive to deviations from their expectations.  They suffer a psychological loss when they receive less than what they expect and experience elation when they receive more. For instance, an employee feels disappointed by a raise because she expected the raise to be bigger. These potential psychological losses and gains, in turn, inform the agent in her decision process.The agent compares the material utility $u_2(y_2)$ to a reference point $R_2$. She is experiencing a loss when the material utility is less than the reference point and joy when the material utility is above this reference level.  Without further assumptions on the form of this reference point $R_2$, this leaves the agent with the following utility\begin{equation*}U_2(e_2,R_2,e_1) = u_2(y_2) + I_{u_2(y_2) \geq R_2}G_2(u_2(y_2) - R_2) + I_{u_2(y_2) \leq R_2}L_2(u_2(y_2) - R_2) - C_2(e_2)\end{equation*}The loss function $L_2(x)$ is strictly negative for all $x  0$. At $x = 0$ both functions equal zero. Loss aversion dictates that $G_2(x)  0$. The losses are more painful than equal-sized gains are pleasurable. Traditionally, this reference point is set exogenously (Kahneman and Tversky 1979). Yet, such a fixed reference point does not link the First Mover effort to the Second Mover effort. To model disappointment aversion the reference point must be formed endogenously. Köszegi and Rabin (2007) propose a *choice-acclimating* reference point, where the reference point is equal to the expected utility. In this implementation the reference point is dependent on the effort of both the Second Mover herself and the First Mover's effort. In contrast to the model by Köszegi and Rabin, the Second Mover is aware that her effort choice affects the reference point when optimizing her behavior. In this setting this leads to $R_2 = vP_2(e_2,e_1)$. This step is crucial to allow for a possible discouragement effect. As is custom in the literature, piece-wise linearity is imposed on the gain-loss utility. $l_2$ is the slope of the loss function and $g_2$ is the slope of the gain function. Loss aversion imposes $l_2 > g_2$ and leads to the (strictly positive) definition of the strength of disappointment aversion\begin{equation*}\lambda_2 \equiv l_2 - g_2.\end{equation*}Thus, the disappointment averse expected utility can be formulated as \begin{equation*}EU_2(e_2,e_1) = P_2 v + g_2(v - vP_2) + (1 - P_2)(0 + l_2(0 + vP_2) - C_2(e_2) \\= vP_2 - \lambda_2vP_2(1 - P_2) - C_2(e_2) \end{equation*}The extra term in the middle of the equation is coined the **disappointment deficit** by the authors. This disappointment deficit is proportional to the variance of the Second Mover's two-point distribution of monetary payoffs. Due to disappointment aversion, the Second Mover dislikes variance in her monetary payoff. This is a key distinction to risk aversion, where agents only care about their probability of winning and not about the variance in their potential payoffs. This variance is strictly concave and maximized at $P_2$. When the efforts yield a zero probability for the Second Mover to win, she has a reference point of zero and combined with the probability she can never be disappointed. Yet at this point, small increases in probability lead to large increases in the variance of monetary payoffs. These increases get smaller but continue until $P_2 = \frac{1}{2}$. At this point the variance reaches its highest point. From this point on increases in $P_2$ reduce the variance of monetary payoffs. This relationship can expressed as: **PROPOSITION 2:** *When the Second Mover is disappointment averse, higher First Mover effort discourages the Second Mover: the Second Mover’s optimal effort $e_2$ is always (weakly) decreasing in the First Mover’s effort $e_1$.*In order to derive an analytical expression, the authors assume a quadratic cost of effort of the form \begin{equation*}C_2(e_2) = be_2 + \frac{ce_{2}^{2}}{2}.\end{equation*}With this term the Second Mover's reaction function can be expressed. **PROPOSITION 3:** *Suppose a disappointment averse Second Mover has a quadratic cost function and a strictly concave objective function, i.e., $\gamma_2c_2 − \lambda_2v > 0$. When the action space is continuous, the slope of the Second Mover’s reaction function in the interior is given by*\begin{equation*}\frac{de_2}{de_1} = \frac{-\lambda_2v}{2\gamma^2c - \lambda_2v} < 0\end{equation*}*which becomes strictly more negative in the strength of disappointment aversion $\lambda_2$ and the value of the prize $v$. When the action space is discrete, the discrete analog of the reaction function behaves similarly.* --- Identification Strategy---In their paper Gill and Prowse aim to evaluate whether subjects exhibit disappointment aversion. In order to do that, they set up the aforementioned laboratory experiment. The causal graph below illustrates their identification strategy. We can assume that the effort of the Second Mover's $e_2$ is affected by the value of the prize $v$ and the effort of the First Mover $e_1$. In addition to that there can be learning throughout the ten played periods. Strictly speaking this term is inaccurate, since it actually captures to very different ideas of learning. First, the learning of the Slider Task, where the agent improves his/her performance by experience. But second, there can be meta-learning, where the agent gets more selective under which conditions she exerts effort. ![ERROR:Something went wrong. Here should be a causal graph](files/AdvancedCausalGraph.png)Figure 2: Causal Graph (This graph is drawn with [draw.io](https://app.diagrams.net/).)The advantages of an laboratory experiment are evident in the identification strategy by the authors. As the experimenters have total control over the environmental variables, they can randomize all those links, making for a straightforward identification. The prizes are drawn from a uniform distribution between 0.10 and 3.90 Pounds (with 0.10-steps). This variation also has advantages in the construction of structural equations as will be seen later. The ten playing rounds make it possible to control for individual specific effects and the matching algorithm ensures that there is no feedback back from the Second Mover effort to the First Mover, thus eliminating possible strategic interaction. The learning effect can be, albeit imperfectly, controlled for with round dummies. Thus, all channels can effectively be controlled for.   --- Empirical Results--- Data ExplorationBefore we are turning to the empirical analysis to estimate the strength of disappointment aversion, we can visually inspect the data to see why the authors pursue their empirical strategy. The data consists of 120 subjects, 60 First and 60 Second Mover that are all observed over 10 rounds. Of the 120 subjects, one subject failed to correctly position any slider during the training or playing rounds. The dataset excluding this subject is called **df_2_pref** and is used mainly for the analysis. The points score is interpreted as the effort choice by the subject. Although still discrete, this gives them a finely gradated measure of effort.The variable of interest is the Second Mover's effort and, thus, the authors are performing a conditional analysis of the Second Mover's effort choice. Their reasoning is that this analysis is sufficient for the purpose of identifying and estimating the strength of disappointment aversion. Including the First Mover's effort would make further assumptions necessary and increase the computational complexity substantially.      Replication of Table 1 of the paper.<jupyter_code>df_1, df_2, df_2_pref = get_datasets()
rslt = effort_overview(df_2_pref)
print('Table 1: Summary of the First and Second Mover efforts')
rslt<jupyter_output>Table 1: Summary of the First and Second Mover efforts<jupyter_text>One important inference that can be made is that the maximum of effort is 41 correctly positioned sliders. This is lower than the highest possible maximum of 48 sliders. If, subjects reached this limit, this would suggest that the task is not well suited to identify exerted effort. Another inference that can be made is that the effort choices increase over the rounds, suggesting that learning indeed takes place.  Note: The following graphs and charts are extensions to the paper.<jupyter_code>effort_chart(rslt)<jupyter_output><empty_output><jupyter_text>This chart visualizes Table 1 of the paper. The black bar is indicating the standard deviation of the respective means. Clearly, there appears to be no clear distinction between $e_2$ and $e_1$, when simply inspecting these descriptives. Effort averages are increasing from about 22 correctly positioned sliders to more than 26 over the playing rounds. A closer look at the effort distribution per round shows (Appendix) that there is some variation and a non-neglible amount of subjects are choosing zero effort. Yet, the distribution do appear to behave approximately normal.However, we are also interested in how the effort of the Second Movers responds to the different prizes. This is shown below. Canonically, the increase in incentives increases the willingness to exert effort. This would manifest itself in a strictly monotonous relationship for each individuals. Given prizes are randomly distributed, this should also hold globally. However, this cannot be observed here, suggesting that there are other effects than the monetary incentive at play.<jupyter_code>pyplt.plot(df_2_pref.groupby('prize')['e2'].mean())
pyplt.xlabel('Prize')
pyplt.ylabel('Average Second Mover Effort')
pyplt.show()
print('Figure 4: Average Second Mover Efforts Per Prize Level \n')
stat = df_2_pref['e2'].corr(df_2_pref['prize'])
print(f'The correlation between Prize and Second Mover effort is {stat:0.3f}')
joint_effort(df_2_pref)
print('Figure 5: Correlation between First and Second Mover Effort')<jupyter_output>Figure 5: Correlation between First and Second Mover Effort<jupyter_text>As mentioned above, the link the authors want to investigate is whether the effort $e_1$ of the First Mover negatively impacts the choice of effort $e_2$ made by the Second Mover. A naive strategy checks for the correlation between those two effort choices. Where disappointment aversion predicts a negative relationship, the canonical model suggest that there is no link between the effort choices. The correlation indicates that there is little to no correlation, and if at all, the effort choices are positively correlated. This positive relationship is suggested by peer effects (Falk and Ichino 2006), where subjects imitate the behavior of their peers and compete by matching or beating the observed effort levels. Naturally, this is an insufficient analysis without controlling for any of the other effects on Second Mover effort described in the identification strategy. Next, the authors use a reduced form analysis to examine the Second Mover's behavior.     Reduced Form Analysis  ModelEach subject plays ten rounds resulting in data with a panel structure. Here, proposition 1 predicts that when controlling for prize and round effects, the effort of the Second Mover will not depend on the First Mover's effort. In contrast, disappointment aversion predicts a negative relationship. The model to be estimated is:\begin{equation}e_{2,n,r} = \beta_1 + \beta_2v_{n,r} + \beta_3e_{1,n,r} + \beta_4e_{1,n,r}  *  v_{n,r} + d_r + \omega_n + \epsilon_{n,r} \end{equation}for   n = 1,...,N and r = 1,...10. Where n denotes the Second Mover and r indicates the paying round. The inclusion of the interaction term is informed by Proposition 3, where the negative effect of the First Mover's effort is larger at higher prizes. Round dummies $d_r$ are included with the first round being the omitted dummy. The round-invariant Second Mover effects are denoted by $\omega_n$ and are assumed to be identically and independently distributed over Second Movers with a variance of $\sigma_\omega^2$. $\epsilon_{n,r}$ is an observable that varies over Second Movers, it is also assumed to be identically and independently distributed over rounds and Second Movers with variance $\sigma_\epsilon^2$. Note: In the following cells the analysis with R is set up. R is used because using RandomEffects of the linearmodels package does not reproduce the results.<jupyter_code>%%R

# import data 
df <- read_dta("data/paper_data/master_data.dta")

# convert the data frame to a data format recognizable by 'plm' 
pdf <- subset(pdata.frame(df, index = c("Subject", "Period")))
pdf_pref <- subset(pdf, !Subject==302)
%%R

# Random Effects Regressions
re <- plm(Effort~e1+prize+E1timesPrize+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = pdf, model = "random")
re_pref <- plm(Effort~e1+prize+E1timesPrize+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = pdf_pref, model = "random")
# Fixed Effects Regressions
fe <- plm(Effort~e1+prize+E1timesPrize+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = pdf, model = "within")
fe_pref <- plm(Effort~e1+prize+E1timesPrize+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = pdf_pref, model = "within")

# Standard Errors of Omega and Epsilon
pref_1 <- round(sqrt(re_pref$ercomp$sigma2[2]),digits=3)
pref_2 <- round(sqrt(re_pref$ercomp$sigma2[1]),digits=3)
se_re_1 <- round(sqrt(re$ercomp$sigma2[2]),digits=3)
se_re_2 <- round(sqrt(re$ercomp$sigma2[1]),digits=3)<jupyter_output><empty_output><jupyter_text>Results     Replication of the parameters and standard errors in Table 2 of the paper. For this model, the theory suggests three hypotheses:* Second Mover effort does **not** depend on the corresponding First Mover effort, controlling for prize and round effects **(Proposition 1)**.* Second Mover effort, in fact, depends (negatively) on the respective First Mover effort, with controls for prize and round effects **(Proposition 2)**.* The negative effect of the First Mover's effort on the Second Mover's effort is larger at higher prices **(Proposition 3)**.<jupyter_code>%%R

stargazer(re_pref, re, type="text", title="Table 2: Random Effects Regressions",
          dep.var.labels="Second Mover Effort",
          #omit=c("TT2","TT3","TT4","TT5","TT6","TT7","TT8","TT9","TT10"), # unchecking this gives the coefficients reported in the paper.
          covariate.labels=c("First Mover Effort", "Prize", "Prize x First Mover Effort"),
          column.labels=c("Preferred Sample", "Full Sample"),
          add.lines = list(
            c("Standard Error of Omega", pref_1 , se_re_1),
            c("Standard Error of Epsilon",pref_2,se_re_2)),
          no.space=TRUE)<jupyter_output>Table 2: Random Effects Regressions
=======================================================
                               Dependent variable:     
                           ----------------------------
                               Second Mover Effort     
                           Preferred Sample Full Sample
                                 (1)            (2)    
-------------------------------------------------------
First Mover Effort              0.044          0.047   
                               (0.049)        (0.049)  
Prize                          1.639***      1.655***  
                               (0.602)        (0.592)  
Prize x First Mover Effort     -0.049**      -0.050**  
                               (0.023)        (0.023)  
TT2                            1.689**        1.655**  
                               (0.709)        (0.699)  
TT3                            3.084***      3.034***  
                               (0.713)        (0.702)  
TT4        [...]<jupyter_text>Proposition 3 can be accepted, the interaction between the First Mover's effort and the prize is significantly negative, suggesting that the effect of the First Mover's effort is larger at higher prices. Furthermore, the interaction term (at the average price) and the First Mover effort together are negative, albeit not significant. This supports Proposition 1 in favor of Proposition 2. Notable but only mentioned in a footnote are the large and highly significant round effects that tend to increase over the rounds. As indicated in Section 4, this can be due to learning.  The following regression shows the fixed effects regression that is mentioned in the text but not explicitly shown.<jupyter_code>%%R

stargazer(fe_pref,fe, type="text", title=" Table 3: Fixed Effects Regressions",
          dep.var.labels="Second Mover Effort",
          omit=c("TT2","TT3","TT4","TT5","TT6","TT7","TT8","TT9","TT10"),
          covariate.labels=c("First Mover Effort", "Prize", "Prize x First Mover effort","Intercept"),
          column.labels=c("Preferred Sample", "Full Sample"),
          no.space=TRUE)<jupyter_output>Table 3: Fixed Effects Regressions
==========================================================================
                                         Dependent variable:              
                           -----------------------------------------------
                                         Second Mover Effort              
                              Preferred Sample           Full Sample      
                                     (1)                     (2)          
--------------------------------------------------------------------------
First Mover Effort                  0.047                   0.050         
                                   (0.050)                 (0.049)        
Prize                             1.716***                1.701***        
                                   (0.605)                 (0.594)        
Prize x First Mover effort        -0.051**                -0.051**        
                                   (0.024)                 (0.02[...]<jupyter_text>Fixed vs. Random EffectsThe authors opt to use a random effects linear regression. Random effect models are closely related to the fixed effects models. Here, the different assumptions made by the two approaches are discussed. This is followed by the replication of the Hausman test that can be found in Table 2 of the paper. Both approaches are workhorses when working with panel/cross sectional time series data. The key distinction between the two models are the different assumptions made about the omitted variables. Implicitly, the random effects model assumes that there are no omitted variables correlated with the explanatory variables. The entities are drawn randomly from a population. If this is accurate, the model yields unbiased results and produces the smallest standard errors. In comparison, the fixed effects model makes no such strong assumptions about the correlation of omitted variables with the explanatory variables. In fact, the approach can control for omitted variables bias. The subjects serve as their own controls. For this to hold true, the fixed effects has to be time-invariant. However, being able to control for omitted variable bias comes at the cost of producing higher standard errors. The **Hausman test** provides a means for comparing these two models. It tests for the independence of $\omega_n$ and the explanatory variables $x_{it}$. It looks as follows:\begin{equation}W = \frac{(\tilde\beta_{RE} - \tilde\beta_{FE})^2}{Var(\tilde\beta_{RE}) - Var(\tilde\beta_{FE})} \sim \chi_1^2\end{equation}The null hypothesis $H_0$ is that there is no covariance between the explanatory variables and the specific effects. Making the random effects model a suitable candidate. If this hypothesis is rejected, the $H_1$ leaves us with the fixed effects model coefficients as the only consistent coefficients.         Replication of the Hausman test in  Table 2 of the paper.<jupyter_code>%%R

# Hausman Test testing the Random Effects assumption
hausman <- phtest(fe,re)
hausman_pref <- phtest(fe_pref,re_pref)

hausman_pref<jupyter_output>Hausman Test

data:  Effort ~ e1 + prize + E1timesPrize + TT2 + TT3 + TT4 + TT5 +  ...
chisq = 2.6008, df = 12, p-value = 0.9978
alternative hypothesis: one model is inconsistent<jupyter_text>The Hausman Test cannot reject $H_0$ and, thus, the use of random effects is supported. This holds true for the preferred sample as well as the full sample.  Further InvestigationOne advantage of laboratory experiments is that environmental variables can be easily altered in a controlled fashion. However, this often does leave only few covariates to further investigate. Especially with the data provided by the authors, there are only limited options on how to expand their reduced form findings. In an earlier working paper the authors include men and women dummies, unfortunately they were not available in the provided dataset. Extension: The following regressions toggle around with different interaction terms.<jupyter_code>%%R

# Split interaction term on the first and the last rounds
new_split <- pdf_pref
new_split$Period <- as.numeric(new_split$Period)
split <- pdf_pref
split$Period <- as.numeric(split$Period)

# Reported split
split$last <- ifelse(split$Period <= 5, 0, split$E1timesPrize)
split$first <- ifelse(split$Period > 5, 0, split$E1timesPrize)

# New Split
new_split$last <- ifelse(new_split$Period <= 4, 0, new_split$E1timesPrize)
new_split$first <- ifelse(new_split$Period > 4, 0, new_split$E1timesPrize)

# Random Effects regression
split_interaction <- plm(Effort~e1+prize+last+first+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = split, model = "random")
new_interaction_split <- plm(Effort~e1+prize+last+first+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = new_split, model = "random")

stargazer(split_interaction,new_interaction_split, type="text", title="Table 4: Interaction Term Round Splits",
          dep.var.labels="Second Mover Effort",
          omit=c("TT2","TT3","TT4","TT5","TT6","TT7","TT8","TT9","TT10"),
          covariate.labels=c("First Mover Effort", "Prize", "Last","First","Intercept"),
          column.labels=c("Split at round 5", "Split at round 4"),
          no.space=TRUE)<jupyter_output>Table 4: Interaction Term Round Splits
====================================================
                          Dependent variable:       
                   ---------------------------------
                          Second Mover Effort       
                   Split at round 5 Split at round 4
                         (1)              (2)       
----------------------------------------------------
First Mover Effort      0.044            0.043      
                       (0.049)          (0.050)     
Prize                  1.640***         1.619***    
                       (0.604)          (0.605)     
Last                   -0.047**         -0.050**    
                       (0.024)          (0.024)     
First                  -0.050**         -0.046*     
                       (0.024)          (0.025)     
Intercept             19.843***        19.687***    
                       (1.415)          (1.416)     
----------------------------------------------------
Observ[...]<jupyter_text>The specification where the interaction term is split at round 5 is mentioned in the paper. Using a different break point, where the interaction term the first four rounds is denoted by first and the last six rounds are denoted by last. This follows from the visual inspection of the effort distribution per round that can be found in the Appendix. The new split is very similar to the older specification. Nonetheless, the interaction for the first rounds in the new specification is not significant at the 5% level anymore. Yet, further inspecting the coefficients does not lead to any new conclusions. The last rounds have a larger negative effect than the first round but the differences are minor and the test below affirms that the difference is not significant.<jupyter_code>%%R

# Test equality of the interaction term for the first and the last five rounds.
linearHypothesis(split_interaction, c("first=last"))

# Test for equality of the split interaction term with a break at round 4
linearHypothesis(new_interaction_split, c("first=last"))
%%R

# New dataset.
new_interact <- pdf_pref

# Interaction term partioned for prize level
new_interact$small <- ifelse(new_interact$prize < 2, new_interact$E1timesPrize, 0)
new_interact$medium <- ifelse(new_interact$prize >= 2 & new_interact$prize <= 2.60, new_interact$E1timesPrize, 0)
new_interact$high <- ifelse(new_interact$prize >= 2.70, new_interact$E1timesPrize, 0)

# interaction term for different prize level
new_spec <- plm(Effort~e1+prize+small+medium+high+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = new_interact, model = "random")

# No prize controls
no_prizes <- plm(Effort~e1+TT2+TT3+TT4+TT5+TT6+TT7+TT8+TT9+TT10, data = pdf_pref, model = "random")

# Render the table
stargazer(no_prizes,new_spec, type="text", title="Table 5: New Interaction Term Specifications",
          dep.var.labels="Second Mover Effort",
          omit=c("TT2","TT3","TT4","TT5","TT6","TT7","TT8","TT9","TT10"), # unchecking this gives the coefficients reported in the paper.
          covariate.labels=c("First Mover Effort", "Prize", "Small","Medium","High"),
          column.labels=c("No Prize Controls", "Interaction per Prize Level"),
          no.space=TRUE)<jupyter_output>Table 5: New Interaction Term Specifications
================================================================
                                Dependent variable:             
                   ---------------------------------------------
                                Second Mover Effort             
                   No Prize Controls Interaction per Prize Level
                          (1)                    (2)            
----------------------------------------------------------------
First Mover Effort      -0.036                  0.024           
                        (0.026)                (0.052)          
Prize                                         1.608***          
                                               (0.601)          
Small                                          -0.032           
                                               (0.029)          
Medium                                         -0.031           
                                            [...]<jupyter_text>In the regression table above, two specifications can be seen. The first specification, also reported in the paper, eliminates all prize control and can report a negative but insignificant effect of $e_1$ on $e_2$. The other specification looks as follows:\begin{equation}e_{2,n,r} = \beta_1 + \beta_2v_{n,r} + \beta_3e_{1,n,r} + \beta_4Small_{n,r} +\beta_5Medium_{n,r} + \beta_6High_{n,r} + d_r + \omega_n + \epsilon_{n,r} \end{equation}where* $Small_{n,r} : e_{1,n,r}*v_{n,r}$ for $v_{n,r} < 2$* $Medium_{n,r} : e_{1,n,r}*v_{n,r}$ for $2 \leq v_{n,r} \leq 2.60$* $Large_{n,r} : e_{1,n,r}*v_{n,r}$ for $v_{n,r} > 2.60$denote the interaction term for different prize ranges. This can inform whether this relationship is stable throughout the whole prize range. As can be seen, the effect decreases with the prize value. Whereas the interaction on Small and Medium prize levels is not significant at any level, the interaction with the high prizes is borderline significant (10%).   Structural Model  Empirical SpecificationAfter not accumulating convincing evidence of disappointment aversion in the reduced form analysis the authors turn to structural estimation strategies. Thanks to the variations induced by the experimental design, the data is well-suited for such an analysis. Beyond, the estimation of disappointment aversion, this approach also allows for the estimation of population-level heterogeneity.In order to estimate disappointment aversion the authors use the following specifications:\begin{equation}\lambda_{2,n} \sim N(\tilde\lambda_2,\sigma_\lambda^2)\end{equation}for n = 1,...,N.Here, $\lambda$ is held fixed on the subject-level. It can vary throughout Second Movers but does not vary over rounds. In addition to $\lambda$ the convexity parameter $c_{n,r}$ is estimated at round and subject-level:\begin{equation}c_{n,r} = \kappa + \delta_r + \mu_n + \pi_{n,r}\end{equation}for n = 1,...,N; r = 1,...10.$\kappa$ represents a component common across all Second Movers. $\delta_r$ denotes the round effects that allow the marginal costs of effort to vary over rounds (where $\delta_1$ is omitted). For instance, if due to learning the tasks gets easier in the later periods $\delta_r$ is negative and decreasing over rounds. $\mu_n$ indicates the unobserved differences in the cost of effort on the subject-level that are constant over rounds. $\pi_{n,r}$ denotes the unobserved differences that vary over rounds and Second Movers. For $\pi$ and $\mu$ both scale and shape parameters of a Weibull distribution are estimated. The reasoning for opting for a Weibull distribution is that it has positive support allowing to impose convex cost of effort function on all subjects. In total, this leaves them with the following 17 unknown parameter:     Extension: Description and trial values for the unknown parameters.<jupyter_code>print('Table 6: Parameter description')
params_description(return_descriptives=True)<jupyter_output>Table 6: Parameter description<jupyter_text>Regarding the selection of the bounds, the authors do not provide any decision rule, however they do run the estimation with different specifications. From their Matlab codes the upper and lower bounds of the values of their preffered specification can be extracted. The bounds are configured for estimation with *estimagic*. Setting bounds is a decisive factor in the success of optimization. Structural models usually provide sufficient information in order to select sensible bounds. The starting trial values are the mean between the bounds with the round effects, set at zero, being the exception. The cost parameter $b$ reported in the table below sticks out but since it is part of the cost of effort function: \begin{equation}C_2(e_{2,n,r}) = be_{2,n,r} + \frac{c_{n,r}e_{2,n,r}^{2}}{2}\end{equation}where the mean effort choices are between 20 and 30 correctly positioned sliders, this seems reasonable.   Method of Simulated Moments The parameter of interest are estimated by the **Method of Simulated Moments (MSM)** that was pioneered by McFadden (1989) and by <NAME> (1989). The idea is, similar to the Generalized Method of Moments (GMM), to use a vector of moments M with $k \geq 17$, where every included moment is dependent on the endogenous variables. The step that differentiates MSM from GMM is the use of simulated samples. In each of the S simulated sample the Second Mover face the same prizes and First Mover effort as observed in the actual sample. The behavior of the Second Movers in the simulated sample is guided by the trial  $\theta_t$ values of the unknown parameters $\theta$. The unobservables $\mu$ and $\pi$ are assigned to the Second Movers with the specified distributions. For all Second Movers and each round the utility for all 49 possible effort choices is computed and the simulated action corresponds to the effort choice with the highest achieved utility. In order to compare the simulated samples to the experimental sample the vector of moments $M_s(\theta_t)$ are computed. These moments are used in the following metric:\begin{equation*}J(\theta_t) =  \left(M - \frac{1}{S} \sum\limits_{S=1}^S M_S(\theta_t)\right)'W_N\left(M - \frac{1}{S} \sum\limits_{S=1}^S M_S(\theta_t)\right) \end{equation*}This is the core function of the MSM approach. The objective of this metric is $J(\theta_t): \hat\theta = arg min_{\theta_t} J(\theta_t)$. It minimizes the distance between the average of the simulated moments and the empirical moments **M**. Thus, the moments induced by the structural parameters have to be as close to the actual moments as possible. $W_N$ is the weighting matrix of $k \times k$ dimension. For this simulation, the authors specify the diagonal elements of the weigthing matrix as the inverse of N times the bootstrapped variance of the moments and zeroes on the non-diagonal elements. In total, 38 moments are used to estimate the 17 unknown parameters.        Note: The following attempt of implementation follows the steps outlined by the respy documentation closely.    Empirical MomentsIn order to estimate the parameters that are of interest, moments need to be simulated. A dataframe fashioned after Table 4 is provided below. It gives an overview over the moments the authors choose to be estimated.      Replication of the descriptions in Table 4.<jupyter_code>print('Table 7: Description of the moments')
moments_description()<jupyter_output>Table 7: Description of the moments<jupyter_text>The functions needed for the estimation of those moments are found in the *auxiliary/msm.py*-file. They are stored in the dictionary to access them later. For all those moments separate functions are used, eventhough some functions have similarities with each other. This was done to follow the guides outlined on the *respy* website as closely as possible. Yet, one difference is that in this situation the moments are computed for the whole sample, whereas the *respy* outline computes them at a period level.<jupyter_code>calc_moments = {
    'effort_std': effort_std,
    'corr_round_lag1': pooled_autocorr_lag1,
    'corr_round_lag2': pooled_autocorr_lag2,
    'round_change_std': round_change_std,
    'mean_period_1': period1_average,
    'mean_period_2': period2_average,
    'mean_period_3': period3_average,
    'mean_period_4': period4_average,
    'mean_period_5': period5_average,
    'mean_period_6': period6_average,
    'mean_period_7': period7_average,
    'mean_period_8': period8_average,
    'mean_period_9': period9_average,
    'mean_period_10': period10_average,
    'cond_corr_e2_prize': cond_corr_e2_prize,
    'cond_corr_e2_e1': cond_corr_e2_e1,
    'cond_corr_e2_e1timesprize': cond_corr_e2_e1timesprize,
    'perc17_cond_corr_e2_prize': perc17_cond_corr_e2_prize,
    'perc33_cond_corr_e2_prize': perc33_cond_corr_e2_prize,
    'perc50_cond_corr_e2_prize': perc50_cond_corr_e2_prize,
    'perc66_cond_corr_e2_prize': perc66_cond_corr_e2_prize,
    'perc83_cond_corr_e2_prize': perc83_cond_corr_e2_prize,
    'perc17_cond_corr_e2_e1': perc17_cond_corr_e2_e1,
    'perc33_cond_corr_e2_e1': perc33_cond_corr_e2_e1,
    'perc50_cond_corr_e2_e1': perc50_cond_corr_e2_e1,
    'perc66_cond_corr_e2_e1': perc66_cond_corr_e2_e1,
    'perc83_cond_corr_e2_e1': perc83_cond_corr_e2_e1,
    'perc17_cond_corr_e2_e1timesprize': perc17_cond_corr_e2_e1timesprize,
    'perc33_cond_corr_e2_e1timesprize': perc33_cond_corr_e2_e1timesprize,
    'perc50_cond_corr_e2_e1timesprize': perc50_cond_corr_e2_e1timesprize,
    'perc66_cond_corr_e2_e1timesprize': perc66_cond_corr_e2_e1timesprize,
    'perc83_cond_corr_e2_e1timesprize': perc83_cond_corr_e2_e1timesprize,
    'low_effort_low_prize': low_effort_low_prize,
    'low_effort_high_prize': low_effort_high_prize,
    'high_effort_low_prize': high_effort_low_prize,
    'high_effort_high_prize': high_effort_high_prize,
    'low_effort_prop': low_effort_prop,
    'high_effort_prop': high_effort_prop
}<jupyter_output><empty_output><jupyter_text>These function, in turn, are used to calculate the empirical moments. These are the moments that are computed with the experimental sample without any simulation. They are found below.       Replication of the observed moments in Table 5.<jupyter_code>empirical_moments = observed_moments(df_2_pref)
empirical_moments<jupyter_output><empty_output><jupyter_text>Note: Some moments were not precisely specified in Table 4. The different computation specifications for these moments can be found in the Appendix.  Weighting MatrixThe weighting matrix is important to account for differences in magnitude between the moments. It assigns each moment a different weight and hence controls the importance the optimizing function gives to each moment. In the paper it is specified that the diagonal elements of the weighting matrix are the inverse of N times the variances of the sample moments and zeros elsewhere. 2000 Bootstrap samples with replacement are used to compute $W_n$. For my weighting matrix, I am orienting myself at the example [here](https://respy.readthedocs.io/en/latest/how_to_guides/msm_estimation_exercise.html), where the diagonal elements are the inverse variance of the bootstrapped moments. The function needed to be adapted to suit the objects in this notebook. For example, bootstrap sampling is used with replacement because the observed sample is small. Given cell timeout constraints with Travis CI, only 20 bootstrap samples are used to compute the weighting matrix. I'd suggest running the function with atleast 100 bootstraped samples, if possible.     Note: The following functions are still at a premature stage.<jupyter_code>start = datetime.now()
weighting_matrix = get_weighting_matrix(
   data=df_2_pref,
   empirical_moments=empirical_moments,
   calc_moments=calc_moments,
   n_bootstrap_samples=20,
   n_observations_per_sample=59,
   replace_missing_variances=None)
end = datetime.now()
time_taken = end - start
print(time_taken)
weighting_matrix<jupyter_output><empty_output><jupyter_text>SimulationAs I mentioned above, I was not able to replicate the Method of Simulated Moments. Yet, I was able to get the simulation running. In the simulation below, first the effort choices are simulated induced by the prespecified trial values. For each subjects and round every possible effort choices is simulated and the utility maximizing choice is drawn. Then, using these choices the moments are computed. Finally, they are added together and divided by the number of simulations.<jupyter_code>params = params_description()
simulated_moments = get_simulated_moments(df=df_2_pref, params=params, calc_moments=calc_moments, empirical_moments=empirical_moments, num_simulation=2, seed=123)
simulated_moments.round(3)<jupyter_output>/Users/vincentselz/opt/anaconda3/lib/python3.7/site-packages/pandas/core/indexing.py:205: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_with_indexer(indexer, value)<jupyter_text>Thankfully, the moments provide a good overview of what went wrong in the simulation. It appears in general the level of efforts are too low in the simulated samples. As can be seen in the last row not even one simulated effort was above 35 correctly placed sliders. In addition to that the variance is lower than in the original sample. To optimize these values is precisely the task of the MSM. It chooses such parameter values that the simulated moments fit the empirical moments as closely as possible. The actual simulation runs 30 times, thus simulating 17,700 pairings. Here, the simulation is run only 2 times because of the cell timeout. I'd suggest using atleast 10 simulations.  Further StepsThe framework from above now can be used to construct the criterion function. Here, my contribution ends. In order to elicit the optimal parameter values the above mentioned criterion function $J(\theta_t)$ is used. The simulation depends on three different random draws, which are held constant throughout the estimation. This ensures that as the parameter values $\theta_t$ are adjusted, the simulated samples only vary due to changes in $\theta_t$ and not due to the exogenously induced random draws. Due to the discrete nature of the optimization, the authors opt to use Simulated Annealing (1994) instead of gradient or Hessian-based optimization methods to solve for the MSM estimates.      --- Critical Assessment---The saying is that when having a well designed experiment, the following empirical analysis is straight forward. This study turns this saying around and shows that structural models can help overcome difficulties in identifying causal effects. In this experiment, results of the reduced form evidence provided little to no evidence for disappointment aversion. The interaction term between First Mover effort and prize was significant but the main variable of interest, the sole First Mover effort, was not. Here, the use of structural estimation salvaged the situation.   In general, I argue that the experiment was not well suited for the identification. The highly significant dummy variables for round effects in the reduced form analysis, suggest that the design did pick up various confounds that were not intended. The significant negative $\delta_r$ in the structural estimation corroborates that learning, in fact, did take place through the paying rounds. The experimenters set up two training rounds, yet it appears this was not enough. Expanding the training rounds might help filter these confounds out of the experimental sample. The empirical analysis by the authors is sound and well justified. Yet, I do not agree with their assessment that the reduced form analysis offers evidence consistent with disappointment aversion. For this to be the case the impact of $e_1$ on $e_2$ has to be significantly lower than zero which is not true. In their structural model they test various specifications that would favor explanations other than disappointment aversion for the observed behavior. Unfortunately, I was not able to reproduce the MSM, else I could have commented more extensively on the intrinsics of those specifications.From a experimenter's perspective the study supplies a novel approach to real effort tasks. The slider task provides a convincing method to elicit finely grained discrete effort choices. Yet, for a sample of N subjects 2N need to be paid, since the First Movers also need to be compensated - this appears to be cost-inefficient. Potentially, similar to the training round, the First Mover effort can be simulated making the double payments redundant. Of course, it remains to be seen whether the knowledge of having a tournament with a real person has an effect on the Second Mover. The primary contribution by this paper is twofold. First, it anwers the question what an agent's reference point constitutes. In line with Abeler (2011), their evidence suggests that the reference points is given by the expected reference point as the model of reference-dependent preferences predicts. Furthermore, the structural analysis allows them to estimate the strength of loss-aversion around the reference point and provides estimates about the heterogeneity of disappointment aversion in the underlying population. Since their model does not fix the reference point at the point of optimization, as the parsimonious model of reference-dependent preferences does, this makes the observed behavior applicable to the situation where the agents can affect their probability of success e.g. competing about promotions. Second, the paper contributes to the discussion about how quickly reference points form. Given that the Second Mover almost immediately observes the First Mover's effort choice and optimizes her behavior, it indicates that reference points form instantly.   --- Appendix---  Effort Distribution Per Round<jupyter_code>effort_distr_per_round(df_2_pref)
print('Figure 6: Effort Distribution Per Round')<jupyter_output><empty_output><jupyter_text>Prize DistributionThe authors state that the prizes for the tournament are drawn from a uniform distribution. An inspection shows the randomization is successful. The visualization but more importantly the $\chi^2$-Test confirms that it is not significantly different from a uniform distribution.<jupyter_code>prize_distribution(df_2_pref)
print('Figure 7: Prize Distribution')
chisquare(df_2_pref['prize'])<jupyter_output><empty_output><jupyter_text>Moment Computation SpecificationsAs mentioned above, the description in Table 4 sometimes do not match the used computation methods or the description is not specific enough to replicate the solution. Here, I set out how the original results can still be obtained.  First, the conditional means in the table are misspecified. Whereas Table 4 states that the specified effort and prizes need to be strictly higher or lower, their Matlab function uses non-strict inequality. The respective functions can be found in the subdirectory MatLabCode under moments.m line 168 and following. They use the *sign* function to indicate when prices are higher or efforts are higher thant the respective threshold, then subsequently delete them. Yet, this still leaves the Second Mover efforts that are equal to the threshold in the sample.Second, for the autocorrelation the authors use the pooled autocorrelation. First, all differences from round r-1 to round r are computed and then the correlation is calculated. The implicit assumption here is that that the variances on the subject-level are equal. In turn, the pooled sample variance decreases the denominator, leading to higher autocorrelation estimates. The alternative is to compute the autocorrelations separately for each subject and to use the mean of these correlations:<jupyter_code>stat1 = pooled_autocorr_lag1(df_2_pref)
stat2 = calc_autocorrelation_lag1(df_2_pref)

print('The different autocorrelation specifications: \n')
print(f'Reported Autocorrelation: {stat1:0.3f} Mean Autocorrelation: {stat2:0.3f}')<jupyter_output>The different autocorrelation specifications: 

Reported Autocorrelation: 0.652 Mean Autocorrelation: 0.123
{'fraction_non_alphanumeric': 0.07601139807975026, 'fraction_numerical': 0.020835980180409098, 'mean_word_length': 4.033159769799945, 'pattern_counts': {'":': 0, '<': 106, '<?xml version=': 0, '>': 79, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 27, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '293188', 'license': 'MIT', 'n_tokens_mistral': 14914, 'n_tokens_neox': 13788, 'n_words': 6728, 'path': 'Project-VincentSelz.ipynb', 'repo_name': 'VincentSelz/microeconometrics-course-project-VincentSelz', 'size': '55136'}