<jupyter_start><jupyter_text>--- Replication Project for Microeconometrics course | Summer 2021, M.Sc. Economics, Bonn University | [<NAME>](https://github.com/aysuavci) Replication of Zimmermann (2020)    --- This notebook contains the replication of the following study:> [<NAME>. (2020). The Dynamics of Motivated Beliefs. American Economic Review, 110(2), 337–363](https://www.aeaweb.org/articles?id=10.1257/aer.20180728). Notes:* I try to remain true to the original naming of the variables, however, I renamed all of the figures and tables according to sections in this notebook to avoid any confusions that might arise due to having two replications.* For convenience, I decided to keep some of the coding in the notebook which I have seen essential for understanding the steps taken in replication. * The responsitory can be downloaded from [here](https://github.com/OpenSourceEconomics/ose-data-science-course-project-aysuavci), for the best viewing.  **Table of Contents*** [Introduction](sec1)* [Background for the Experiment and Hypothesis](sec2)* [Experiment Design](sec3)* [Identification](sec4)* [Empirical Strategy](sec5)* [Replication of Zimmermann (2020)](sec6)    * [Main Results](sec6_1)        * [Data & Descriptive Statistics](sec6_1_1)        * [Belief Adjustment Distribution¶](sec6_1_2)        * [Belief Adjustment Distribution & Test performances](sec6_1_3)        * [Regression Outputs](sec6_1_4)    * [Robustness Checks](sec6_2)        * [Appendix A.4 - Alternative Definition of Positive/Negative Feedback](sec6_2_1)        * [Appendix A.6 - Updating in the Short-Run](sec6_2_2)        * [Appendix A.7 - No Feedback Condition](sec6_2_3)        * [Appendix A.8 - Figures Bayesian Posteriors](sec6_2_4)* [Extension](sec7)    * [Replication of Eil and Rao (2011)](sec7_1)        * [Introduction](sec7_1_1)        * [Experiment Design](sec7_1_2)        * [Emprical Strategy](sec7_1_3)        * [Replication](sec7_1_4)        * [Comparison with Zimmermann (2020)](sec7_1_5)    * [Further Extension](sec7_2)        * [Alternative Data Visualization](sec7_2_1)        * [Further Analysis with No Feedback Group](sec7_2_2)            * [Emprical Strategy](sec7_2_2_1)            * [Estimation](sec7_2_2_2)              * [Conclusion](sec8)* [References](sec9)<jupyter_code>%matplotlib inline
!pip install stargazer
!pip install matplotlib
!pip install -U matplotlib
import numpy as np
import pandas as pd
import pandas.io.formats.style
import seaborn as sns
import statsmodels as sm
import statsmodels.formula.api as smf
import statsmodels.api as sm_api
import matplotlib as plt
from IPython.display import HTML
from stargazer.stargazer import Stargazer, LineLocation
from statsmodels.iolib.summary2 import summary_col
from auxiliary.auxiliary_tools import *
from auxiliary.auxiliary_plots import *
from auxiliary.auxiliary_tables import *<jupyter_output><empty_output><jupyter_text>--- 1.Introduction ---The study of Zimmermann(2020) is a lab experiment aiming to examine how motivated beliefs held by individuals continues to be sustained after receiving positive or negative feedback.The study can be divided into 3 parts:1. The first part of the study-named “**Motivated Belief Dynamics**” in the paper- is the main study for investigating the causality between different types of feedback (positive or negative) received and reconstruction of belief patterns depending on the elicitation time (directly or 1 month) of the belief after the experiment. 2. The second part of the study-named “**The Role of Memory**” in the paper- investigates the asymmetry in the accuracy of feedback recall and also recall of IQ test in general that they solve as a part of the experiment. 3. The final part of the study-named “**The Trade-Off between Motivated and Accurate Belief**” in the paper- that questions whether incentivizing for recall accuracy could mitigate the motivated reasoning that participants employ.In this project notebook, the main study-first part- will be replicated; the part where Zimmermann (2020) used also **difference-in-difference (DID) models** for their estimations. In the next section (Section 2), I will give a quick literature background for the experiment and also specify the researcher’s hypothesis. In Section 3, I will explain the relevant parts of the experimental design and introduce treatment groups and variables for convenience. The identification part (Section 4) will explain the causality studied by Zimmermann (2020) and the empirical strategy part (Section 5) will present the established models for estimations. Section 6 is the replication part where the figures and tables from the main study and relevant Appendix parts are presented. The extension of this project is located in Section 7 where I replicate the study of Eil and Rao (2022)  as well as including some data visualization and further analysis. Finally, Section 8 includes my concluding remarks. --- 2. Background for the Experiment and Hypothesis ---Personally, I have always been interested in the question of how individual beliefs are formed and sustained or changed over time. In the neoclassical economics framework, any changes in beliefs follow Bayes Rule’s since it is accepted that all agents are rational and unbiased. However, many studies in the literature of motivated beliefs showed that individuals deviate from these Bayesian predictions and manipulate their beliefs in a self-serving way; and some studies like Eil and Rao 2011 or Zimmermann 2020 tried to address the asymmetry in belief updating as well. I see the study of Zimmermann (2020) as an influential study to study motivated belief dynamics because of the experiment design that is been employed. The design of Zimmermann (2020) allows comparing the belief updating across signals received (asymmetry) and also across time (short-run and long-run updating) by employing control and various treatment groups. **Main Hypothesis:** *Individuals’ engagement with belief updating follows an asymmetric pattern, putting more weight on positive signals and less on the negative signals, which is more pronounced for long-term since negative signal’s effects fade over time.*Using DID models for analysis is suitable for this study because, first of all, it is impossible to make one experiment subject expose to two different treatments while keeping everything constant in this experiment to examine individual-level effects. Secondly, since this is a randomly assigned experiment we can assume subjects do not differ between the control group and treatment groups. Also, between the control group and treatment group Zimmermann (2020) made sure to keep everything constant and only made the feedback condition vary. Also, DID was easily implemented as an interaction term between time treatment and feedback dummy allowing for causal inferences on belief dynamics in line with the researcher’s goal and with the group of interest which is the individuals who received a negative signal and in the treatment group 1 month and their differences compared to other groups in belief updating. In addition to these, employing DID models allowed Zimmermann (2020) to add the fixed rank effects interacted with treatment which serve as a control for the possible characteristic differences due to rank groups between treatments. Also, having a control condition for 1-month treatment which serve as a robustness check for potential beliefs changes due to time trends that may exist in the absence of feedback. --- 3. Experimental Design ---1. Subjects completed an IQ test where they need to solve 10 Raven matrices.2. Subjects were randomly assigned to groups of 10.3. Subjects were asked to estimate their belief about the likelihood of being ranked in the upper half of their group in terms of a percentage point. Full probability distributions for every possible rank is also elicited.4. Subjects were given noisy but true information about their rank by randomly selecting 3 participants from their group and telling them whether they were rank higher or lower than these 3 participants. If a subject is told that they ranked higher than 2 or 3 participants in their group, the subject is considered to be in the positive feedback group. They considered being in the negative feedback group if otherwise.5. After receiving the feedbacks, subjects are divided into two treatment groups randomly: `condifence_direct` and `confidence_1monthlater`.6. In the `confidence_1monthlater` treatment, subjects are asked to elicit their beliefs 1 month later; and in the `condifence_direct` treatment, group subjects are asked to elicit their beliefs after the feedback. `condifence_direct`has also two subgroups: `condifence_direct_immediate` and `condifence_direct_15minutelater`.**Main Variables**|Treatment groups|Feedback types|Outcomes|Other variables||---|---|---|---||`confidence_1monthlater`|Positive|Belief adjustment|Bayesian belief adjustment|||`condifence_direct_immediate`|Negative||rank||`condifence_direct_15minutelater`||||`no_feedback`||| --- 4. Identification ---  The first part of the experiment is dedicated to investigating the causality between the effect of feedback and belief adjustments of participants that are in two different treatment groups that vary depending on time: _CondifenceDirect_ and _Condifence1month_. The causality is established by two key components in the experiment. First of all, Zimmermann(2020) elicit peoples prior beliefs about their probability of ranking in the upper half of the group and then they elicited again after participants received the feedback. So, for each participant, they had a clear measure of belief updating which is possibly caused by the feedback and in line with the type of feedback. The second and the most important component of the experiment, that ensured causal identification, is the noisy feedback component. In this experiment rather than randomly assigning a subject into a negative or positive feedback group directly, Zimmermann(2020) randomly choose 3 other participants within the subject’s group to compare their rank and provide them with 3 comparisons. In this way, subjects indeed received true feedback but a noisy one. So, it is possible for two subjects with the same rank to receive different types of feedback. By adopting such an experimental design, Zimmermann(2020) ensured that potential asymmetries in the belief dynamics cannot be explained by the individual difference between participants. In addition to investigating the causal relationship between belief dynamics and feedback, another goal of the experiment is to observe the effect of time on this relationship. For that, Zimmermann(2020) assigned participants randomly to two groups: Confidence1month and CondifenceDirect. ConfidenceDirect is also divided into two subgroups(immediate and 15 minutes later belief elicitation) to investigate the possible short-term dynamics in belief adjustment, however, that part of the study will not be covered in this replication. --- 5. Empirical Strategy --- In the study, the belief adjustment is defined as the difference between the elicited belief after the feedback and the belief before the feedback, formally\begin{equation}Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i}\end{equation}where $Pr(upperhalf)_{i}$ represents the subject ${i}$'s given probability of being in the upperhalf of the group.To compare belief adjustments between positive and negative feedbacks, a normalized version of the belief adjustment norm is also formed as below:$$beliefadjustmentnorm_i = \left\{    \begin{array}\\        Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i} & \mbox{if } \ feedback \ positive \\        (-1) x (Pr(upperhalf)^{post}_{i} - Pr(upperhalf)^{prior}_{i}) & \mbox{if } \ feedback \ negative \\    \end{array}\right.$$The difference-in-difference models, formally in the form of \begin{equation}beliefadjustmentnorm_i = \alpha + \beta feedback_i + \gamma T_i + \delta I_i + X_i \gamma + \epsilon_i\end{equation}are estimated for the purpose of establishing dynamic belief patterns. $feedback_i$ is the dummy variable for feedback and $T_i$ is the type of treatment. $I_i$ is for the interation term that is a generated dummy for the interest group, interest group being subjects who are in the `confidence_1monthlater`and received a negative feedback. In other words, if subjects were in `confidence_1monthlater` treatment and received a negative information; treatment dummy, feedback dummy and interaction term are all equal to 1. In this case, parameter $\delta$ is therefore captures the belief dynamics. In other words, if the subject is in `confidence_1monthlater` treatment and received a negative information; treatment dummy, feedback dummy and interaction term are equal to 1.In order to form a causal inference, Zimmermann (2020) control for rank and IQ test scores; these variables are captured by $X_i$, set of control variables. As another control variable, Bayesian belief adjustments are also used for some specifications. Zimmermann (2020) calculated Bayesian predictions, using individuals full prior probability distribution and the feedback type that they received. The Bayesian belief adjustment is then, produced in the same way as the belief adjustment\begin{equation}Pr(upperhalf)^{postBayes}_{i} - Pr(upperhalf)^{prior}_{i}\end{equation}where $Pr(upperhalf)^{postBayes}_{i}$ is the calculated Bayesian prediction for the subject $i$.Another specification within the study is adding the rank fixed effects interacted with treatment. In this way, Zimmermann (2020) was able to control the differentiation of possible different characteristics due to ranking between treatment groups.Zimmermann (2020) choose to perform ordinary least square (OLS) estimates with type HC1 heteroscedasticity robust covariance. Most of the main results are produced with this linear regression and I will use this type of regression in the table replications. --- 6. Replication of Zimmermann (2020) --- 6.1. Main Results  6.1.1 Data & Descriptive Statistics<jupyter_code>df = pd.read_stata('data/data.dta')
#Adjustments for better viewing
pd.set_option("display.max.columns", None)
pd.set_option("display.precision", 7)
pd.set_option("display.max.rows", 30)
#Viewing data
df.head()
#Renaming treatgroup for easier coding
tg_conditions = [
    (df['treatment'] == 'belief_announcement'),
    (df['treatment'] == 'confidence_1monthlater'),
    (df['treatment'] == 'confidence_direct_15minuteslater'),
    (df['treatment'] == 'confidence_direct_immediate'),
    (df['treatment'] == 'memory'),
    (df['treatment'] == 'memory_high'),
    (df['treatment'] == 'nofeedback'),
    (df['treatment'] == 'tournament_announcement'),
]
tg_values = [1, 2, 3, 4, 5, 6, 7, 8]
df['treatgroup'] = np.select(tg_conditions, tg_values)<jupyter_output><empty_output><jupyter_text>Throughout this study feedback groups will be addressed using the dummy variable `dummynews_goodbad`. As it is mentioned above, if a subject received more than 2 positive comparisons they are considered to be in the positive feedback group; if else they are considered to be in the negative feedback group.<jupyter_code>#Creating dummy variable for good/bad news (if good = 0, if bad = 1)
df["dummynews_goodbad"] = np.nan
df.loc[(df['pos_comparisons'] == 2) | (df['pos_comparisons'] == 3), 'dummynews_goodbad'] = 0
df.loc[(df['pos_comparisons'] == 0) | (df['pos_comparisons'] == 1), 'dummynews_goodbad'] = 1
#Belief adjustment
df["beliefadjustment"] = df["posterior_median"] - df["prior_median"]
#Normalized belief adjustment
df["beliefadjustment_normalized"] = np.nan
df.loc[df['dummynews_goodbad'] == 0, 'beliefadjustment_normalized'] = df['beliefadjustment']
df.loc[df['dummynews_goodbad'] == 1, 'beliefadjustment_normalized'] = df['beliefadjustment']*-1<jupyter_output><empty_output><jupyter_text>6.1.2 Belief Adjustment Distribution<jupyter_code>Main_Figure1(df)<jupyter_output><empty_output><jupyter_text>Figure 1 is an easy way to view the belief adjustments of the subjects, even though, they aren't enough for causal inference. As it is observable from Panel A. subjects adjusted their beliefs in line with the feedback they have received. For instance, in the positive feedback group, we see the accumulation of observations between 0 and +50 while an opposite pattern is observable for the negative feedback group.In Panel B. we see that for the individuals who have received positive feedback, the pattern observe is not so much different from the pattern in Panel A. However, the graph significantly changes for the negative feedback when subjects elicit their beliefs 1 month after the feedback.  6.1.3 Belief Adjustment Distribution  & Test performances  **Calculating Bayesian Predictions**<jupyter_code>#Bayesian predictions
get_Bayesian_predictions(df)
#Calculating Bayesian Posterior Median Beliefs
df["posterior_median_bayes"] = (df["post_1"] + df["post_2"] + df["post_3"] + df["post_4"] + df["post_5"])*100
##Bayesian predictions for belief adjustment
df["beliefadjustment_bayes"] = df["posterior_median_bayes"] - df["prior_median"]

#Normalization of Bayesian predictions
df["beliefadjustment_bayes_norm"] = np.nan
df.loc[df['dummynews_goodbad'] == 0, 'beliefadjustment_bayes_norm'] = df['beliefadjustment_bayes']
df.loc[df['dummynews_goodbad'] == 1, 'beliefadjustment_bayes_norm'] = df['beliefadjustment_bayes']*-1<jupyter_output><empty_output><jupyter_text>There are two sub treatment groups in the immediate treatment: confidence_direct_15minuteslater' and 'confidence_direct_immediate'. Zimmermann(2020) made this differentiation for a robustness check to see whether there would be a difference between eliciting beliefs immediately after feedback or 15 minutes later. However, the main results derived from the comparison between 1-month treatment and pooled treatment which combines 15 minutes later and immediate treatment. So, a dummy is created for further analysis; with 1 indicating treatment 1 month and 0 indicating pooled direct treatment.<jupyter_code>#Generate dummy for direct treatment and 1 month treatment
df["dummytreat_direct1month"] = np.nan
df.loc[(df['treatgroup'] == 3) | (df['treatgroup'] == 4), 'dummytreat_direct1month'] = 0
df.loc[df['treatgroup'] == 2, 'dummytreat_direct1month'] = 1
#Grouping IQ test scores
df["test_1"] = np.nan
df.loc[(df['rscore'] == 0) | (df['rscore'] == 1) | (df['rscore'] == 2) | (df['rscore'] == 3) | (df['rscore'] == 4), 'test_1'] = 1
df.loc[(df['rscore'] == 5), 'test_1'] = 2
df.loc[(df['rscore'] == 6), 'test_1'] = 3
df.loc[(df['rscore'] == 7) | (df['rscore'] == 8) | (df['rscore'] == 9) | (df['rscore'] == 10), 'test_1'] = 4
#Average prior belief
df['prior_av'] = df.groupby(['test_1', 'dummytreat_direct1month'])["prior_median"].transform('mean')
#Average posterior belief if feedback is positive
df['post_av_pos'] = df[df["dummynews_goodbad"] == 0].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median"].transform('mean')
#Average posterior belief if feedback is negative
df['post_av_neg'] = df[df["dummynews_goodbad"] == 1].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median"].transform('mean')
Main_Figure2(df)<jupyter_output><empty_output><jupyter_text>Figure 2 plots the individuals' prior and posterior belief of being in the upper half of their group for differents levels in test performances. Test performances are grouped as 1, 2, 3 and 4; with 1 being the lowest score group. Also, posteriors are plotted with two different lines.It is easy to see that in Panel A. ConfidenceDirect, subject's posterior beliefs visibly differ from their prior beliefs in line with the feedback they received. However, in Panel B, the posterior beliefs of subjects that received negative feedback are visibly closer to the prior beliefs.Compared to the first graph, here causal inference is made possible by grouping individuals according to their test scores.These figures are also a great way to observe ex-ante overconfident-average priors lie consistently above 50 percent- which can be theoretically explained by Bayesian updating (see. Benoît and Dubra, 2011) and also visible in the figures from the study of Zimmermann (2020).  6.1.4. Regression Outputs  In the study, Zimmermann (2020) perform 8 OLS estimations(see in Table 1) with type HC1 heteroscedasticity robust covariance. :(1) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type) in case of positive feedback(2) Same as (1) but controling for $X_i$ (rank and normalized bayesian belief adjustment)(3) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type) in case of negative feedback(4) Same as (3) but controling for $X_i$ (rank and normalized bayesian belief adjustment)(5) Regressing $y_i$ (normalized belief adjustment) on $T_i$ (treatment type), $feedback_i$ (feedback type) and $I_i$ (interaction term indicating whether individual is in the interest group or not)(6) Same as (5) but controling for $X_i$ (rank and normalized bayesian belief adjustment)(7) Same as (5) but including rank fixed effects with their interaction with treatment(8) Same as (7) but controling for $X_i$ (normalized bayesian belief adjustment)<jupyter_code>df_good = pd.DataFrame({"beliefadjustment_normalized": df[df["dummynews_goodbad"] == 0]['beliefadjustment_normalized'], "dummytreat_direct1month": df[df["dummynews_goodbad"] == 0]['dummytreat_direct1month'], "rank": df[df["dummynews_goodbad"] == 0]['rank'], "beliefadjustment_bayes_norm": df[df["dummynews_goodbad"] == 0]['beliefadjustment_bayes_norm']})
#(1)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month", data=df_good)
reg_1 = model_ols.fit(cov_type='HC1')
#(2)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month + rank + beliefadjustment_bayes_norm", data=df_good)
reg_2 = model_ols.fit(cov_type='HC1')
df_bad = pd.DataFrame({"beliefadjustment_normalized": df[df["dummynews_goodbad"] == 1]['beliefadjustment_normalized'], "dummytreat_direct1month": df[df["dummynews_goodbad"] == 1]['dummytreat_direct1month'], "rank": df[df["dummynews_goodbad"] == 1]['rank'], "beliefadjustment_bayes_norm": df[df["dummynews_goodbad"] == 1]['beliefadjustment_bayes_norm']})
#(3)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month", data=df_bad)
reg_3 = model_ols.fit(cov_type='HC1')
#(4)
model_ols = smf.ols(formula="beliefadjustment_normalized ~ dummytreat_direct1month + rank + beliefadjustment_bayes_norm", data=df_bad)
reg_4 = model_ols.fit(cov_type='HC1')<jupyter_output><empty_output><jupyter_text>Next regressions are DID models, therefore interaction term for our interest group is added.<jupyter_code>#Generating interaction term
df["interact_direct1month"] = df["dummytreat_direct1month"]*df["dummynews_goodbad"]
#(5)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month", data=df)
reg_5 = model_ols.fit(cov_type='HC1')
#(6)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + rank + interact_direct1month + beliefadjustment_bayes_norm", data=df)
reg_6 = model_ols.fit(cov_type='HC1')<jupyter_output><empty_output><jupyter_text>Next spesifications are with rank fixed effects to include their interaction with treatment. So rank dummies are generated and added to the regression as well as their interaction term: `rankdummy`$_i$ *`dummytreat_direct1month`.<jupyter_code>#Rank dummies
get_rank_dummies(df)
#Intereaction term: Rank dummy*Treatment dummy
get_rank_interation_term(df)
#(7)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month + rankdummy1 + rankdummy2 + rankdummy3 + rankdummy4 + rankdummy5 + rankdummy6 + rankdummy7 + rankdummy8 + rankdummy9 + rankdummy1_interact + rankdummy2_interact + rankdummy3_interact + rankdummy4_interact + rankdummy5_interact + rankdummy6_interact + rankdummy7_interact + rankdummy8_interact + rankdummy9_interact", data=df)
reg_7 = model_ols.fit(cov_type='HC1')
#(8)
model_ols = smf.ols(formula= "beliefadjustment_normalized ~ dummytreat_direct1month + dummynews_goodbad + interact_direct1month + beliefadjustment_bayes_norm + rankdummy1 + rankdummy2 + rankdummy3 + rankdummy4 + rankdummy5 + rankdummy6 + rankdummy7 + rankdummy8 + rankdummy9 + rankdummy1_interact + rankdummy2_interact + rankdummy3_interact + rankdummy4_interact + rankdummy5_interact + rankdummy6_interact + rankdummy7_interact + rankdummy8_interact + rankdummy9_interact", data=df)
reg_8 = model_ols.fit(cov_type='HC1')
Main_Table_1(df)<jupyter_output><empty_output><jupyter_text>From the first two regressions, it is clear that belief adjustments of individuals in the direct treatment group and the 1-month treatment group did not differ significantly. In other words, time has no significant effect on individuals belief adjustment if they have received positive feedback. These findings overlap with the finding in Zimmermann(2020) that state positive feedback has a persistent effect on belief dynamics.The (3) and (4) regression also correlates with Zimmermann(2020) findings since it is clear that the belief adjustments are significantly different between treatment groups at the .01 level, referring to the short-term effect of negative feedback.As it is observable from the column 5 and 6 the interaction term for negative feedback and 1-month treatment group is significant again at the .01 level, in line with the findings from the previous columns. The regression output columns with control variables (rank and Bayesian belief adjustment) also gave the same results in terms of significance levels which strengthens the results. Again these are also in line with the findings of Zimmermann (2020).The last columns are the DID models with rank effects interacted with treatment group and also includes rank dummy controls, as it is observable the interaction term for negative feedback and 1-month treatment group is significant again at the .05 level in (7) and at the .01 level in (8).Overall, all of the results and all of the values are exactly the same as the ones in the study of Zimmermann (2020); confirming the researcher's results. --- 6.2. Robustness Checks ---Some parts of the Appendix that would be relavant for the extension part of this project is added to the project notebook.  6.2.1 Appendix A.4 - Alternative Definition of Positive/Negative Feedback    **Heterogenous Feedbacks** Basically, to be considered to receive a positive or negative feedback, a subject should receive all the comparisons as the same kind; positive feedback if he/she received 3 positive comparisons and negative feedback if he/she received 3 negative comparisons.<jupyter_code>#Creating the new feedback dummy accordingly
df["dummynews_goodbad_h"] = np.nan
df.loc[df['pos_comparisons'] == 3, 'dummynews_goodbad_h'] = 0
df.loc[df['pos_comparisons'] == 0, 'dummynews_goodbad_h'] = 1
Appendix_Table_1(df)<jupyter_output><empty_output><jupyter_text>All of the values mimic the values in Table 1 in terms of significance, which means the definition of feedback in the main study was robust. However, for the negative feedback, the coefficients are larger than the previous results which might suggest more negative feedback may affect belief adjustment more negatively. --- 6.2.2 Appendix A.6 - Updating in the Short-Run  ---Looking specifically at short-run updating is not the main aim of Zimmermann(2020), however, the researcher finds it relevant to include the results from data. As it is also mentioned in the study, previous findings on short-term belief updating suggest two phenomena: conservatism (Möbius et al., 2013) and asymmetry (Eil and Rao, 2011). **Conservatism** According to Bayes' rule, people should adjust their belief by about 20 percentage points, on average after a signal. However, Zimmermann (2020) found a way lower normalized belief adjustment value of 11.80 while the correlation between Bayesian predictions and belief adjustment is being 0.459.<jupyter_code>df_short = df[df['treatgroup'] == 4]
df_short['beliefadjustment_normalized'].mean()
correlation = df_short[['beliefadjustment_normalized','beliefadjustment_bayes_norm']].corr(method='pearson')
correlation<jupyter_output><empty_output><jupyter_text>**Asymmetry** In order to investigate asymmetry, Zimmermann (2020) analyze the effect of Bayesian predictions on belief adjustments, separately for positive and negative feedback.<jupyter_code>Appendix_Table_3(df)<jupyter_output><empty_output><jupyter_text>As it is suggested by the findings in Column 3, DID reveals no significant relationship. In other words, the Bayesian predictions don’t significantly differ between feedback groups. However, clearly, the coefficient for Bayesian prediction is slightly larger in positive information than in negative information which suggests some asymmetry. --- 6.2.3. Appendix A.7 - No Feedback Condition  ---<jupyter_code>Appendix_Figure_1(df)<jupyter_output><empty_output><jupyter_text>The figure clearly demonstrates that in absence of feedback, many subjects do not change their beliefs and a clear symmetry is observable for those who changed their beliefs.<jupyter_code>#Stats for control group; the only numbers I couldn't get exactly from the study of Zimmermann (2020) is the numbers below.
#The reported mean belief adjustment being 0.22, with a standard deviation of 17.83.
df[df['treatgroup'] == 7]['beliefadjustment'].describe()<jupyter_output><empty_output><jupyter_text>--- 6.2.4. Appendix A.8 - Figures Bayesian Posteriors  ---<jupyter_code>#Calculating Bayesian Posterior Averages
df['prior_av_b'] = df.groupby(['test_1', 'dummytreat_direct1month'])["prior_median"].transform('mean')
df['post_av_pos_b'] = df[df["dummynews_goodbad"] == 0].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median_bayes"].transform('mean')
df['post_av_neg_b'] = df[df["dummynews_goodbad"] == 1].groupby(['test_1', 'dummytreat_direct1month'])["posterior_median_bayes"].transform('mean')
Appendix_Figure_2(df)<jupyter_output><empty_output><jupyter_text>This figure is the same as Figure 2 only here Bayes prediction of posterior beliefs are used instead of actual posteriors. --- 7. Extension  --- The extension consist of two parts.In the first part, I will replicate the first part of the study of Eil and Rao (2011) which is a quite similar experiment in terms of experimental design and empirical strategy that has been used. I think replicating an experiment that Zimmermann(2020) had taken reference would be beneficial to understand the motives and aims of Zimmermann(2020) while forming his study in the way it has been done. Also, I made use of some parts of the Eil and Rao (2011) study in the second part of the extension.In the second part, I will present the extensions I have added such as data visualizations and further analysis on the effect of feedback. --- 7.1. Replication of Eil and Rao (2011)   As an extension, I decided to replicate the study of Eil and Rao (2011) which is a relevant experiment that is also quite oftenly mentioned by Zimmermann (2020) in their study.  7.1.1. Introduction  The experiment consists of 3 main parts. I decided to replicate the first part of the experiment that analyzes the posterior beliefs of the subjects receiving the most “extreme” feedback(all positive or all negative). Eil and Rao (2011) main aim in this experiment is to observe the asymmetry in the belief updating when subjects receive different types (positive and negative) of signals. Unlike Zimmermann (2020), Eil and Rao (2011) didn’t vary the time between treatments but varied the scoring method(referred to as image task) of subjects. The subjects who are assigned to an image task, either scored according to their IQ test or physical attractiveness(beauty) and also a control group task.   7.1.2. Experiment Design  The experiment is also quite similar to the study of Zimmermann(2020) by nature.1. Subjects were first either taken an IQ test and assigned to groups of 10 or went through a series of speed dates with 10 other people and asked to give a rating to each in terms of their physical attractiveness from 1 to 10.2. After the tasks, subjects were asked to elicit their belief about their ranking within their group of 10.3. Then subjects received 3 cards that revealed whether they were ranked higher or lower than their randomly picked groupmates. For the control task, they were told the number on the cards was randomly picked. Belief elicitation was done in three rounds since subjects were asked to elicit their beliefs about their ranks within the group after each card.**Variable Definitions**|Variable name|Definition|Beauty|IQ|Control||---|---|---|---|---||`meanbelief_priorbayesimage`|Mean Bayesian belief prediction calculated from prior belief|√|√|||`meanbelief_priorbayescard`|Mean Bayesian belief prediction calculated from prior belief|||√||`frac_upimage`|0 if good news, 1 if bad news|√|√|||`frac_upcard`||||√||`mb_fracup`|interaction term: $meanbelief_priorbayesimage_i$*$frac_upimage_i$|√|√|||`mb_fraccard`|interaction term: $meanbelief_priorbayescard_i$*$frac_upcard_i$|||√|  7.1.3. Emprical Strategy   Even though, a model is not specified in the study of Eil and Rao (2011). One can come up with the following DID model for the estimations presented in the Table 1:\begin{equation}meanbelief_i = \alpha + \beta meanBayesian_i + \gamma feedback_i + \delta I_i + \epsilon_i\end{equation}where $feedback_i$ is the dummy variable for feedback and $meanBayesian_i$ is the mean Bayesian belief prediction. $I_i$ is for the interaction term that is generated by $meanBayesian_i$*$feedback_i$. Bayesian predictions are calculated using reported priors and the received signal type; then the term mean Bayesian, `meanbelief_priorbayesimage`, is produced by taking the expected rank according to posteriors reported through the rounds.  7.1.4. Replication<jupyter_code>#Importing the adjusted and merged data set provided by the Eil and Rao (2011)
df_ex = pd.read_stata('data/extension.dta')<jupyter_output><empty_output><jupyter_text>The first figure presented by Eil and Rao (2011) is formed by regressing the subject's observed mean belief on the predicted mean belief using Bayes' Rule and grouping subjects by the feedback type that they have received.<jupyter_code>Extension_Figure_1(df_ex)<jupyter_output><empty_output><jupyter_text>At first glance, a visible asymmetry can be observed in the Beauty group between good and bad news; while such asymmetry is not that pronounced in the IQ test group. We interpret this asymmetry by the difference in slopes of the lines. As it can be seen in Panel A. bad news condition has a flatter slope, meaning that actual posteriors' have a shorter value length which means even though bayesian posteriors might suggest extreme updating the actual updating is relatively more conservative. In other words, the slope of the bad news line suggests an insensitivity to signal strength. Compared to bad news, we see a steeper slope for good news which suggests an exaggerated response to signal strength. Therefore, an asymmetry in responses for Beauty treatment is pronounced. If we look at Panel C. Control group we see a constant slope which implies that such asymmetries are the results of image treatments. The slope of the good news line in IQ treatments is also steeper compared to the control group however, not as much as it is in Beauty treatment.Another feature to analyze in this figure is the observations scattered around the fitted line. If we compare the two treatment groups, it is clear that beliefs are noisier for bad news as it is for good news.Also, it is worth noting that having slopes equal to zero and negative intercept would imply optimism in beliefs in general.<jupyter_code>#Used regressions in Extension Figure 2 with the standard errors clustered at subject level
reg_B_b_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                           data=df_ex[(df_ex['frac_upimage'] == 0) & (df_ex['IQtask'] ==0)], group_var='ID')
reg_B_g_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                           data=df_ex[(df_ex['frac_upimage'] == 1) & (df_ex['IQtask'] ==0)], group_var='ID')
    
reg_IQ_b_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                            data=df_ex[(df_ex['frac_upimage'] == 0) & (df_ex['IQtask'] ==1)], group_var='ID')
reg_IQ_g_ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage',
                            data=df_ex[(df_ex['frac_upimage'] == 1) & (df_ex['IQtask'] ==1)], group_var='ID')
    
reg_C_b_ = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard',
                           data=df_ex[(df_ex['frac_upcard'] == 0)], group_var='ID')
reg_C_g_ = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard',
                           data=df_ex[(df_ex['frac_upcard'] == 0)], group_var='ID')
Extension_Figure_2(df_ex)<jupyter_output><empty_output><jupyter_text>Panel. D represents the plots of the R^2s from the regressions used in Extension Table 1 but divided according to the type of feedback. Unfortunately, even though I used regressions with standard errors clustered at the subject level as is described by Eil and Rao (2011); I couldn’t get the R^2 value of the IQ test/good newsgroup right, which is confusing since I get exactly the same result from the pooled regression in the Extension Table 1. According to Eil and Rao (2011), the $R^2$ for IQ/good newsgroup should be below 0.7.Taking the result of Eil and Rao (2011) as a reference point, R^2 differ for the Beauty and IQ group while that is obviously not the case for the Control group. **Table 1**<jupyter_code>#Used regressions in Extension Table 1 with the standard errors clustered at subject level
reg_B = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage', data=df_ex[(((df_ex['frac_upimage'] == 0) | (df_ex['frac_upimage'] == 1)) & (df_ex['IQtask'] ==0))], group_var='ID')
reg_IQ = cluster_fit('meanbeliefimage ~ meanbelief_priorbayesimage + mb_fracup + frac_upimage', data=df_ex[(((df_ex['frac_upimage'] == 0) | (df_ex['frac_upimage'] == 1)) & (df_ex['IQtask'] ==1))], group_var='ID')
reg_C = cluster_fit('meanbeliefcard ~ meanbelief_priorbayescard + mb_fracupcard + frac_upcard', data=df_ex[(df_ex['frac_upcard'] == 0) | (df_ex['frac_upcard'] == 1)], group_var='ID')
Extension_Table_1(df_ex)<jupyter_output>===================================================
                            Beauty     IQ   Control
---------------------------------------------------
Intercept                  2.05***  1.05    2.29***
                           (0.52)   (0.76)  (0.76) 
R-squared                  0.87     0.87    0.72   
R-squared Adj.             0.86     0.87    0.71   
frac_upcard                                 -0.53  
                                            (0.95) 
frac_upimage               -2.31*** -0.58          
                           (0.54)   (0.81)         
mb_fracup                  0.41***  0.07           
                           (0.10)   (0.13)         
mb_fracupcard                               -0.00  
                                            (0.19) 
meanbelief_priorbayescard                   0.59***
                                            (0.10) 
meanbelief_priorbayesimage 0.64***  0.85***        
                           (0.08)   (0.10)         
===========[...]<jupyter_text>---*See below for a reader friendly version of the Extension Table 1 that I created with the same values.*||Beauty|IQ|Control||---|---|---|---||Intercept|**2.05** |1.05 |**2.29** |||(0.52)|(0.76)|(0.76)||Bayesian μ|**0.64** |**0.85** |**0.59** |||(0.08)|(0.85)|(0.10)||Bayesian μ *1{All positive feedback}|**0.41** |0.07 |-0.00 |||(0.10)|(0.13)|(0.19)||1{All positive feedback}|**-2.31** |-0.58 |-0.53 |||(0.54)|(0.81)|(0.95)||Intercept|**2.05** |**1.05** |**2.29**|||(0.52)|(0.76)|(0.76)||$R^2$|0.87|0.87|0.72 ||$R^2$ Adj.|0.86|0.87 |0.71|||Standard errors in parentheses. * p<.1, *p<.05*, ***p<.01| The findings found in Extention Table1 are in furtherance of the interpretations made for the Extention Figure2. Like it is suggested by the figure, the asymmetry in belief dynamics-the difference between positive and negative feedback conditions- is only significant in Beauty at the 0.01 level of significance.  7.1.5. Comparison with Zimmermann (2020)   It is easy to see the two experiments share a lot of experimental and empirical strategies. First of all, both studies have a longitudinal feature since they elicit subjects’ beliefs two times. Secondly, the two studies have the same focus which is the asymmetry in belief dynamics that arise as a result of feedback type. In addition, both studies make use of DID models in their predictions. Nevertheless, while Eil and Rao (2011) are used the heterogeneous definition of feedbacks, Zimmermann(2020) choose to allow homogenous comparison types within feedback groups- even though a robustness check is made (see Appendix A.4)-.  A fundamental difference is the employment of treatment groups. While Eil and Rao (2011) were interested in belief updating in different tasks (Beauty and IQ), Zimmermann (2020) was interested in belief updating when elicitation time differs (Direct and 1 month). Therefore, all of the results Eil and Rao (2011) found was derived from short-run updating. At this point, it is intriguing that whether the difference between the tasks would be as pronounce as it was; if Zimmermann (2020) also had a Beauty image task that is also measured after 1 month. --- 7.2. Further Extension  --- 7.2.1. Alternative Data Visualizations<jupyter_code>Extension_Figure_3(df)<jupyter_output><empty_output><jupyter_text>As a small extension, I decided to create a figure to combine the figure from Appendix A.8 with Figure 2 since it would be a good way to view posterior beliefs in comparison with Bayesian predictions.If we look at the first row we see that observed beliefs follow a relatively similar pattern, in terms of subjects' reported percentages of being in the upper half of the group, to Bayesian predictions compared to the second row where graphs differ visibly.Also, in the short run, it is easy to see that people are conservative in belief updating. For instance, in the group of high test scores and positive feedback, according to Bayes' Rule, one would expect a posterior slightly above %90 while it is only slightly above %80; suggesting a difference near %10 percent.The opposite can be observable in long-run belief updating in the negative feedback group since in every test score group people reported posterior beliefs that are much higher than Bayesian predictions-especially for the individuals with lower test scores. What I found interesting in the results from long-run, is the difference in belief updating when feedback is positive. In the last figure, we see that no matter the test scores individuals tend to report a belief near %80 percent when they receive positive feedback. I think this would be a good starting point for further research in dynamic beliefs. ---Making use of Extention Figure 2 from the study of Eil and Rao (2011), I created two similar data visualization using the data of Zimmermann (2020).<jupyter_code>Extension_Figure_4(df)<jupyter_output><empty_output><jupyter_text>In both figures, negative feedback lines are flatter and specifically very flat in Panel B. 1 month treatment. Meaning that in both treatment groups subjects'  weighed negative information less heavily than positive information and even much less heavily in 1-month treatment.<jupyter_code>Extension_Figure_5(df)<jupyter_output><empty_output><jupyter_text>In this visiualization, it is easier to see the asymmetry between positive and negative feedback. If we look at Panel A. Positive Feedback, we see that slopes of fitted lines of direct and 1month treatment are almost the same; while in Panel B. they clearly differ.  7.2.2 Further Analysis with No Feedback Group  As a control group, Zimmermann (2020) also added a group where no feedback is given to the subjects. The main aim of adding this group was the investigate the potential time trends in beliefs that may exist independently of receiving feedback. However, I couldn't find any further analyzes with this group neither in the study or appendix-except the visiualization in Appendix A.7.So as a further analyze I decided to investigate the difference between feedback(positive or negative) and no feedback groups within the 1-month condition to see whether receiving feedback itself cause a difference in belief adjustment.  7.2.2.1. Estimation Strategy  For my analysis, I made use of a simple difference-in-difference model of the following kind:\begin{equation}belief_i = \alpha + \beta dummyfeedback_i + \gamma secondwave_i + \delta I_i + \epsilon_i\end{equation}where $dummyfeedback_i$ is the dummy variable for receiving feedback and $secondwave_i$ is the dummy for posterior belief. $I_i$ is for the interaction term that is generated by $dummyfeedback_i$*$secondwave_i$. If the subjects obtained feedback and the belief is a posterior belief, (i)feedback dummy (ii)secondwave dummy and (iii) interaction term are equal to 1; making the $ \delta$ the coefficient of interest.For the estimation, I reshaped data to have a single belief variable combining posterior and prior beliefs and to have a time variable that qualifies in which time the belief is elicited.  7.2.2.2. Estimation<jupyter_code>#Making a copy of the original data to start the reshape
df_control = df
#Generating an ID number for each participant
df_control['ID'] = df_control.index + 1
#Filtering data to have only 1 month and no feedback treatment
df_control = df[(df['treatgroup'] == 2) | (df['treatgroup'] == 7)]
#Filtering and adjusting data
df_control = df[(df['treatgroup'] == 2) | (df['treatgroup'] == 7)]
df_control = df_control[['ID', 'treatment', 'posterior_median', 'prior_median']]
df_control.rename(columns = {"prior_median": "belief1", "posterior_median": "belief2" }, inplace=True)
df_control["ID"] = df_control.index
#Transforming data from a wide format to long format
df_control = pd.wide_to_long(df_control, stubnames='belief', i="ID", j="time")
#Creating a column with time index
df_control = df_control.reset_index(level=1)
#Final data
df_control
get_variables_for_extension(df_control)
Extension_Table_2(df_control)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0821950020308258, 'fraction_numerical': 0.022232198208597877, 'mean_word_length': 5.171503957783641, 'pattern_counts': {'":': 10, '<': 97, '<?xml version=': 0, '>': 95, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '144510', 'license': 'MIT', 'n_tokens_mistral': 13298, 'n_tokens_neox': 12101, 'n_words': 5507, 'path': 'project_notebook.ipynb', 'repo_name': 'aysuavci/ose-data-science-course-project-aysuavci', 'size': '46789'}