<jupyter_start><jupyter_text>https://www.princeton.edu/~otorres/Panel101R.pdfhttps://dss.princeton.edu/training/Panel101.pdfhttps://www.sas.upenn.edu/~fdiebold/Teaching104/Econometrics.pdf<jupyter_code># Exploring panel data

library(foreign)
Panel <- read.dta("http://dss.princeton.edu/training/Panel101.dta")


head(Panel)
summary(Panel)
coplot(y ~ year|country, type="l", data=Panel)        # Lines
coplot(y ~ year|country, type="b", data=Panel)        # Points and lines<jupyter_output><empty_output><jupyter_text>Bars at top indicates corresponding graph (i.e. countries)from left to right starting on the bottom row (Muenchen/Hilbe:355)  Exploring panel data<jupyter_code>#install.packages("car")
library(foreign)
Panel <- read.dta("http://dss.princeton.edu/training/Panel101.dta")
library(car)
scatterplot(y~year|country, boxplots=FALSE, smooth=TRUE, reg.line=FALSE, data=Panel)
library(car)
scatterplot(y~year|country, boxplots=FALSE, smooth=TRUE, data=Panel)<jupyter_output>Loading required package: carData<jupyter_text>Heterogeneity: unobserved variables that do not change over time FIXED-EFFECTS MODEL(Covariance Model, Within Estimator, Individual Dummy Variable Model, Least Squares Dummy Variable Model)<jupyter_code>install.packages("gplots")
library(foreign)
Panel <- read.dta("http://dss.princeton.edu/training/Panel101.dta")
library(gplots)
plotmeans(y ~ country, main="Heterogeineity across countries", data=Panel)
# library(gplots)
# plotmeans(y ~ country, main="Heterogeineity across countries", data=Panel)
# plotmeans
# draw a 95% confidence interval around the means 
detach("package:gplots")
# Remove package ‘gplots’ from the workspace
ols<- lm(y ~ x1, data=Panel)
summary(ols)
yhat <-ols$fitted
plot(Panel$x1, Panel$y, pch=19, xlab="x1", ylab="y")
abline(lm(Panel$y~Panel$x1),lwd=3, col="red")
fixed.dum <-lm(y ~ x1 + factor(country) -1, data=Panel)
summary(fixed.dum)
yhat<-  fixed.dum$fitted
# > library(car)
scatterplot(yhat~Panel$x1|Panel$country, boxplots=FALSE, xlab="x1", ylab="yhat",smooth=FALSE)
abline(lm(Panel$y~Panel$x1),lwd=3, col="red")<jupyter_output><empty_output><jupyter_text>Comparing OLS vs LSDV modelEach component of the factor variable (country) is absorbing the effects particular to each country. Predictor x1 was not significant in the OLS model, once controlling for differences across countries, x1 became significant in the OLS_DUM (i.e. LSDV model).<jupyter_code># install.packages("apsrtable")
library(apsrtable)
apsrtable(ols,fixed.dum, model.names = c("OLS", "OLS_DUM"))   # Displays a table in Latex form
# install.packages("plm")
library(plm)
fixed <-plm(y ~ x1, data=Panel, index=c("country", "year"), model="within")
summary(fixed)<jupyter_output><empty_output><jupyter_text>Display the fixed effects (constants for each country)<jupyter_code>fixef(fixed)<jupyter_output><empty_output><jupyter_text>Testing for fixed effects, null: OLS better than fixed<jupyter_code>pFtest(fixed, ols)    
# Testing for fixed effects, null: OLS better than fixed<jupyter_output><empty_output><jupyter_text>RANDOM-EFFECTS MODEL(Random Intercept, Partial Pooling Model)<jupyter_code>random <-plm(y ~ x1, data=Panel, index=c("country", "year"), model="random")
summary(random)
# Setting as panel data (an alternative way to run the above model
Panel.set<-  pdata.frame(Panel, index = c("country", "year"))
# Random effects using panel setting (same output as above)
random.set<-  plm(y ~ x1, data = Panel.set, model="random")
summary(random.set)<jupyter_output><empty_output><jupyter_text>Fixed or Random: Hausman testIf the p-value is significant (for example <0.05) then use fixed effects, if not use random effects<jupyter_code>phtest(fixed, random)<jupyter_output><empty_output><jupyter_text>OTHER TESTS/ DIAGNOSTICS<jupyter_code>fixed.time <- plm(y ~ x1 + factor(year), data=Panel, index=c("country", "year"), model="within")
summary(fixed.time)
fixef(fixed.time)
# Testing time-fixed effects. The null is that no time-fixed effects needed
pFtest(fixed.time, fixed)
plmtest(fixed, c("time"), type=("bp"))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13705335291238374, 'fraction_numerical': 0.011013215859030838, 'mean_word_length': 7.514583333333333, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 37, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '46181', 'license': 'MIT', 'n_tokens_mistral': 1480, 'n_tokens_neox': 1408, 'n_words': 324, 'path': 'panel_data/panel100_R.ipynb', 'repo_name': 'jonduan/education_analytics', 'size': '4086'}