<jupyter_start><jupyter_text>Applied Econometrics (Part 4) Lviv Data Science Summer School <NAME> & <NAME>**July 2019**  Density / distribution / probability forecasts- Point forecasts and uncertainty of this information- How do we specify the certainty?- In contrast to single value forecast, probabilistic forecasts assign a probability to each of a number of different outcomes.- Traditionally point forecasts: first two moments What is the distribution?<jupyter_code>options(repr.plot.width=7, repr.plot.height=4)
## Normal distribution
x <- rnorm(100000, 0, 1)
x <- (x - mean(x))/sd(x) ## z-score
emp_q = as.numeric(quantile(x, probs = c(0.02775, 0.159, 0.5, 0.841, 0.97725)))
emp_q
hist(x, breaks = 100, main="2 sigmas") ## normally 3 :)
abline(v = emp_q, col="red")
plot(ecdf(x)) # empirical cumulative distribution function<jupyter_output><empty_output><jupyter_text>Can we model the distribution ? - cumulative distribution is similar to sigmoid function Sigmoid$$ F(x) = \frac{1}{1+\exp(-x)}$$<jupyter_code>fx = function(x) {    
    1/(1+exp(-x))
}
plot(fx(seq(-6, 6, length.out=20)), type="o", ylab="", main="Sigmoid")<jupyter_output><empty_output><jupyter_text>Modeling probability$$ F(y|x) = \Pr(Y \leq y| X = x) $$Logistic regression$$\Pr(Y \leq y | X = x) = \frac{\exp(x)}{1 + \exp(x)}$$<jupyter_code>y = seq(-6, 6, length.out=20)
pr = exp(y)/(1 + exp(y))
plot(y, pr, type="o", main="Conditional cumulative distribution function", ylab="Probability")
abline(v=0, lwd=0.1)<jupyter_output><empty_output><jupyter_text>Logistic regression Theory:In the linear regression we have $$ y^{(i)}=β_0+β_1x^{(i)}_1 + … + β_p x^{(i)}_p $$To get the probabilities between 0 and 1, we wrap it with the sigmoid/logistic function$$ P(y^{(i)} = 1) = \frac{1}{1+\exp(- (β_0+β_1x^{(i)}_1 + … + β_p x^{(i)}_p ))}$$And the log-odds we have$$\log\left(\frac{ P( y = 1 )}{ 1 − P ( y = 1 )} \right ) = \log \left( \frac{P ( y = 1 )}{ P ( y = 0 )} \right) = β_0 + β_1 x_1 + … + β_p x_p $$   - it calculates probability of each observation in data set- the coefficients from logistic regression tell us how much that particular variables contribute to the log odds. Pros:- easy classification model    - simplicity close to OLS- gives probabilitiesCons:- not easy interpretation (multiplicative weights) -> marginal effects- complete separation - if a feature completely separates the two classes logit cannot be trained (convergence)  Excercise:A traditional data set from UCLA which has 4 variable called admit, GRE score, GPA and rank of their undergrad school.<jupyter_code>df1 <- read.csv("https://stats.idre.ucla.edu/stat/data/binary.csv")
head(df1)
df1$rank <- as.factor(df1$rank)
# head(df)

fit1 <- glm(admit ~ gre + gpa + rank, data = df1, family = binomial)
summary(fit1)
test <- data.frame(gre = 500, gpa = 3.1, rank = as.factor(1))
pred <- predict(fit1, df1, type="respons")
## Exercise: plot probability
plot(pred)
plot(sort(pred)) ## create axis X correctly<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15546639919759278, 'fraction_numerical': 0.03142761618187897, 'mean_word_length': 5.020120724346077, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '568186', 'license': 'MIT', 'n_tokens_mistral': 1107, 'n_tokens_neox': 1001, 'n_words': 323, 'path': 'L4.ipynb', 'repo_name': 'barunik/Lviv2019', 'size': '3003'}