Standard method for calculating contribution of individual variable to outcome

Question: I'm looking at the change in vote shares for an election between two periods. I'd like to say something about the contribution of one variable in particular towards the election swing I document. I'm wondering if there is a standard approach in the literature.
I'd like to be able to say something like X accounts for 5 points of the 20 point swing.
Is there a standard way to calculate a variable's contribution? Predict the outcome with X set equal to zero and compare the difference? Standardize the coefficients and take the value of the beta coefficient as the percent contribution?
Comment: "Predict the outcome with X set equal to zero and compare the difference? Standardize the coefficients and take the value of the beta coefficient as the percent contribution?" -- these are both perfectly valid ways to define "contribution." Which one makes sense is entirely up to you. Or more likely, entirely up to which method makes sense to use given your data. Or are you wondering whether one will produce estimates that are in some sense "more valid" than the other?
Comment: what data have you got?
Comment: Vote shares at the equivalent of the congressional district level + controls + variable of interest plus an IV.
Comment: I was wondering if there is a commonly accepted approach in the literature to this sort of calculation.
Answer: Given that you have panel data, the "standard approach" would be to use a fixed effects regression in order to eliminate time-invariant unobserved candidate and state characteristics. The earliest example I can provide you with is Levitt (1994). He wanted to know how campaign spending by incumbent candidates and challengers affects the election outcome. If you intend to do something similar, you have to ask yourself the question: "Is there something about my candidates and/or their states that I don't observe but which is correlated with my variable of interest and the election outcome?"
In the case of Levitt (1994), campaign spending is likely to depend on a candidate's quality. The guy who rallies the masses with his great speeches is likely to attract more funding, so he can spend more on his campaign. But then his great speeches are likely to affect the election outcome, too! The problem now is that we do not observe this guy's quality. What does this mean for your regression? If you regress
$$\text{Re-elect}_{it} = \alpha + \beta \text{spending}_{it} + X'_{it}\gamma + \epsilon_{it}$$
where the outcome is 1 if the incumbent gets re-elected and zero otherwise, spending denotes a candidates campaign spending and $X'_{it}$ includes some characteristics of the candidate and the state of the election. If quality is unobserved, this will be included in the error term resulting in a correlation with spending. Quality is positively correlated with both being elected and with campaign spending, so your estimated $\beta$ will be upward biased (see here for an explanation of this kind of omitted variable bias).
Under the assumption that quality is fix over the sample period, you can use a fixed effects regression (see the lecture here for a quick introduction) to eliminate such unobserved time-invariant effects. This approach is still taken in more recent studies like Leigh (2009) who studies the effect of changes in the world economy on electoral outcomes. Besides the fixed effects he uses a conditional logit model to better account for the binary nature of the outcome variable but in your case this is not feasible because you only have 2 time periods (and then conditional logit will not be consistent as this method requires many periods). The fixed effects estimator for a binary dependent variable is nonetheless a good approximation of the actual effect so you shouldn't worry about such technicalities too much.
Other elections that have received a lot of attention are those in the Weimar Republic before the Nazis seized power. The usual question here is whether the economic depression led to voter radicalization (see Stögbauer, 2001, or de Bromhead et al, 2012). If you have GIS/geographic information on your states you can use a spatial fixed effects regression model like Stögbauer (2001). If you have parties that receive a zero vote share for many elections you can use the fixed effects maximum likelihood Tobit estimator that de Bromhead et al. (2012) use. Probably this goes beyond of what you wanted but the main message is: fixed effects models are the main tool for estimating such vote share regressions back in 1990 and nowadays. The nature of the data may change the estimator a little as in the last two papers but the main point is still to eliminate the unobservables that are fixed over time.
A last thing to consider: if you are interested in a variable's effect on an electoral swing you need not worry about unobserved time-invariant variables. What you still need to worry about is unobserved time-varying variables that are correlated with your regression variables. In this case, fixed effects cannot save you anymore and you will require an instrument. Whether you have a problem like this requires a bit of thought, depends on your data at hand and your research question. Given that you didn't tell us what variable you are interested in, this is the last piece of (general) advice I can provide in this answer.
Comment: Thanks for such a long and detailed answer! My specification is indeed fixed-effects, along with instrumental variables and a lot of hand-waving... I'm not so worried about the specification; I want to know if there is a 'received method' that these authors use to quantify the contribution of a variable to a swing: beta-coefficients, predict with regressor set to zero, or some other technique.
Comment: It depends on the unit of measurement of your explanatory variable how you interpret the coefficient. For instance, in the Levit paper the $\beta$ on campaign spending gives the percentage point increase in the probability of winning the election due to an extra dollar of campaign spending. Had he used the log of campaign spending, the $\beta$ would be the incr. in the probability of winning for a 1% increase in campaign spending. So it really depends on your explanatory variable. Otherwise the comment by ssdecontrol applies.
Comment: I went through the papers again and none of them standardize their coefficients or manipulate them in any other way except for transformations of the variables as was described in the previous comment.
Answer: Looking over a fair amount of papers, it seems that a common way to do this is to:

Fit whatever specification you have chosen
Set the variable of interest equal to a value motivated by theory. E.g. if you're looking at the effects of library growth on elections, then set growth equal to zero
Predict vote share with new value.
Contribution of variable = difference in predicted and actual result


How can I get/set an SqlAlchemy query value with the Column object

Question: To make some code more generic, I'm trying to retrieve/set values in a database row using the Column object.
For example, I'll have this setup:
col = MyTable.MyColumn
result = session.query(...).one()

Now I want to get/set the values, semantically like below:
cur_value = result[col]
result[col] = new_value

What is the correct syntax to use the Column object to get/set values in the result?
col will not be a static value, but dynamically taken from a map.
Comment: `cur_value =result.col, result.col = new_val db.session.commit()` als this may help https://stackoverflow.com/questions/9667138/how-to-update-sqlalchemy-row-entry
Answer: You can do this:
cur_value = getattr(result, col.key)
setattr(result, col.key, new_value)

Comment: The column objects are not strings, they are SqlAlchemy Column objects.  If I knew how to get the resulting name of that column, this would work.
Comment: @edA-qamort-ora-y use: `col = MyTable.MyColumn.key`... this will give you your column name.


define $ f:[0,1] \rightarrow [0,1]$ by $f(x ) = (2^k-1)/2^k ~for ~ x\in [ (2^{k-1}-1)/2^{k-1},~ (2^k-1)/2^k], k \geq 1$

Question: define $ f:[0,1] \rightarrow [0,1]$ by $f(x ) = (2^k-1)/2^k ~for ~ x\in [ (2^{k-1}-1)/2^{k-1},~ (2^k-1)/2^k], k \geq 1$
Then prove that:
$1/20<\int_{0}^{1} f(x) dx<211/213$
Attempt: I was able to prove $\int_{0}^{1} f(x) dx$ lies between $(1/2,1)$ by taking the smallest value and largest value of f but got no idea as to how to bring those kind of figures in the range.
f is piecewise constant. That makes computing the integral just summing the values of f multiplied by length of the corresponding interval. But, what is special about the numbers on both sides of the inequality then?
Thanks
Comment: Nothing at all, it's just for confusion.
Comment: $f$ is piecewise constant. That makes computing the integral just summing the values of $f$ multiplied by length of the corresponding interval.
Comment: yeah, I too followed the same procedure. But, what is special about the numbers on both sides of the inequality then?
Answer: The integral evaluates to a sum
$$\int_0^1 f(x) dx=\sum_{k=1}^\infty \frac{1}{2^k}\left(1-\frac{1}{2^k}\right) = \sum_{k=1}^\infty \frac{1}{2^k} - \sum_{k=1}^\infty \frac{1}{4^k} = \frac{2}{3}$$
Clearly, $1/20<2/3<211/213$.
Comment: Two-thirds, actually.
Comment: Yes, thank you. Edited.


Survey Method on Personal Issues

Question: A statistician friend of mine told me of an interesting technique used to obtain honest responses on surveys that dealt with sensitive issues. I recall the general gist of the method, but am wondering if anyone knows the details and if it is referenced anywhere.
The story was that the Florida AMA wanted to assess drug use among physicians. They sent out a questionnaire with one die. IIRC, the instructions were something along the lines of "roll the die. If you have ever taken drugs or get a 6, write down six; otherwise write down whatever number comes up". The idea being that if someone pulled the doctor's questionnaire and saw a six, he could say he didn't take drugs, he just happened to roll a six.
Comment: this very question hints at the "personal is political" motto of some years back, where everyone was stoned and everything was political, now everyone can still be stoned, but we try to downplay the political repercursions ;)
Answer: It's an old (1965) and well-documented method called randomized response.  It's used in various situations such as survey interviews or informal or ad hoc surveys conducted in classrooms or lecture halls. It's useful to think about the sample size that would be required to yield a given level of precision for the estimates one would obtain.
Comment: plus one should asume that the participants will actually give a truthful answer if given the random event that represents it. In the final analysis does not say much, one can still lie if one wants to. It is supposed to provide the capability to answer "almost anonymously" in a survey about issues that may be "criminalised", involve polarised or under-represented social issues, etc..

Survey Method on Personal Issues


Function completly monotonic

Question: I am reading the book of Püss (Evolutionary Integral Equations and Applications pg. 101) about subordination principle and it have one thing that i dont understand. If $c$ is a completly positive function how we can conclude that the function
$$h(\tau; \lambda)=\frac{e^{-\frac{\tau}{\widehat{c}(\lambda)}}}{\lambda \widehat{c}(\lambda)}$$
is completly monotonic in $(0,\infty)$ for all $\tau >0$? Here $\widehat{c}$ is the Laplace Transform of $c$. I am trying as follow...
Since $c$ is completly positive function, exist a function $b:[0,\infty)\to \mathbb{R}_+$ suth that $c=b'$ (derivative of $b$) such that $db$ is positive measure, that is, exist a creep function $k:(0,\infty)\to \mathbb{R}_+$ such that
$$\widehat{db}(\lambda)\widehat{dk}(\lambda)=\frac{1}{\lambda}$$
By the proposition 4.5 of Püss the function $e^{-\frac{\tau}{\widehat{c}(\lambda)}}$ is completly monotonic. We obtain too that $\frac{1}{\widehat{c}(\lambda)}$ is a Bernstein function, then $\widehat{c}(\lambda)$ is completly monotonic. But I cant conclude that $h$ is completly monotonic.
Answer: You forgot say that $c$ is completly continuous. But lets go, note that
$\widehat{db}(\lambda)=\widehat{c}(\lambda), \ \lambda>0$ so
$$\widehat{dk}(\lambda)=\frac{1}{\lambda \widehat{c}(\lambda)}, \ \lambda>0.$$
Since $k$ is a creep function, by the Bernstein Theorem
$$\lambda \mapsto \frac{1}{\lambda \widehat{c}(\lambda)}$$
is completly monotonic. But the completly monotonic functions are close to multiplication, hence $h(\tau;\lambda)$ is completly monotonic on $(0,\infty)$ for all $\tau>0$.

Function completly monotonic


Why is the sum of individual Spearman's rho squared less than 1 as opposed to Pearson's r in a synthetic example?

Question: A relatively low number of iid random vectors of a relatively high dimension (10,000) are added up together element wise:
$$\sum_{i=1}^{n}X_i=Y$$
where $dim(X_i)=dim(X_j)=dim(Y),\forall i,j$ and $dim(X_i)\gg n$
My modeling suggests that
$$E \left[\sum_{i=1}^{n}\rho_{X_i,Y}^2\right]<1$$
for the described case.
More generally
$$E \left[\sum_{i=1}^{n}\rho_{X_i,Y}^2\right]<E \left[\sum_{i=1}^{n}r_{X_i,Y}^2\right]$$
and
$$E \left[\sum_{i=1}^{n}r_{X_i,Y}^2\right]\ge1$$
where $\rho$ is Spearman's rank correlation coefficient, $r$ is Pearson correlation coefficient.
Computational assumptions
For computing $\rho_{X,Y}^2$ and $r_{X,Y}^2$ in R I use
cor(X, Y, method='spearman')^2

and
cor(X, Y, method='pearson')^2

respectively.
I also assume $X_i \sim \mathcal{N}(0, 1)$ but the following works for some other common distributions as well.
Fixed number of variables, fixed dimension, varied correlation function
Given just 3 of 10,000-vectors the distributions of Spearman's
$$\rho_{X_1,Y}^2+\rho_{X_2,Y}^2+\rho_{X_3,Y}^2$$
and Pearson's
$$r_{X_1,Y}^2+r_{X_2,Y}^2+r_{X_3,Y}^2$$
are as follows:

Fixed number of variables, varied dimension
Here's how the distributions of $\rho_{X_1,Y}^2+\rho_{X_2,Y}^2+\rho_{X_3,Y}^2$ and $r_{X_1,Y}^2+r_{X_2,Y}^2+r_{X_3,Y}^2$ look like next to each other for various dimensions. It appears as with the increase of the number of dimensions both distributions travel to the left but Pearson's stops at 1.0 while Spearman's continues to move below 1.0.

Fixed dimension, varied number of variables
When the number of variables increases, the Spearman's distribution seems to drift even further below 1.0 in the beginning and then comes back and exceeds 1.0 whereas Pearson's doesn't go below 1.0 at all and bounces back at the high number of variables just as Spearman's does:

Varied dimension, varied number of variables
A 3D plot to corroborate the above.

Questions

Is my modeling correct?
Why is the Pearson sum of squares always greater than Spearman's on average?
Why does the Pearson sum of squares never get below 1.0 on average whereas Spearman's does?

Here's my code:
library(ggplot2)
library(dplyr)
library(purrr)
library(plotly)

r_squared_sum <- function(num_vars, distrib_func, rsq_func, iterations=1000) {
    R_sq_sums <- c()
    for (i in 1:iterations) {
        data <- c()
        for (j in 1:num_vars) {
            data <- append(data, list(distrib_func()))
        }
        Y = Reduce(`+`, data)
        R_sq_sum = Reduce(`+`, lapply(data, partial(rsq_func, y=Y)))

        R_sq_sums <- append(R_sq_sums, R_sq_sum)
    }
    return(data.frame(r_squared_sum=R_sq_sums))
}

# ---------------------------------------------
# Fixed number of variables, fixed dimensions,
# varied correlation function, histogram
# ---------------------------------------------
data_spearman <- r_squared_sum(
    num_vars=3,
    distrib_func=partial(rnorm, 10000, 0, 1),
    rsq_func=function (x, y) cor(x, y, method='spearman')^2
)
data_spearman$method <- 'spearman'

data_pearson <- r_squared_sum(
    num_vars=3,
    distrib_func=partial(rnorm, 10000, 0, 1),
    rsq_func=function (x, y) cor(x, y, method='pearson')^2
)
data_pearson$method <- 'pearson'

ggplot(
    bind_rows(data_pearson, data_spearman),
    aes(x=r_squared_sum, fill=method, color=method)
) + geom_histogram(position="identity", alpha=0.8)

# -------------------------------------------------------
# Fixed number of variables, varied dimension, boxplots
# -------------------------------------------------------
dimension_range <- c(10, 100, 1000, 10000)
distrib_func_range <- lapply(
    dimension_range,
    partial,
    .f=partial(rnorm, mean=0, sd=1)
)
data_frames <- lapply(
    distrib_func_range,
    partial(
        r_squared_sum,
        num_vars=3,
        rsq_func=function (x, y) cor(x, y, method='spearman')^2
    )
)
for (i in 1:length(dimension_range)) {
    data_frames[[i]]$dimension <- as.character(dimension_range[i])
data_frames[[i]]$method <- 'spearman'
}
data_spearman <- bind_rows(data_frames)

data_frames <- lapply(
    distrib_func_range,
    partial(
        r_squared_sum,
        num_vars=3,
        rsq_func=function (x, y) cor(x, y, method='pearson')^2
    )
)
for (i in 1:length(dimension_range)) {
    data_frames[[i]]$dimension <- as.character(dimension_range[i])
data_frames[[i]]$method <- 'pearson'
}
data_pearson <- bind_rows(data_frames)

ggplot(
    bind_rows(data_pearson, data_spearman),
    aes(y=dimension, x=r_squared_sum, fill=method, color=method)
) + geom_boxplot()

# -------------------------------------------------------
# Fixed dimension, varied number of variables, boxplots
# -------------------------------------------------------
num_vars_range = c(10, 100, 1000)
data_frames <- lapply(
    num_vars_range,
    partial(
        r_squared_sum,
        distrib_func=partial(rnorm, 10000, 0, 1),
        rsq_func=function (x, y) cor(x, y, method='spearman')^2
    )
)
for (i in 1:length(num_vars_range)) {
    data_frames[[i]]$num_vars = as.character(num_vars_range[i])
data_frames[[i]]$method <- 'spearman'
}
data_spearman <- bind_rows(data_frames)

data_frames <- lapply(
    num_vars_range,
    partial(
        r_squared_sum,
        distrib_func=partial(rnorm, 10000, 0, 1),
        rsq_func=function (x, y) cor(x, y, method='pearson')^2
    )
)
for (i in 1:length(num_vars_range)) {
    data_frames[[i]]$num_vars = as.character(num_vars_range[i])
data_frames[[i]]$method <- 'pearson'
}
data_pearson <- bind_rows(data_frames)

ggplot(
    bind_rows(data_pearson, data_spearman),
    aes(y=num_vars, x=r_squared_sum, fill=method, color=method)
) + geom_boxplot()

# ---------------------------------------------------------
# Varied dimension, varied number of variables, 3D surfaces
# ---------------------------------------------------------
num_vars_range = seq(10, 100, by=10)
dimension_range <- seq(100, 1000, by=100)
distrib_func_range <- lapply(
    dimension_range,
    partial,
    .f=partial(rnorm, mean=0, sd=1)
)
nrow <- length(num_vars_range)
ncol <- length(distrib_func_range)

data_pearson <- matrix(data=NA, nrow=nrow, ncol=ncol)
for (i in 1:nrow) {
    for (j in 1:ncol) {
         r_squared_sums <- r_squared_sum(
            num_vars_range[[i]],
            distrib_func_range[[j]],
            function (x, y) cor(x, y, method='pearson')^2,
            iterations=100
        )
        data_pearson[i, j] <- mean(r_squared_sums$r_squared_sum)
    }
}

data_spearman <- matrix(data=NA, nrow=nrow, ncol=ncol)
for (i in 1:nrow) {
    for (j in 1:ncol) {
         r_squared_sums <- r_squared_sum(
            num_vars_range[[i]],
            distrib_func_range[[j]],
            function (x, y) cor(x, y, method='spearman')^2,
            iterations=100
        )
        data_spearman[i, j] <- mean(r_squared_sums$r_squared_sum)
    }
}

fig <- plot_ly(
    x=dimension_range,
    y=num_vars_range,
    showscale=FALSE,
    showlegend=TRUE,
    name='sums of r^2',
    width=700,
    height=500
) %>% layout(
    scene=list(
        xaxis=list(title="vectors"),
        yaxis=list(title="dimensions"),
        zaxis=list(title="mean sum r_sq")
    )
)
fig <- fig %>% add_surface(
    z=data_spearman,
    name='spearman',
    colorscale=list(c(0, 1), c("#00BFC4","#00BFC4")),
    hovertemplate=paste(' vectors: %{x}
', 'dimensions: %{y}
', 'mean sum r_sq: %{z}')
)
fig <- fig %>% add_surface(
    z=data_pearson,
    name='pearson',
    colorscale=list(c(0, 1), c("#F8776D","#F8776D")),
    hovertemplate=paste(' vectors: %{x}
', 'dimensions: %{y}
', 'mean sum r_sq: %{z}')
)
fig <- fig %>% add_surface(
    z=matrix(data=1, nrow=nrow, ncol=ncol),
    name='level',
    colorscale=list(c(0, 1), c("#696969","#696969")),
    hovertemplate=paste(' vectors: %{x}
', 'dimensions: %{y}
', 'mean sum r_sq: %{z}')
)
fig

I've implemented similar modeling in Python (my main language) with the same results.
Comment: That helps a little, thanks.  But how does your code assess this?  It seems you have hard-coded the value of $n=3.$  Unless I am mistaken in my reading of the code, all you're doing is using Monte-Carlo to estimate a quantity you could compute analytically; and you aren't studying what happens as $n$ grows large, as suggested by the language you use.
Comment: That code differs from what you have expressed mathematically.  You aren't evaluating the expectation of the squared correlation of two $3$-vectors: you are evaluating the squared correlation of two $10000$-vectors.  When "10000" is made less, the sum of squared correlations often exceeds $1.0.$
Comment: Your code does not appear to study the situation you have described mathematically.  Your equations concern the correlations between individual $X_i$ and the *final* $Y,$ which we ought to more explicitly write as $Y_n = X_1+X_2+\cdots+X_n.$ Is this what you really intend??
Comment: I'm not sure I completely understand your concern @whuber but I'm adding up a finite number (say $n$) of iid $X_i$s to get $Y$ and then try to measure Spearman's rho squared between each $X_i$ and $Y$ which ideally should all be the same and equal to $\frac{1}{n}$ each. Does it make it clearer?
Comment: Thanks for pointing out the caveats @whuber. I've rewritten the post.
Comment: Your results don't come anywhere near being correct for the situation you posit at the outset of the post: the expectations will substantially exceed $1.$  So, either your mathematical symbols do not express the problem correctly or there are bugs in the code.
Comment: Sorry for over-complicating the code and/or not being correct with my math. The idea boils down to the following in R:
`X1 = rnorm(10000, 0, 1); X2 = rnorm(10000, 0, 1); X3 = rnorm(10000, 0, 1); Y = X1 + X2 + X3; rsq_sum = cor(X1, Y, method='spearman')^2 + cor(X2, Y, method='spearman')^2 + cor(X3, Y, method='spearman')^2`.

`rsq_sum` is *always* below 1.0 while if I change the `method` to `'pearson'` it is at 1.0 on average. My question is why this is the case. Is it the implementation of `cor`? Is it my understanding of the math?
Comment: For me the sum of Pearson correlations approaches $\sqrt{n}$. What did I do differently from you (see my deleted question)?
Comment: I see now, you are using the square.
Comment: Thanks for pointing that out @whuber. I realise that, as you say, for vectors of lower dimensions the sum will grow beyond 1.0 quickly with the increase of the number of vectors themselves. I'd guess that for much higher dimensions the sample means would first deviate from 1.0 to the left and then with enough number of vectors the means would return back to 1.0 eventually exceeding it indefinitely as the number of vectors grows to infinity. So e.g. for 10000 dimensions it'd take a couple thousand $X_i$s for the mean to exceed 1.0. This appears to be true for both Pearson's and Spearman's.
Comment: So I guess my question can now be reformulated as follows: in the case of a sum of a relatively small number of relatively high dimensional random vectors why does the sum of individual $r^2$ behave as above?
Answer: TLDR; the setting (some sum of variables) where you observe a different Spearman's rank correlation and Pearson correlation can be greatly simplified.
If you use $X$ as normal distributed, then your question is equivalent to the case of bivariate normal distribution for which the Spearman's rank correlation is lower than the Pearson correlation.
For that case the Spearman's rank correlation can be computed as
$$r_{x,y} = \frac{6}{\pi} \sin^{-1}(\rho/2)$$
See the question: Finding correlation between CDF of two normal distributions
Simplifying the expression
The expectation of a sum equals the expectation of the components.
So we can also express this as a comparison for only one of the components. Your problem is equivalent to:
$$E[\rho_{X_1,Y}^2] < 1/n$$
$$E[r_{X_1,Y}^2] = 1/n$$
We can simplify this further. This pair of variables ${X_1,Y}$, with $Y$ as a sum of $n$ terms is effectively the correlation with a sum of two variables instead of $n$ variables.
$$Y = \sum_{i=1}^n X_i = X_1 + Z \\ \qquad \text{with } Z = \sum_{i=2}^n X_i \sim N(0,\sqrt{n-1})$$
The correlation of the distribution
The Pearson correlation of the distribution for the population we can compute exactly
$$r_{X_1,X_1+Z}^2 = \frac{\text{Cov}(X_1,X_1+Z)^2}{{\text{Var}(X_1)\text{Var}(X_1+Z)}}$$
and we can express the covariance of a sum as $$\text{Cov}(A,B+C) = \text{Cov}(A,C) + \text{Cov}(A,B)$$ which in your specific case becomes $$\text{Cov}(X_1,X_1+Z) = \text{Cov}(X_1,Z) + \text{Cov}(X_1,X_1) = 0 + \text{Var}(X_1)$$ If we fill that in then we get to
$$r_{X_1,X_1+Z}^2 = \frac{\text{Var}(X_1)^2}{{\text{Var}(X_1)\text{Var}(X_1+Z)}}$$
We also have
$$\text{Var}(X_1+Z) = \sum_{i=1}^{n}\text{Var}(X_i) = n \text{Var}(X_1)$$
where the latter equality is because all $\text{Var}(X_i)$ have the same value of the $X_i$ are similarly distributed.
And
$$r_{X_1,X_1+Z}^2 = \left(\frac{\text{Var}(X_1)}{\sqrt{\text{Var}(X_1)n \text{Var}(X_1)}}\right)^2 = \frac{1}{{n}}$$
This is the correlation for the population of $X_1$ and $Y$. The expectation of the sample correlation will approach towards this value of we increase the sample size $n$. It will not be exactly the same.
In the case that the $X_i$ are normal distributed then the exact distribution for the sample correlation is this monster. I could not easily find computations of the expectation value of the observed squared correlation coefficient, so I did a simple computation and it might be that we have the following relationship
$$E[\hat{r}^2] = \frac{n}{n-1} r^2$$
where $n$ is the sample size. The computation below is a simulation that verifies that this relationship might be true.
    set.seed(1)

    p = 3
    n = 10

    corsample = function() {
       X = matrix(rnorm(p*n),n)
       X1 = X[,1]
       Y = rowSums(X)
       cor(X1,Y)^2
    }

    rho = replicate(10^3,corsample())
    mean(rho)
    (1/3)*n/(n-1)

A way to approximate the expectation value of the sample correlation for large samples.
The sample Pearson correlation is the square of the following sum
$$\hat{r}_{x_1,y}^2 = \left(\frac{\frac{1}{n}\sum_{i=1}^{n} (x_i-\bar{x})(y_i-\bar{y}) }{s_{x_1}s_z} \right)^2 $$
It is not very rigorous but when $n$ is very large then we might approximate the expectation of this product as a product of expectations and also the sample deviation and sample mean can be approximated by the population deviation and population mean, $s_{x_1} \approx \sigma_{x_1}$, $s_{y}^2 \approx \sigma_{y}^2$, $\bar{x} = \mu_x$ and $\bar{y} = \mu_y$. For large sample $n$ this seems reasonable and we may expect any bias from this approximation to reduce to zero. Let's also assume without loss of generality that $\mu_x=\mu_y=0$
So
$$E[\hat{r}_{x_1,y}^2] \approx \left(\frac{\frac{1}{n}\sum_{i=1}^{n} E[x_iy_i] }{\sigma_{x_1}\sigma_y} \right)^2 = \frac{E[x^2]^2}{n\sigma_x^4} = \frac{1}{n}$$
In a similar fashion we should be able to obtain the expectation value for the Spearman correlation, but now we will work with a transformed variable. We are not looking for the correlation of the variables $x$ and $y$ but instead of the variables $rank(x)$ and $rank(y)$. For large $n$ these ranks will get closer to the quantiles of the distribution and we use these to approximate the limit for large $n$.
In the code below is a computation that uses the distribution of the ranks and a computational integration. For three variables the integration results in
### Spearman
[1] 0.3127959
### Pearson
[1] 0.3333071

And your expectation for the sum of the case with three variables is expected to approach $0.9384$.
We could say intuitively that the difference between the two correlations is in this transformation. With the Pearson correlation we compute correlation of $x$ and $y$ with the Spearman correlation we compute the correlation of transformed versions of $x$ and $y$.
So even if you would be able to increase your sample to a very large amount, then the expectation value for the Spearman's correlation will be lower in the case of a normal distribution of $X$.

set.seed(1)
n = 3

x_rank = function(x) {
   p = pnorm(x,0,1)-0.5
}

y_rank = function(x) {
   p = pnorm(x,0,sqrt(n))-0.5
}

Sigma = matrix(c(1,1,1,n),2)
m = solve(Sigma)/2
const = det(2*pi*Sigma)^(-0.5)

mvdnorm = function(x,y, Sigma) {
    const*exp(-m[1,1]*x^2 - m[2,2]*y^2 - 2*m[1,2]*x*y)
}

dx = 0.1
range = seq(-8,8,dx)

xy = 0
rxry = 0
for (x in range) {
   for (y in range) {
       rxry = rxry + mvdnorm(x,y,Sigma) *x_rank(x)*y_rank(y)*dx^2
       xy = xy + mvdnorm(x,y,Sigma)*x*y*dx^2
   }
}

var_rx = sum(dnorm(range,0,1)*x_rank(range)^2)*dx
var_ry = sum(dnorm(range,0,sqrt(n))*y_rank(range)^2)*dx
var_x = sum(dnorm(range,0,1)*range^2)*dx
var_y = sum(dnorm(range,0,sqrt(n))*range^2)*dx

rxry^2/var_rx/var_ry
xy^2/var_x/var_y

rxry^2/var_rx/var_ry*n

v = MASS::mvrnorm(10^5,c(0,0),Sigma)

plot(v[,1],v[,2], pch = 19, col = rgb(0,0,0,0.03), cex = 0.5, xlab = "X", ylab = "Y", main = "r^2 = 0.3333")
plot(x_rank(v[,1]),y_rank(v[,2]), pch = 19, col = rgb(0,0,0,0.03), cex = 0.5, xlab = "quantile X", ylab = "quantile Y", main = "rho^2 = 0.3128")

cor(x_rank(v[,1]),y_rank(v[,2]))^2
cor((v[,1]),(v[,2]))^2

Comment: Thanks for taking the effort of answering this mess of a question. Couple notes. The [linked post](https://math.stackexchange.com/questions/2954999/finding-correlation-between-cdf-of-two-normal-distributions) seems to be about Pearson correlation of CDFs, not ranks like Spearman. The expression $E[\hat{r}^2] = \frac{n}{n-1} r^2$ is known as the [Bessel's correction](https://en.wikipedia.org/wiki/Bessel%27s_correction) and your assumption is correct.
Comment: What really bugs me about this is that the above inequality between Spearman's and Pearson's seem to persist for some other distributions like uniform and exponential. I'm wondering whether there's a more general phenomenon going on between the two.
Comment: It doesn't seem that there can be a nice formula for the difference (factor?) between the two correlations especially one that works across all (can this be proven?) possible distributions. So I'm curious whether there's a way for the Spearman's to be brought in line with the behavior of Pearson's numerically. E.g. can we simply scale the whatever value of $\sum_{i=1}^{n}\rho_{X_i,Y}^2$ to 1.0 and apply the same factor to all the $\rho_{X_i,Y}^2$ and then claim the resulting $C\cdot \rho_{X_i,Y}^2$ to be the respective shares of the total variance? Would this be theoretically sound?
Comment: @ayorgo I don't think that there is a general formula. Take for instance a sum of three heavy tailed distributions like a t-distribution with $\nu=3$ `set.seed(1); n = 10^6; d = 3; X = matrix(rt(3*10^6,3),n); Y = rowSums(X); X1 = X[,1]; cor(X1,Y)^2*3; cor(X1,Y, method ="spearman")^2*3` now the squared Spearman's rank correlation is around 0.86, a bit lower.
Comment: @ayorgo Responses to your first comment (1) In the limit of a large sample size, the Spearman's rank correlation will approach the Pearson correlation of the quantiles (the CDFs). (2) Indeed it looks like some sort of Bessel's correction. But, I would not consider it the same. Bessel's correction is for the estimate of the variance (and could be generalized to covariance) and it is not a correction for the correlation.
Comment: @ayorgo There's no equality with a fixed constant, but still an open question is the inequality. Let $X$ and $Y$ be two independent variables with variance $\sigma_X^2$ and $\sigma_Y^2$, then the square of the correlation between $X$ and $X+Y$ is$$r_{X,X+Y}^2=\frac{\sigma_X^4}{\sigma_X^4+\sigma_X^2\sigma_Y^2}$$ My intuition tells me that this correlation between $X$ and $X+Y$ must be a maximum. When we apply a formula, then information gets lost and the correlation should decrease. A correlation between $f(X)$ and $g(X+Y)$ can't be larger than a correlation between the original $X$ and $X+Y$.
Comment: I have turned this into a more general question https://stats.stackexchange.com/questions/581143/can-there-be-functions-g-and-f-such-that-rho-fx-gxy2-rho-x-xy


Showing $A^2 - 3A^{147} + 2I = \begin{pmatrix} 9 & -9\\ 3 & -3\end{pmatrix}$.

Question: Let $ A = $$ \begin{pmatrix} 9 & -9\\ 3 & -3\end{pmatrix} $$ $. Show
$$A^2 - 3A^{147} + 2I =  \begin{pmatrix} 9 & -9\\ 3 & -3\end{pmatrix}$$
Comment: The given equation is not fulfilled.
Comment: @Shaun Alright I will keep induction constantly in my mind from now on seeing it's needed in a lot of exercises. If you mean how I like the site so far, it's great but I still need to get used to LaTeX.
Comment: I'd start by taking powers of $A$ to see if there's a pattern. If I spot one, I'd (most likely) prove it's there by induction then use it to calculate $A^{147}$. The solution should then be straightforward.
Comment: Also, please share your thoughts so far :)
Comment: Actually, I meant your thoughts on the problem so far. Some people downvote questions that don't have any signs of effort on the part of the person asking. Regardless, I'm glad you like it so far. [Here's a MathJax tutorial](http://meta.math.stackexchange.com/questions/5020/mathjax-basic-tutorial-and-quick-reference) $\ddot\smile$
Comment: I'm not very comfortable showing my math thought processes to the public so far :P But yeah when I first saw the problem I was trying to find a connection between the powers. However, I completely ignored the given matrix at the start.
Answer: Note that you can write $A$ as $A=3B$ where $$B=\begin{pmatrix} 3 & -3\\ 1 & -1\end{pmatrix}.$$
Now, you can compute easily that $B^2=2B$ and so by induction you'll have $B^n=2^{n-1}B$.
Therefore $A^n=(3B)^n=3^n\times2^{n-1}B=6^{n-1}A$. I am sure from this point you can find what you wanted, can't you?
Comment: Yes I've solved it a while ago but I can't accept a comment as an answer because it doesn't let me, so I'll just accept yours.
Answer: Once you compute the characteristic polynomial you get $\lambda^2-6\lambda=0$. Thus $A^2-6A=O$.
Therefore $A^{n}=6^{n-1}A$. See if you can proceed from here.
Comment: Oh I can't use eigenvalues to compute this yet :) Not because I don't know them but because I'm trying to solve some exercises which are before the chapter of eigenvalues/eigenvectors.
Comment: In that case just compute $A^2$ and see what happens.It will be easy for you to spot that $A^2=6A$.


Convergence for a Random Normal Process defined by Recursion

Question: A question from my Random Processes exams:
Let $ W_0,W_1,W_2,...$ be a sequence of independent Gaussian random variables with Mean 0 and Variance $ \sigma ^ 2 > 0 $.
Define the sequence $ (X_n : n \geq 0) $ recursively by $ X_0 = 0 $ and $ X_{k+1} = \frac{X_k + W_k}{2}$.
Determine whether the sequence $(X_n)$ converges in each
of the four senses, namely mean square, almost sure, probability and distribution.
My work:
$$ X_1 = \frac{W_0}{2}$$
$$ X_2 = \frac{W_1 + X_1}{2} = \frac{W_0}{2^2} + \frac{W_1}{2}$$
Thus,
$$X_n = \sum_{i=0}^{n-1}{\frac{W_k}{2^{n-k}}}$$
Now, how to utilise the fact that these are $\pmb{i.i.d. RV's}$ and get to different convergences?
Sum of Normal $\pmb{i.i.d. RV's}$ Normal.
Thus, $\mu(X_n) = 0$
Also,
$$Var(X_n) = Var\bigl(\sum_{i=0}^{n-1}{\frac{W_k}{2^{n-k}}}\bigr)=\bigl(\sum_{i=0}^{n-1}{\frac{Var(W_k)}{4^{n-k}}}\bigr)$$
$$Var(X_n) = \sigma ^ 2 \sum_{i=0}^{n-1}{\frac{1}{4^{n-k}}} = \sigma ^ 2 \bigl( \frac{\frac{1}{4}(1-\frac{1}{4^n})}{1-\frac{1}{4}} \bigr)$$
$$\lim_{n\to\infty} Var(X_n) = \frac{\sigma ^ 2}{3}
$$
How, to move forward with this?
Comment: Can you find the distribution and mean and variance of $X_n$?  For example $X_1$ has a normal distribution with mean $0$ and variance $\frac{\sigma^2}{4}$
Comment: I think you made a mistake in your variance computation. If you know the mean and variance of each of the normal rv $X_n$, then by Lévy's convergence theorem you can use the pointwise convergence of their characteristic functions to show that the sequence converges in distribution to a normal rv $X$. Then you can compute $E[(X-X_n)^2]$ to show convergence in quadratic mean, which implies convergence in probability.
Comment: Do you know what the distribution for a sum of iid normal rv is?
Comment: Thanks @lonzaleggiera. Made the correction.
Comment: 1) Almost Sure Convergence: $$\Bbb P \Bigl (\ \omega \ | \lim_{n\to \infty} X_n(\omega) = X(\omega) \Bigr) = 1$$

2) Convergence in Probability: $$\lim_{n\to \infty} \Bbb P \Bigl ( |X_n - X| > \epsilon \Bigr) = 0$$

3) Convergence in Mean Squared Sense: $$ \lim_{n\to \infty} \Bbb E \bigl [ (X_n-X)^2 \bigr] = 0$$

4. Convergence in Distribution: $$\lim_{n\to\infty} F_{X_n}(x) = F_X(x)$$
Comment: @Joe Could you please elaborate how to move forward in proving $E[(X-X_n)^2]$ convergence. Because $E[X_n ^2] = \frac{\sigma ^ 2 }{3}$ and here X=$\mu(X_n)$=0
Comment: By the same argument above, $X - X_n$ is a normal rv with a variance that you can find using the same computations. Instead of starting the sum at $i=0$, the sum would start at $i=n$.
Comment: But for $Var(X-X_n) = Var(X) - Var(X_n)$. They need to be independent. How could that be proven? Also, X~$N(0,\frac{\sigma ^ 2}{3})$ right??
Comment: They definitely are not independent. I originally thought your sequence was $X_n = \sum_i^{n-1} W_i/2^i$. Now that I see its $X_n = \sum_i^{n-1} W_i/2^{n-i}$, I don't think it does converge in qm, but I'm not sure without writing it out. I'd trying writing out the formula for $X_m-X_n$ for $m>n$, in terms of $W_i$, and taking the limit as $m\to \infty$ to see what the variance of $X-X_n$ is.
Answer: You've already shown that $\ \big(X_n\big)\ $ converges in distribution to $\ \mathscr{N}\left(0,\frac{\sigma^2}{3}\right)\ $.  Since almost sure convergence and convergence in mean square both imply convergence in probability, then if you can show that $\ \big(X_n\big)\ $ doesn't converge in probability, as I believe you can, then it can't converge almost surely or in mean square either.
Suppose $\ \big(X_n\big)\ $ converges in probability to $\ X\ $.  Then, for any positive integers, $\ m,n\ $, and any positive real $\ \epsilon\ $,
\begin{align}
P\big(|X-X_m|>\epsilon\big)&=P\big(|X_n-X_m-(X_n-X)|>\epsilon\big)\\
&\ge P\big(|X_n-X_m|-|X_n-X|>\epsilon\big)\\
&\ge P\big(\{|X_n-X_m|>2\epsilon\}\setminus \{|X_n-X|>\epsilon\}\big)\\
&\ge P\big(|X_n-X_m|>2\epsilon)-P\big(|X_n-X|>\epsilon\big)\ .
\end{align}
Since $\ W_0, W_1,\dots, W_n, \dots\ $ are independent normal, then $\ X_n-X_m\ $ is normal with mean $0$ and variance
\begin{align}
&\hspace{1em}Var\big(X_n\big)+Var\big(X_m\big)-2E(X_nX_m)\\
&=\frac{\sigma^2\left(1-4^{-n}\right)}{3}+\frac{\sigma^2\left(1-4^{-m}\right)}{3}-\frac{\sigma^2\left(1-4^{-\min(m,n)}\right)}{2^{m+n-1}3}\\\
&\ge\frac{\sigma^2}{3}
\end{align}
for all $\ m, n\ $.
Under the supposition that $\ \big(X_n\big)\ $ converges in probability to $\ X\ $, there must exist a positive integer $\ N\ $ such that $\ P\big(|X_N-X|>\frac{\sigma}{2\sqrt{3}}\big)<1-\mathscr{N}(0,1)(1)\ $. Then for any$\ m\ $, we have from above that
\begin{align}
P\bigg(|X-X_m|>\frac{\sigma}{2\sqrt{3}}\bigg)&\ge P\left(|X_N-X_m|>\frac{\sigma}{\sqrt{3}}\right)-P\left(|X_N-X|>\frac{\sigma}{2\sqrt{3}}\right)\\
&\ge P\left(|X_N-X_m|>\sqrt{Var\big(X_N-X_m\big)}\right)\\
&\hspace{2em}-\big(1-\mathscr{N}(0,1)(1)\big)\\
&=1-\mathscr{N}(0,1)(1)\ ,
\end{align}
from which it follows that $\ \big(X_n\big)\ $ does not converge in probability to $\ X\ $, thus contradicting the original assumption that it did.


Can infinite $T_0$ space have less open sets than points?

Question: It is pretty easy to check, that if $X$ is $T_1$, then it has to have at least as many open sets as points, because every singleton would be closed. On the other hand, if we don't assume anything about $X$, we could choose indiscrete topology and get positive answer.
In the middle, we have $T_0$. My intuition says, that it shouldn't be possible, but I don't know why.
Answer: The proposed by you solution for $T_1$ case works almost the same in $T_0$ case, except you should not focus on singletons, but rather on closures of singletons.
Let $(X,\tau)$ be a $T_0$ topological space. Note that two points $x,y$ are topologically indistinguishable if and only if $\overline{\{x\}}=\overline{\{y\}}$. Thus in $T_0$ spaces $x=y$ if and only if $\overline{\{x\}}=\overline{\{y\}}$. This shows that
$$f:X\to \tau$$
$$f(x)=X\backslash\overline{\{x\}}$$
is an injection, regardless of whether $X$ is infinite or not.
Answer: Remark: This is correct but unnecessarily complicated.
No, it cannot. Suppose $X$ is an infinite set and $(X,\mathcal{T})$ is a $T_0$ topological space. Note that $\mathcal{T}$ must be infinite or otherwise this space would have at least some pairs of distinct points that are topologically indistinguishable.
Given any infinite set $A$, let $S(A)$ denote the set of unordered pairs $\{a,b\}$ of distinct points in $A$. Since $A$ is infinite, we have $|A\times A|=|A|$. If $B=A\times A\backslash \{(a,a)\mid a\in A\}$, then there is a canonical surjection $B\to S(A)$ taking an ordered pair $(a,b)$ of distinct points to the unordered pair $\{a,b\}$. One can then extend this to a surjection $A\times A\to S(A)$ to get $|S(A)|\leq |A|$. It's not hard to construct an injection $A\to S(A)$ and so we have $|A|=|S(A)|$.
Applying the previous paragraph to our situation, we have $|X|=|S(X)|$ and $|\mathcal{T}|=|S(\mathcal{T})|$. Thus, to prove that $|X|\leq |\mathcal{T}|$ it suffices to produce an injection $S(X)\to S(\mathcal{T})$.
Define $f:S(X)\to S(\mathcal{T})$ by $f(\{x,y\})=\{X\backslash\overline{x},X\backslash\overline{y}\}$ where $\overline{x}$ denotes the closure of the single-point set $\{x\}$. We'll check some things about $f$ using the following key fact about $T_0$ spaces.
Fact: Since $X$ is $T_0$, if we are given $x,y\in X$, then we have $x=y$ if and only if $\overline{x}=\overline{y}$.
Claim 1: $f$ is well-defined.
Pf. If $x\neq y$, then because $X$ is $T_0$, $\overline{x}\neq \overline{y}$. Thus $X\backslash\overline{x}\neq X\backslash\overline{y}$ showing the pair $\{X\backslash\overline{x},X\backslash\overline{y}\}$ is an element of $S(\mathcal{T})$. We conclude that $f$ is well-defined.
Claim 2: $f$ is injective.
Pf. Suppose $a\neq b$, $x\neq y$ and $f(\{a,b\})=f(\{x,y\})$. Then $\{X\backslash\overline{a},X\backslash\overline{b}\}=\{X\backslash\overline{x},X\backslash\overline{y}\}$ are equal 2-element sets. If $X\backslash\overline{a}=X\backslash\overline{x}$ and $X\backslash\overline{b}=X\backslash\overline{y}$, then $\overline{a}=\overline{x}$ and $\overline{b}=\overline{y}$, giving $a=x$ and $b=y$. If $X\backslash\overline{a}=X\backslash\overline{y}$ and $X\backslash\overline{b}=X\backslash\overline{x}$, the same reasoning gives $a=y$ and $b=x$. Either way, $\{a,b\}=\{x,y\}$. This proves $f$ is injective.
Comment: This is correct but ended up way more complicatd than it needed to be. The other answer should be accepted. The key "fact" is what really gets the job done in the end.
Comment: Wow, thank you! I'm not used to work with $T_0$ spaces, and I didn't think about the fact you used. With it, it is all immediately clear.


How can I loop a sound without using 'clip' in java before writing it into a file

Question: First of all thanks everyone for helping me with my previous questions.
In the following code I am taking two frequencies alternatively and writing them into a .wav format, to run it in my Windows Media Player, for a specific time given by the user.
What I want is to understand how to loop those frequencies to run alternatively for that specified time, like the Ambulance's siren, and in my program both frequencies are getting played just once, alternatively.
For example, if I am specifying the time as 10 seconds, then both the frequencies run for 5 seconds each at a stretch. But what I want is that the 1st frequency to run for a second or two seconds(as the user specifies) and then the 2nd frequency to run for that similar second and then again the 1st frequency and it should keep on going till the specified time.
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.util.Scanner;
import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.LineUnavailableException;

public class AudioWrite2New {

    public static void main(String[] args) throws IOException, InterruptedException, LineUnavailableException {

        Scanner in = new Scanner(System.in);
        final double SAMPLING_RATE = 44100;             // Audio sampling rate
        int time = in.nextInt();                        //Time specified by user in seconds
//        int time2 = in.nextByte();
        int frequency1 = in.nextInt();                   //Frequency1 specified by the user in hz
        int frequency2 = in.nextInt();                   //Frequency2 specified by the user in hz

        float buffer[] = new float[(int) (time/2 * SAMPLING_RATE)];   //Size of buffer[], which in case of 10 seconds is 441000
        float buffer1[] = new float[(int) (time/2 * SAMPLING_RATE)];   //Size of buffer1[], which in case of 10 seconds is 441000

        for (int sample = 0; sample < buffer.length; sample++) {
            double cycle = sample / SAMPLING_RATE;                  //Fraction of cycle between samples
            buffer[sample] = (float) (Math.sin(2 * Math.PI * frequency1 * cycle));
            //buffer1[sample] = (float) (Math.sin(2 * Math.PI * frequency2 * cycle));
        }
        for (int sample = 0; sample < buffer1.length; sample++) {
            double cycle = sample / SAMPLING_RATE;                  //Fraction of cycle between samples
            //buffer[sample] = (float) (Math.sin(2 * Math.PI * frequency1 * cycle));
            buffer1[sample] = (float) (Math.sin(2 * Math.PI * frequency2 * cycle));
        }
        //System.out.println(buffer[1]);
        byte byteBuffer[] = new byte[buffer.length * 2];            //Size of byteBuffer, in this case 882000
        byte byteBuffer1[] = new byte[buffer1.length * 2];            //Size of byteBuffer, in this case 882000

        int count = 0;
        for (int i = 0; i < byteBuffer.length; i++) {
            final int x = (int) (buffer[count++] * Short.MAX_VALUE);
            byteBuffer[i++] = (byte) x;
            byteBuffer[i] = (byte) (x / 256);
        }

        count = 0;
        for (int i = 0; i < byteBuffer1.length; i++) {
            final int x = (int) (buffer1[count++] * Short.MAX_VALUE);
            byteBuffer1[i++] = (byte) x;
            byteBuffer1[i] = (byte) (x / 256);
        }

        //For merging the two frequencies
        byte[] merge = new byte[byteBuffer.length + byteBuffer1.length];
        System.arraycopy(byteBuffer, 0, merge, 0, byteBuffer.length);
        System.arraycopy(byteBuffer1, 0, merge, byteBuffer.length, byteBuffer1.length);

        File out = new File("E:/RecordAudio17.wav"); //The path where user want the file data to be written

        //Construct an audio format, using 44100hz sampling rate, 16 bit samples, mono, and big
        // endian byte ordering
        AudioFormat format = new AudioFormat((float) SAMPLING_RATE, 16, 1, true, false);

        // It uses bytebuffer as its buffer array that contains bytes that may be read from the stream.
        ByteArrayInputStream bais = new ByteArrayInputStream(merge);

        //Constructs an audio input stream that has the requested format and length in sample frames, using audio data
        //from the specified input stream.
        AudioInputStream audioInputStream = new AudioInputStream(bais, format, buffer1.length + buffer.length);

        //Writes a stream of bytes representing an audio file of the specified file type to the external file provided.
        AudioSystem.write(audioInputStream, AudioFileFormat.Type.WAVE, out);

        audioInputStream.close();       //Closes this audio input stream
    }
}

And as I am quite new in Java and JavaSounds, therefore sometimes I might ask few foolish or irrelevant questions. So please bear with me as that is the only way in which I can learn.
Thanks.
Comment: you know all these complicated things and you dont know how to put one Statement and one Statement together - that means you dont know Programming - refer to these guys that helped you http://stackoverflow.com/questions/40586715/play-2-different-frequencies-alternatively-in-java
Comment: @gpasch Before posting my question I have tried to solve it, but I was not getting my desired output. Sometimes it don't strike. And yes I am quite new in programming so maybe I am not that good at it. But I am not a a quitter and I solved it eventually.
Thanks anyways.
Answer: I resolved it. Although there is one minor part that I need to handle. I will update it as soon as I am done with it.
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.IOException;
import java.util.Scanner;
import javax.sound.sampled.AudioFileFormat;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.LineUnavailableException;

public class AudioWrite2New {

    public static void main(String[] args) throws IOException, InterruptedException, LineUnavailableException {

        Scanner in = new Scanner(System.in);
        final double SAMPLING_RATE = 44100;             // Audio sampling rate
        int time = in.nextInt();                        //Time specified by user in milliseconds
        int time2 = in.nextByte();
        int frequency1 = in.nextInt();                   //Frequency1 specified by the user in hz
        int frequency2 = in.nextInt();                   //Frequency2 specified by the user in hz

        float buffer[] = new float[((int) (time * SAMPLING_RATE))/1000];   //Size of buffer[], which in case of 10 seconds is 441000
        float buffer1[] = new float[((int) (time * SAMPLING_RATE))/1000];   //Size of buffer1[], which in case of 10 seconds is 441000

        //for (int a = 1; a <= time2 / 2; a++) {
        for (int sample = 0; sample < buffer.length; sample++) {
            double cycle = sample / SAMPLING_RATE;                  //Fraction of cycle between samples
            buffer[sample] = (float) (Math.sin(2 * Math.PI * frequency1 * cycle));
            //buffer1[sample] = (float) (Math.sin(2 * Math.PI * frequency2 * cycle));
        }
        for (int sample = 0; sample < buffer1.length; sample++) {
            double cycle = sample / SAMPLING_RATE;                  //Fraction of cycle between samples
            //buffer[sample] = (float) (Math.sin(2 * Math.PI * frequency1 * cycle));
            buffer1[sample] = (float) (Math.sin(2 * Math.PI * frequency2 * cycle));
        }
        //System.out.println(buffer[1]);
        byte byteBuffer[] = new byte[buffer.length * 2];            //Size of byteBuffer, in this case 882000
        byte byteBuffer1[] = new byte[buffer1.length * 2];            //Size of byteBuffer, in this case 882000

        int count = 0;
        for (int i = 0; i < byteBuffer.length; i++) {
            final int x = (int) (buffer[count++] * Short.MAX_VALUE);
            byteBuffer[i++] = (byte) x;
            byteBuffer[i] = (byte) (x / 256);
        }

        count = 0;
        for (int i = 0; i < byteBuffer1.length; i++) {
            final int x = (int) (buffer1[count++] * Short.MAX_VALUE);
            byteBuffer1[i++] = (byte) x;
            byteBuffer1[i] = (byte) (x / 256);
        }

        int iterations = (1000*time2)/(2*time);
        byte[] merge = new byte[iterations*(byteBuffer.length + byteBuffer1.length)];
        for (int i = 0; i<iterations; i++)
        {
            //arraycopy(Object src, int srcPos, Object dest, int destPos, int length)
            //For merging the two frequencies
            System.arraycopy(byteBuffer, 0, merge, 0+i*(byteBuffer.length + byteBuffer1.length), byteBuffer.length);
            System.arraycopy(byteBuffer1, 0, merge, byteBuffer.length+i*(byteBuffer.length + byteBuffer1.length), byteBuffer1.length);
        }

        File out = new File("E:/RecordAudio17.wav"); //The path where user want the file data to be written

        //Construct an audio format, using 44100hz sampling rate, 16 bit samples, mono, and big
        // endian byte ordering
        AudioFormat format = new AudioFormat((float) SAMPLING_RATE, 16, 1, true, false);

        // It uses bytebuffer as its buffer array that contains bytes that may be read from the stream.
        ByteArrayInputStream bais = new ByteArrayInputStream(merge);

        //Constructs an audio input stream that has the requested format and length in sample frames, using audio data
        //from the specified input stream.
        AudioInputStream audioInputStream = new AudioInputStream(bais, format, (buffer1.length + buffer.length) * (time2/4));

        //Writes a stream of bytes representing an audio file of the specified file type to the external file provided.
        AudioSystem.write(audioInputStream, AudioFileFormat.Type.WAVE, out);

        //}
        //audioInputStream.close();       //Closes this audio input stream
    }
}


Placebo test for the validity of difference-in-differences analysis results for limited data period

Question: I am doing an analysis of a strategy implemented by a four-year-old corporate company and this implementation was given in different cities which all of its employees were received the treatment or not in 2016-2019. The strategy change date is 2017.
I apply the quasi-experimental method (difference-in-differences) and I am concerned about the validity of my results. Therefore, for the validity check of my difference-in-differences analysis, I am planning to do placebo tests with fake treatment dates before and after the actual date of treatment. However, due to the company was established in 2016, there is no data before 2016. Although I can use "2018" as a fake treatment year, I am wondering how can I do a placebo test before the actual date with a single period (just 2016) data and do I need to use any fake treatment year before the actual date? Is it enough to use only 2018 as a fake treatment date for placebo test to check the validity of the actual treatment date (2017) results?
Any alternative suggestions for the difference in differences placebo test or robustness of the difference-in-differences test results would be much appreciated.
Comment: Just so I am clear, you have yearly data from 2016 to 2019 (4 years) and your data is at the employee level/city level? If so, is your data panel or repeated cross-sections?
Comment: It is panel data. There is a single period "2016"  (pre-treatment data) before the treatment date "2017".
Comment: So it's a 'company-year' panel? Some companies receive the treatment and others do not. Or do you have $i$ individuals also nested within those companies?
Answer: It is often difficult to buttress claims of trend equivalence before the onset of treatment with only one pretreatment period. Serial observations are often required to demonstrate parallel trends, which (in my opinion) requires at least three pretreatment periods to support it.

Although I can use "2018" as a fake treatment year, I am wondering how can I do a placebo test before the actual date with a single period (just 2016) data and do I need to use any fake treatment year before the actual date?

Applied researchers often use "event study" estimates to support claims of common group trends. They may, for instance, interact a treatment indicator with a series of time dummies for all years before and after the onset of treatment, leaving out the period immediately before treatment, or some more distant pre-event period, as a reference (baseline). You cannot possibly do this with one pre-event time period. Researchers often do this to test the validity of the difference-in-differences design. Put more simply, you shouldn't be capturing significant treatment effects in the pretreatment epoch. Treatment effects should concentrate around the exposure period. Any strong nonzero effects in the periods leading up to the treatment could be interpreted as selection bias.
You indicated in your question that you wish to treat a post-treatment year (e.g., 2018) as a placebo treatment year. Is the treatment only in effect for one year before it is officially removed? I don't know the intimate details of your study, but I would advise against using 2018 as a fake treatment year. Once treatment is removed, are there any lingering (persistent) effects beyond conclusion of the intervention/treatment? If you're going to use 2018 a placebo treatment year, then your treatment should have no impact on your outcome(s) in that year.

Is it enough to use only 2018 as a fake treatment date for placebo test to check the validity of the actual treatment date (2017) results?

Perhaps. If treatment reverts back to some pretreatment level and there are absolutely no lingering effects that you need to account for, then this could support the validity of your approach. In other words, it shows treatment effects concentrating around your exposure period.
You could also investigate fake outcomes. Suppose you are interested in the effects of your intervention across multiple outcome measures. Try to find outcome variables that would, in theory, not be impacted by your treatment. It is comforting to find insignificant treatment effects with outcomes that do not (should not) vary with treatment exposure. This could be used as a robustness check.
In sum, unless you have some strong theoretical basis for assuming trends do not deviate drastically in the periods immediately before the onset of treatment, it will be difficult to support claims of common trends across treatment and control groups.
The best course of action would be to acquire more pre- or post-event data. You could still do a difference-in-differences design, just be prepared to answer these questions.


Lista em Python de números positivos e negativos

Question: Como faço para que o usuário digitar números positivos e negativos na mesma linha e a lista exibi-los em ordem do menor para o maior, de preferência que não use loop?
Exemplo de números digitados
[1, 2, 3, 4, 5, -1, -2, -3, -4, -5]

Exemplo de como a saída deve ser:
[-5, -4, -3, -2, -1, 1, 2, 3, 4, 5]

Answer: Com o método sort(). Exemplo:
lista = []
i = 0
while i < 6:
    lista.append(int(input()))
    i += 1
lista.sort()
print(lista)

Para inverter a ordem é só fazer:
lista.sort(reverse = True)

sem o  while:
lista = []
lista.append(int(input()))
lista.append(int(input()))
lista.append(int(input()))
lista.append(int(input()))
lista.sort()
print(lista)

inserir todos os itens de uma vez:
lista = input().split()
lista.sort()
print(lista)

Nesse último caso a lista vai ser do tipo string pq o método split() só pode ser usado em strings, mas caso você precise que seja de outro tipo, basta converter depois. Lembrando que os valores devem estar separados por espaços.
Comment: Fala Alex, como a Laís explicou está certinho, porém ficaria sem lógica fazer esta operação sem loop. Pois assim seria um código muito ruim de escrever, a não ser que seja um requisito não usar nenhum tipo de loop, o que pra mim não faria sentido já que facilita e muito nossa vida como programador.
Comment: @LaísGodinho, você poderia fazer `lista = [ int(a) for a in input().split() ]` se não me engano. Daí você tem, novamente, os inteiros para brincar
Comment: Pra o usuário digitar os números positivos e negativos aleatoriamente onde a lista deve organiza-los do maior pro menor, o exemplo que fiz seria a saída do código
Comment: verdade, falta de atenção minha kkkkk, mas o método sort() vai ordenar de qualquer forma.
Comment: editei a resposta
Comment: Como fica esse código sem usar *loop*? (para fins didáticos e entendimento)
Comment: a diferença é que você teria que colocar um lista.append(int(input())) pra cada item que você quiser adicionar na lista
Comment: Como é que insere todos os códigos na mesma linha?
Comment: editei na resposta, pra ficar melhor de ver
Comment: por quê tem tantos **lista.append(int(input()))?**
Comment: foi só pra ilustrar que precisa de um pra cada item que você quiser adicionar (no exemplo que você pediu ser usar looping)
Comment: `lista = input().split()
lista.sort()
print(lista)` está estranho ela mostra os resultados assim: **['-2', '-3', '-3', '-6', '1', '3', '4', '5']**, a parte de números negativos não fica na ordem como os números positivos


iOS Sprite Kit - SKSpriteNode's .centerRect property not working

Question: I was going through the SpriteKit documentation by Apple and came across a really useful feature that I could use when programming my UI. The problem is I can't get it to work.
Please see this page and scroll down to "Resizing a Sprite" - Apple Docs
I have literally copied the image dimensions and used the same code incase I was doing something wrong. But I always end up with a stretched looking image rather than the correct "end caps" staying the same scale.
I am referring to this code:
SKSpriteNode *button = [SKSpriteNode spriteWithImageNamed:@"stretchable_button.png"];
button.centerRect = CGRectMake(12.0/28.0,12.0/28.0,4.0/28.0,4.0/28.0);

What am I doing wrong? Is there a step I have missed?
EDIT:
Here is the code I have been using. I stripped it of my button class and tried to use it with an SKSPriteNode but still the problem persists. I also changed the image just to make sure it wasnt that. The image im using is a 32x32 at normal size.
SKSpriteNode *button = [SKSpriteNode spriteNodeWithImageNamed:@"Button.png"];
[self addChild:button];

button.position = ccp(200, 200);
button.size = CGSizeMake(128, 64);
button.centerRect = CGRectMake(9/32, 9/32, 14/32, 14/32);

Comment: When/how do you add the button to your view?
Comment: I add the button in my SKScene's init method
Answer: The .centerRect property works as documented if you adjust the sprites .scale property.
Try:
SKTexture *texture = [SKTexture textureWithImageNamed:@"Button.png"];
SKSpriteNode *button = [[SKSpriteNode alloc] initWithTexture:texture];
button.centerRect = CGRectMake(9/32, 9/32, 14/32, 14/32);
[self addChild:button];
button.xScale = 128.0/texture.size.width;
button.yScale = 64.0/texture.size.height;

Comment: Is there a cut and paste error on that last line of code?  Should it be yScale & height?
Answer: 9/32 is integer division, so the result passed to CGRectMake is zero. Ditto the other three parameters. If you use floating point literals like the example you cite, you might get better results.
Answer: Here's a refresh of how exactly this works. By the way, my image size width is 48 pixels and height is 52 pixels, but this doesn't matter at all. Any image can be used:
SKSpriteNode *button = [SKSpriteNode spriteNodeWithImageNamed:@"Button.png"];

//(x, y, width, height). First two values are the four corners of the image that you DON'T want touched/resized (They will just be moved).
//The second two values represent how much of images width & height you want cut out & used as stretching material. Cut out happens from the center of the image.
button.centerRect    = CGRectMake(20/button1.frame.size.width, 20/button1.frame.size.height, 5/button1.frame.size.width, 15/button1.frame.size.height);

button.position      = CGPointMake(self.frame.size.width/2, self.frame.size.height/2); //Positions sprite in the middle of the screen.
button.xScale        = 4; //Resizes width (This is all I needed).
//button.yScale      = 2; //Resizes height (Commented out because I didn't need this. You can uncomment if the button needs to be higher).
[self addChild:button];

Read the section called "Resizing a Sprite" in this document: https://developer.apple.com/library/ios/documentation/GraphicsAnimation/Conceptual/SpriteKit_PG/Sprites/Sprites.html#//apple_ref/doc/uid/TP40013043-CH9-SW10
'Figure 2-4  A stretchable button texture' demonstrates how the (x, y, width, height) works.
Answer: Based on rwr's answer here is a working init method for a SKSpriteNode. I use this in my own game. Basically you make insets of 10px all around the the output image. And then call it like this:
[[HudBoxScalable alloc] initWithTexture:[atlas textureNamed:@"hud_box_9grid.png"] inset:10 size:CGSizeMake(300, 100) delegate:(id<HudBoxDelegate>)clickedObject];

-(id) initWithTexture:(SKTexture *)texture inset:(float)inset size:(CGSize)size {
    if (self=[super initWithTexture:texture]) {

        self.centerRect = CGRectMake(inset/texture.size.width,inset/texture.size.height,(texture.size.width-inset*2)/texture.size.width,(texture.size.height-inset*2)/texture.size.height);

        self.xScale = size.width/texture.size.width;
        self.yScale = size.height/texture.size.height;

    }
    return self;
}

Comment: Unfortunately for me this method is not working. I am testing on iOS8.


Help needed in understanding the basics of Cartan decomposition of a Lie algebra

Question: I am trying to learn the basics of Cartan decomposition of Lie algebra, and have come across the following example.
Consider $\mathfrak{gl_n}$ as the Lie algebra of endomorphisms of $\mathbb{C}^n$. Let the Cartan involution on $\mathfrak{gl_n}$ be given by
$\theta (A)=-A^*, $ where $*$ denotes conjugate transpose of $A \in \mathfrak{gl_n}$ and the  killing form is given by  $$(X,Y)=-\frac{1}{2} Tr(A\theta(B)).$$
I have the following questions:
$1)$ What does it mean by $\pm1-$eigenspace of $\theta$ ? This wikipedia page tells that

Since $\theta ^ 2 =1,$ the linear map $\theta$ has the two eigenvalues $\pm1.$

What is the definition of eigen-space in this context? I know eigenspaces only in the context of basic linear algebra. I got confused here.
$2)$ What is the Cartan decomposition of $\mathfrak{gl}_n$ ? Any suggestions for references?
Thanks!
Comment: The [Cartan decomposition](http://en.wikipedia.org/wiki/Cartan_decomposition) refers to a decomposition of a *semisimple* Lie algebra. For question 2.) start therefore with $\mathfrak{sl}_n(K)$.
Comment: $\theta$ acts on matrices. IOW $\theta$ is a mapping from $\mathfrak{gl}_n$ to itself. So its eigenspaces are subspaces of $\mathfrak{gl}_n$. For example $\theta(I_n)=-I_n$, so $I_n$ is an eigenvector belonging to eigenvalue $-1$. As are all the real diagonal matrices.
Comment: Ok! Is it true that the decomposition $\mathfrak{gl}_n = u_n \oplus \mathcal{H}_n$ is orthogonal? Here $\mathfrak{u}_n$ is space of skew-hermitian matrices and $\mathcal{H}_n$ is space of Hermitian matrices. I have been told to prove this. But it does not seem to be the case. Could you look into this if the decomposition is orthogonal? If so how can I prove it? I asked a question here: (http://math.stackexchange.com/questions/1079372/how-to-show-trace-of-ab-is-zero-for-a-in-mathfraku-n-and-b-in-mathcal) Thanks!
Answer: The map $\theta$ is a linear map of the vector space $\mathfrak{gl}_n$.  Eigenvectors of $\theta$ are matrices $X$ such that $-X^* = \lambda X$.  Since $\theta^2$ is the identity function, eigenvalues of $\theta$ must be square roots of $1$, namely $\pm 1$.
The eigenspace corresponding to the eigenvalue $1$ is the set
$$
\left\{X \in \mathfrak{gl_n} : -X^* =X\right\}
$$
This is the set of skew-hermitian matrices.
The $-1$-eigenspace is the set
$$
\left\{X \in \mathfrak{gl_n} : X^* =X\right\}
$$
of hermitian matrices.
Look in Helgason's book Differential Geometry, Lie Groups, and Symmetric Spaces for more.


Replacing tokens in Newsletter

Question: I'm trying to create a custom template using the Drupal Newsletter module.
I need to populate the newsletter with existing content. For this, I think I need to replace a token like [node:nid] in the newsletter body (which is created with CKEditor). However, I have been unsuccessful.
When I use the token filter module, I'm only able to replace global tokens like date or [site:url] and not tokens involving a node. Can someone please help me out?
Comment: Could you  share the link the Drupal newsletter module. ? :)
Comment: @Vic I believe he is referring to the [Newsletter](https://drupal.org/project/newsletter) module.
Comment: This might work for you: [Token Insert](https://drupal.org/project/token_insert) Edit: Module Page Description: This module allows you to insert tokens into a textarea. It supports both plain text and wysiwyg textareas. The format used for the insert is compatible with Token Filter.
Answer: Tokens are contextual based on where they are used, so it's not possible to reference arbitrary content. If you can add fields to the newsletter, the ideal way to resolve this would be to add a node reference field, then create your tokens by doing something like [newsletter:field_node[0]:entity].

Replacing tokens in Newsletter


Resource for writing assembly[shellcode]

Question: I am writing assembly[shellcode] for i386 linux. I wanted to know if there are any man pages or cheats sheets to see which registers need to be set-up for the calls[ read, write, execve] to be made.
Comment: Google "Linux system calls 32 bit".
Answer: You can consult syscall man-page for syscall ABI on different architectures. For a specific syscall, see the man-page of the corresponding C-library wrapper function which usually includes a C library/kernel differences chapter if necessary. For the list of all syscalls see syscalls man-page.
For the actual syscall numbers see asm/unistd.h header.


Get Datas from post Multipart (Content-Disposition: form-data) with symfony2

Question: With AngularJS I post form datas and file in one request. I can't handle datas properly in the controller of symfony2.
I only get a full string of datas with $request->getContent() but it is not parsed.
here is more details :
1 - I send data like this example and it works => http://shazwazza.com/post/Uploading-files-and-JSON-data-in-the-same-request-with-Angular-JS
2 - here are datas sent :
-----------------------------23743060120122
Content-Disposition: form-data; name="model"

{"type_inventaire_id":"8","profondeur_id":"2","code_pays_elv":"FR"}
-----------------------------23743060120122
Content-Disposition: form-data; name="file"; filename="importChep.txt"
Content-Type: text/plain

69182002
22096034
22096033
00110011
//69182007
69182003

-----------------------------23743060120122

3 - i can only get datas with $request->getContent() but it is not parsed
I would like to get it like that :
$file = $request->files->get('file');
$model = $request->get('model');

I look for a solution for one day but i did not found anything...
Anyone have an idea?
Answer: Are you doing a POST or a PUT request? I found out that when using PUT method symfony doesn't give you access to the attributes, however using POST you should be able to access your data:
$file = $request->files->get('file');
$model = $request->request->get('model');

Comment: it is a POST request. I think that it is the MULTIPART the problem.
Comment: did you find a solution for this ?


Disable authentication for an app cloudflare

Question: I have a server and I make a tunnel with cloudflare to be able to access from outside creating an application in Cloudflare Zero Strust and I am trying to disable all the policies so that it allows me to access the application without authentication.

But nothing works, whenever I try to enter my linkstream.domain.org application, it asks me for authentication via email.
The reason is that this application allows me to play streams so I don't want it to have authentication to access.
I've searched and configured everything, but I can't remove this authentication from cloudflare:
Answer: Cloudflare Access is a product that can be used to add authentication to an application. If you want your application to be public (i.e. no authentication), I'd recommend not adding it to Access at all. You can set up a Cloudflare Tunnel without adding any Access application, for example to expose a webserver to the public.
I'd recommend looking also at the Allow policies.


Excluir palavras de um arquivo

Question: Eu tenho um arquivo CSV e tenho essa lista de palavras aqui: https://gist.github.com/alopes/5358189
O arquivo CSV possui 3 colunas, Texto, user.name e Classe e possui cerca de 100K de linhas. Preciso excluir da primeira coluna do CSV todas as palavras que aparecem na lista.
Podem me ajudar?
Comment: Você quer excluir só a palavra ou tirar a linha toda?
Comment: Só a palavra mesmo.
Comment: Temo como colocar alguns registros do seu CSV pra ver como tá o padrão?
Comment: cara, desculpa a demora pra responder, estou extremamente ocupado esses dias, vou ver se posto ainda hoje um trecho do arquivo.
Valeu!
Answer: Usando Perl (desculpa...) mas fácil de traduzir para awk
$ cat stoplist.txt
de
a
o ....

$ cat ex.cvs
meu caro amigo;jjoao;classe a
eu ando a aprender weka;Thyago;classe b
mas a sua sintaxe dá-me algumas dores de cabeça;Thyago;classe a

Seja rmstopwords o seguinte ficheiro Perl:
BEGIN{  $patt="que";      ## contruir uma regexp reg com as palavras
  open(G,"stoplist.txt");
  while(<G>){chomp;
    $patt.="|$_" if $_    ## patt="que|de|a|o|..."
  }
}

$F[0] =~ s/\b($patt)\b//g;  ## no primeiro campo, subst(patt por "")
print join(";",@F)

Aplicado ao nosso ficheiro ex.csv dá:
$ perl -naC -F';' rmovestopwords t.cvs
 caro amigo;jjoao;classe a
 ando  aprender weka;Thyago;classe b
    sintaxe dá- algumas dores  cabeça;Thyago;classe a

Comment: Opa!
Muito obrigado, vou tentar e aviso aqui se funcionou.
Valeu!

Excluir palavras de um arquivo


Systems of linear equations: solving by addition

Question: I have not understood this question. it confused me a lot. I have tried to solve it but the solution was senseless.
$$\begin{cases}
3x + 4y - 5 = 2x + 3y - 1\\
6x - 2y + 2 = 4x - 3y - 5
\end{cases}$$
Can anyone help me with this?
Answer: HINT:
First turn the top and bottom equations into standard form, so you end up with a system that looks like this.
$$\begin{cases}
Ax + By  = C\\
Dx + Ey = F
\end{cases}$$
This can be done by combining the $x$ and $y$-terms on the left and moving the constant terms to the right. Then solve this "normal-looking" system using addition.
Comment: @user22634 You are welcome!


SSO between different federation protocols

Question: Microsof ADFS supports below federation protocols:

SAML 2.0
WS-Federation
OpenID Connect

Would like to know is it possible to achieve single sign on (SSO) and single logout across different protocols? E.g. user X after he/she successfully login to website A which is protected under SAML 2.0, then the user can access website B which is protected under OpenID Connect without re-login? Both website A and website B point to the same IdP.
Answer: There are typically the parties involved. Of these, website B is the modern architecture and can support many forms of authentication:

Website A redirects to SAML Identity Provider (IDP) directly
Website B redirects to Authorization Server (AS) which redirects to the SAML IDP

Single Sign On is represented by a third party session cookie issued for the SAML IDP's domain name. Both apps would redirect to the SAML IDP but only the first would result in a login prompt. Website B may need to send the acr_values OpenID Connect parameter during its redirect, to avoid the user being presented with an authentication selection screen. It should work fine, but test for your providers. The SSO cookie is maintained in current browsers on the second redirect, due to the user gesture of a top level redirect.
Single Sign Out has always been a set of draft standards that work imperfectly in all of the above protocols. Even if the technology works, there would be risks such as also signing the user out of App C, which you do not own. Also, recent browser restrictions on third party cookies mean some techniques may drop the SSO cookie and no longer work. Most stakeholders, including end users and security reviewers, do not expect you to have a perfect Single Logout solution.


Why does my lua script get different values each time I run it?

Question: I have created a simple 2d scene in Love2D with a square that falls until it reaches a certain point and then stops. The problem is that the square stops at a slightly different point every time with no user input or modified code.
Here is my lua
function love.load()
    playY = 0
    playX = 10
    grav = 200
    speed = 100
end

function love.draw()
    --floor
    love.graphics.setColor(0,255,0,255)
    love.graphics.rectangle("fill", 0,465,800,150)
    --player
    love.graphics.setColor(255,255,0,255)
    love.graphics.rectangle("fill", playX,playY,10,10)
    --debug
    love.graphics.print(playY, 100, 5)
    love.graphics.print(playX, 100, 15)
end

function love.update(dt)
    if playY < 454 then
        playY = playY + grav * dt
    end
    if playY == 456 then
        if love.keybord.isDown("right") then
            playX = playX + speed * dt
        end
    end
end

The variable playY shows the player height but it stops at different values every time.

Why is this?
Comment: No user input? this: `love.keybord.isDown("right")` seems to suggest otherwise.
Comment: By no user input I mean I haven't touched the keyboard.
Answer: I haven't used love2d so I could be totally wrong, but based on my experience with various GUI: my guess is that Love2d handles these calls in an idle event loop so you are not guaranteed that the time steps are constant or the same every time you run your program, this will cause the sequence of positions to be different every time (print them, you'll see). Unless love2d has a timer function that has fairly good accuracy regardless of what is happening in the GUI (would be surprising), you'll have to be content with the accuracy (0.5%, not bad) that love2d supports. This means you can't use conditions like if something == 456 because you might miss it, better use a range.
Answer: As mentioned earlier dt is only as consistent as the games refresh rate. This value is multiplied with the velocity to give smooth gameplay.
If this is a player and you wish the y to stop at 456, you can always write
if playY > 456 then
    playY = 456
end

You can pretty much guarantee that the playY will stop at 456 every time because it will snap the player back to that spot.


How to dynamically set component host style Angular 8

Question: I want to dynamically set style for child host component element from parent.
There is a solution to do it individually for each style as:
@HostBinding('style.width') @Input() width = 'auto'
But this way i need to make multiple lines for different styles.
Bellow is a solution to do this with Host binding and input combination.
Answer: Here is a solution that can cover if you want to pass multiple css styles as string or as object with cammelCase conventions:
Parent HTML
<app-button [style]="styleFromParent">Some button</app-button>

Parent component has styleFromParent property and it has simulation if that property is changed at some point:
Parent Component TS
import { Component, OnInit, ChangeDetectorRef } from '@angular/core';

@Component({
  selector: 'app-site-panel',
  templateUrl: './site-panel.component.html',
})
export class SitePanelComponent implements OnInit {
  constructor(private _detectChanges: ChangeDetectorRef) {}
  styleFromParent = { marginTop: '10px', marginLeft: '50px' };

  ngOnInit() {
    setTimeout(() => {
      this.styleFromParent = { marginTop: '20px', marginLeft: '1px' };

      this._detectChanges.detectChanges();
    }, 2000);
  }
}

Child HTML
<ng-content></ng-content>

Child Component TS
In child component part that does parsing if typeof style === 'object' is to parse cammelCase styles into dash based styles and concat them into one line string.
import { Component, OnInit, HostBinding, Input } from '@angular/core';
import { DomSanitizer, SafeStyle } from '@angular/platform-browser';

@Component({
  selector: 'app-button',
  templateUrl: './button.component.html',
})
export class ButtonComponent implements OnInit {
  @HostBinding('style') baseStyle: SafeStyle;

  @Input()
  set style(style: string | object) {
    let mappedStyles = style as string;

    if (typeof style === 'object') {
      mappedStyles = Object.entries(style).reduce((styleString, [propName, propValue]) => {
        propName = propName.replace(/([A-Z])/g, matches => `-${matches[0].toLowerCase()}`);
        return `${styleString}${propName}:${propValue};`;
      }, '');

      this.baseStyle = this.sanitizer.bypassSecurityTrustStyle(mappedStyles);
    } else if (typeof style === 'string') {
      this.baseStyle = this.sanitizer.bypassSecurityTrustStyle(mappedStyles);
    }
  }

  constructor(private sanitizer: DomSanitizer) {}

  ngOnInit() {}
}

Above you can see that baseStyle has HostBinding to style component binding.
When style input is passed setter will triger, check if string or object is passed, parse it to string and sanitize that css and assign it to baseStyle thus host style will change.
EDIT:
As @Scorpioo590 said, this is replacable with [ngStyle].
So this seems unneeded. Maybe at least this can show what can you do with style and HostBinding.
Comment: wasnt unneeded for me as  i wanted to set host from child based on component input. awesome to have this!
Answer: ngStyle is designated for this.
<app-button [ngStyle]="{ marginTop: '10px', marginLeft: '50px' }">Some button</app-button>

Of course, instead of hard coding it into the template like this you could outsource it into the controller.
https://angular.io/api/common/NgStyle
PS: Since style already exists on all elements I'm not sure of the behavoiur of your code
Comment: Yea, this is a good point.
Maybe use custom host binding makes sense if you want to filter out some styles. As for the behaviour of [styles] attribute, it works as expected.


augmentation ideal of restricted universal enveloping algebra

Question: For restricted Lie algebra $L$ we denote its restricted universal enveloping algebra with $u(L)$. How can we prove that the augmentation ideal has codimension $1$?
Answer: By Jacobson's PBW-theorem we have $\dim u(L)=p^{\dim (L)}$. Now the augmentation ideal $Lu(L)=\omega(L)$ is a proper ideal of $u(L)$. Compute its dimension.


What exactly is the standard error of the intercept in multiple regression analysis?

Question: I understand that in multiple regression analysis, for each independent variable, you would graph dependent variable vs independent variable and you would make a line of best fit and calculate the standard error.
However, how does it work for intercept? Can some one give me a concise but clear explanation?
Answer: Your characterization of how multiple regression works is inaccurate.  Your version implies fitting a simple linear regression for each variable in turn (and presumably using each of those slopes as the coefficient for that variable in the multiple regression model).  This notion leaves you with the problem of how to deal with the fact that the intercepts from each simple regression are quite likely to differ.
However, that approach is not how multiple regression works / estimates the parameters.  Instead, all coefficients (including the intercept) are fitted simultaneously.  Using Ordinary Least Squares (OLS), we find coefficient estimates that minimize the sum of the squared errors in the dependent variable.  That is, we minimize the vertical distance between the model's predicted Y value at a given location in X and the observed Y value there.  To find a vector of beta estimates, we use the following matrix equation:
$$
\boldsymbol{\hat\beta} = \bf (X^\top X)^{-1}X^\top Y
$$
It is worth noting explicitly that the coefficients we find this way will not necessarily be the same as those betas found individually.  To understand this further, it may help you to read my answer here: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?
At any rate, the standard errors for a multiple regression model are calculated as:
$$
SE_\boldsymbol{\hat\beta} = \sqrt{{\rm diag}\{ s^2\bf (X^\top X)^{-1}\}}
$$
where $s^2$ is the variance of the residuals and $\rm diag$ refers to extracting the elements on the main diagonal of the matrix.  Since the intercept ($\hat\beta_0$) is first of our regression parameters, it is the square root of the element in the first row first column.
Once we have our fitted model, the standard error for the intercept means the same thing as any other standard error: It is our estimate of the standard deviation of the sampling distribution of the intercept.  For a fuller description of standard errors in a regression context, it may help to read my answer here:  How to interpret coefficient standard errors in linear regression?
A common use of the intercept's standard error would be to test if the observed intercept is reasonably likely to have occurred under the assumption that its true value is some pre-specified number (such as $0$), as @IrishStat notes.
Comment: You said "That is, we minimize the vertical distance between the model's predicted Y value at a given location in X and the observed Y value there" . Did you mean "That is, we minimize the sum of the squares of the  vertical distances between the model's predicted Y value at a given location in X and the observed Y value there based upon all observations." as compared to least absolute deviation (LAD)
Comment: @IrishStat, I was trying to rephrase the initial statement in a more intuitive, but less mathematically rigorous way. I meant squared distances, not absolute distances.
Comment: @gung-ReinstateMonica Do you have any references for how you define the standard error? It's so clean and easy to compute, but I haven't found any books / papers that use this formulation. In addition, the results using your method differ slightly (to maybe 2-3 decimal places) compared to standard statistical packages.
Comment: @PyRsquared, just any regression textbook. I probably copied this out of Kutner et al. *Applied Linear Statistical Models* (which I don't have with me right now). You can also see it on the [Wikipedia page](https://en.wikipedia.org/wiki/Ordinary_least_squares) (although the notation is a little different).  Statistical packages should give you the same values, but they often use slightly different computations to enhance numerical stability.
Comment: @gung-ReinstateMonica Thanks for the quick reply. Ok that's great - I also saw the same formulation [here](https://stats.stackexchange.com/a/27920/178320) where the post mentions the book "Applied Linear Regression by Draper and Smith"
Comment: @PyRsquared, it should be the same in *every* regression book, ;-).  Good luck w/ your project.
Answer: The standard error of the the intercept allows you to test whether or not the estimated intercept is statistically significant from a specified(hypothesized) value ...normally 0.0 . If you test against 0.0 and fail to reject then you can then re-estimate your model without the intercept term being present.
Comment: "...then you can re-estimate your model without the intercept term..." is a statement that needs about a textbook's worth of qualification and caveats.
Comment: as MichaelChirico correctly pointed out NOT including the intercept can have unwanted side effects . See http://www.azdhs.gov/lab/documents/license/resources/calibration-training/09-linear-forced-through-zero-calib.pdf and http://stats.stackexchange.com/questions/113777/importance-of-intercept-term-in-regression-equation


How to get these content types to appear in my view based on role?

Question: I have two content types known as Generic Credit and Generic Expense. There are only three roles (Secretary, Admin, and Developer) that can create and view this type of content for administrative purposes. Additionally, these are important for a view that acts as a budget, which gathers several types of financially-related nodes. All of these content types are used for calculating a budget for teams (comprised of a coach and several students). This budget view can be viewed by the students and coaches, and it filters correctly, showing the appropriate budget for a team.
That is, unless it's pulling a Generic Credit or Generic Expense. When a student or coach without the three creating roles views the budget, these two content types don't appear. However, when a coach with the Admin or Secretary roles uses the view, they are able to see the Generic Credit/Expenses.
So far, I've tried using Node View Permissions, but that didn't have any affect on the views. I've also tried allowing students and coaches to create and edit the content types, but that has proved futile.
Am I missing something here with the permissions? What settings might I need to check?
Answer: You'll need to add a restrictions in your views. There's an access restriction section there where you can set it by role. Select which roles can view it, then save. That should work. Please refer to the picture attached where to find the permission access. Also, try Content Access module to restrict content type per role.
Comment: yes, just click on **Permission** and select role.
Comment: This worked out perfectly - I'm not sure why Node Access Permissions didn't work, but Content Access and modifying the View Access permissions resolved the issue.


will_paginate 3.0 is not compatible with Rails 2.3 or older (RuntimeError)

Question: When I run rails -v the output is Rails 3.0.15. I also have done gem install rails 2.3.5 which is what the environment.rb files says it needs.
Why am I getting this error when my rails aren't older than 2.3.
Comment: And what is your question? Do you want to run Rails 2.x or 3.x? Use will_paginate for Rails 2.x or 3.x?
Comment: I just uninstalled the rails 3.0.5 gem. And now I have rails 2.3.5 and 2.3.11. But still I can't get the server up and running
Answer: You can't mix Rails 2 and Rails 3, even if both gems are installed. There are two options:

Install the 2.3.x version of will_paginate
Upgrade your app to Rails 3
Comment: Actually, I had just done rake gems:install, and didn't notice that the will_paginate gem was installed for Rails 3.0. Thanks for the help!


Centralizar uma única div no body com dsplay: Flex

Question: Tenho o body e uma div com width: 320px;
Gostaria de centralizá-la no body que é flex.
Mas veja só, NÃO quero centralizar seu conteúdo.
Gostaria de centralizar a própria div na body.

    body {
       display: flex;
    }

    div#tudo {
       display: flex;
       width: 320px;
    }

          teste

Uso margin: 0 auto ou tem algum recurso flexbox para isso?
O que eu busco é por uma propriedade que faça semelhante ao resultado do align-self: center. Porém, na horizontal.
Comment: Relacionada ou nova duplicada: https://pt.stackoverflow.com/questions/355954/no-flexbox-por-que-o-justify-self-n%c3%a3o-funciona-nos-filhos-de-um-pai-com-display
Comment: Bom, adicionei informações à pergunta. Se puder ler... Agradeço! Mas vou ler as tuas referências!
Comment: Já tentou utilizar o `margin` que cita? Qual foi o resultado? E se não quer usar flex, não fez muito sentido pedir como faz em flex. Aliás, isso deve ter várias duplicadas no site. Chegou a pesquisar?
Comment: Centralizar na horizontal ou vertical?
Comment: Olá Antônio.  Bom dia! Então, com margim dá certo Mas quero passar a usar o flex. Nao fui muito feliz na minha colocação. O que eu busco na verdade é algo semelhante à align-self mas para a horizontal. Acho que não tem né?
Comment: Existe uma GRANDE diferença em um elemento que é um container flex, e o elemento que é filho de um container flex. Te sugiro fortemente que leia esse guia em português sobre flex https://origamid.com/projetos/flexbox-guia-completo/ é meia horinha que vc vai parar para estudar e vai entender melhor o que eu disse e pq sua pergunta é duplicada. Depois que ler o guia leia o link da duplicada, vai te ajudar ;)
Answer: #envolve {
   display: flex;
   align-items: center;
   justify-content: center;
   width: 320px;
}

      teste

tenta isso pra ver se resolve
Comment: @DemetriusFernandes Talvez tenha entendido errado o propósito do site. O objetivo do Stack Overflow é justamente ser um repositório de conhecimento que utiliza o formato perguntas e respostas, em que, idealmente, a pergunta apresenta um problema e a resposta apresenta uma descrição da solução como propósito de servidor como fonte de pesquisa para a comunidade. Dar um código pronto e dizer para o usuário "se virar sozinho" para entender é exatamente o oposto do que o site propõem. Se quiser, no [meta] já existem várias discussões sobre isso.
Comment: Por que isso resolveria o problema?
Comment: Amigão. Não centralizou. Mas, conforme o Antônio disse, parece que o que estou procurando não existe no flex ou  o que estou querendo fazer não é uma boa prática. Obrigado pelo teu tempo. Estou lendo a referência que ele passou!
Comment: Ok amigo, é sempre bom dar uma estudada antes de usar as propriedades...
Comment: mas na realidade o código que você quer é esse mesmo que eu postei, só não funcionou por que eu esqueci de definir a altura e a largura da div "envolve" com a largura máxima da Tela
Comment: Acho que não é "o código que ele quer", ele quer uma explicação de pq e como a coisa funciona... Tente dar uma enriquecida na resposta, acho que vai ajudar mais falar como pesca do que entregar o peixe
Comment: oxe mas se ele quer entender como funciona ele está no Lugar errado, pra entender como funciona o cara vai ver um Tutorial no youtube ou um Tutorial de algum site, ou algum PDF explicando, eu ein
Comment: kkkk, meninos NÃO briguem. Agradeço a todos igualmente, O que aconteceu foi que eu estudei, depois fui para a prática. Como não funcionou da maneira como esperado depois de muiiiiiiiiito quebrar a cabeça resolvi pedir ajuda aqui no fórum. Valeu e obrigado a todos!


Windows Installer: The system cannot open the device or file specified

Question: I downloaded as Administrator in Chrome an installer .msi file [Citrix XenCenter Management; so it's not an exotic program] on a Windows Server 2012 R2 into a drive F: which is hosted on the SBS and mapped to the Windows Server 2012 R2 Terminal Server session while the login process.
When I double click this file I get this error message:

So, I list these strange facts which somehow tell me this may be a stupid security setting but I am not capable to solve it:

Copying this file into the Desktop it gets executed and installed.
Running this file in SBS from the original location [which is then mapped while the login process] gets also executed and installed.
In this directory there are other .msi files which get executed/installed but also some of them have exactly the same issue and display the error message.

I deactivated the Windows SmartScreen and Network Access Protection Agent but no success.
My question:
How can I fix this issue so these .msi files are installed?
Update:
Once the program has been installed [when I ran it from the Desktop location] then suddenly the .msi file in the mapped drive can be executed, so I get the selections to modify or remove the program.
Comment: Ah, log? Run installer from command line with the flags for writing an output log. Start trying to understand it (it is very convoluted) and find the error.
Comment: Closing: Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.
Comment: @TomTom Thank you for your comment. Because of it I could fix the issue and I answered my question. If you down voted it may be you could reconsider to undo it...
Comment: Fu** - so a ramdisk tool caused it. That is definitely special.
Answer: First, I want to thank TomTom regarding his hint running the installer from the command line with the log flags. I wasn't aware of this.
In my case the problem was very trivial and was caused by the ImDiskToolkit which created a ramdisk in the memory and set the TEMP & TMP environment variables to the new mapped ramdisk drive.
when I checked the log files I saw there were these entries and then I realized few programs couldn't handle it and the installation failed.
MSI (c) (2C:54) [20:59:05:694]: Note: 1: 1402 2: HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer 3: 2
MSI (c) (2C:54) [20:59:05:712]: Error: Getting file path failed: G:\Temp\6\1635c436.msi, error = 1
MSI (c) (2C:54) [20:59:05:712]: Error: Getting file path failed: G:\Temp\6\1635c436.msi, error = 1
MSI (c) (2C:54) [20:59:05:712]: Error: This file path is updated, hence failing to create: G:\Temp\6\1635c436.msi
MSI (c) (2C:54) [20:59:05:712]: Note: 1: 1310 2: 110 3: G:\Temp\6\1635c436.msi

After uninstalling ImDiskToolkit all of these .msi installation files worked.


Why would a captain not enforce the follow-on?

Question: In the India vs Bangladesh test match in February 2017, India did not enforce follow on on the 4th day of test. It would be logical to try to get Bangladesh all out on second innings and then quickly chase down a meagre total.
What can be the possible reason behind Kohli's decision?
Answer: The only thing that comes to my mind is to give bowlers a bit of a breather. Indian bowlers had been bowling since Day 2 (so almost 2 full days) and their legs and fingers (for spinners) must have been aching and tired. Teams typically adopt this practice by not opting for follow-on and give their bowlers a breather before they come out again to bowl out the other team again!
Answer:
It would be logical to try to get Bangladesh all out on second innings and then quickly chase down a meagre total.

Logical when it works :-)
The situation captains are trying to avoid here is something like:

The pitch is still reasonably good for batting.
Tired bowlers aren't incisive at the start of the follow-on, letting some batsmen get set.
The side following on makes a big score, giving them a lead of 200 or so1.
The side which enforced the follow on now has to make 200 on a fifth day pitch. A couple of quick wickets, the jitters set in and they end up losing.

To some extent, not enforcing the follow on is the safe option - knock up a quick 150-200, declare with a total the other side is going to be too scared to even attempt to chase, and you're not going to lose.

Much less likely in this scenario where India had a lead of 300, I admit.
Comment: [This India-Australia test is another example of a greater reversal of fortunes.](http://www.espncricinfo.com/ci/engine/current/match/63920.html)
Comment: @CodeNewbie Possibly. But I'm a Brit, so I'll link that one ;-)
Comment: And I'm Indian, hence my link. :)