How many "elementary" isometries are needed to generate any isometry of an n-dimensional Euclidean space?

Question: I use here the term Euclidean space in the rigorous sense of an affine space over $\mathbb{R}^n$, equipped with the Euclidean inner product (see here). Let $\mathbb{E}^n$ be the Euclidean $n-$space.
Consider the set $S$ of all isometries on $\mathbb{E}^n$. Given any element $f\in S$, intuitively I would expect $f$ to be equivalent to a composition $g$ of the three "elementary" isometries, i.e., rotations, translation and reflections. With equivalent, I mean that $f(P)=g(P) \ \forall P \in \mathbb{E}^n$.

Is this true? Since "elementary" isometries are affine functions (they can all be represented by a multiplication by a matrix or by addition of a constant, i.e., a translation), and since the composition of two affine functions is affine (I think), this would imply that all isometries on $\mathbb{E}^n$ are affine functions, right?
If it is, how many "elementary" isometries are needed to generate any isometry? For example, can we say that for $n\ge2$, any isometry is a composition of a rotation $or$ a reflection plus a translation?
Answer: In a Euclidean vector space of dimension $n$, any isometry is the product of at most $n$ reflections.
The proof consists in defining by induction $r_1, \dots, r_n$ reflections  such that $r_1 \circ f$ stabilizes a line, $r_2 \circ r_1 \circ f$ stabilizes à plane and so on where $f$ is the isometry that you are looking to decompose as a product of reflections.
Knowing that a translation is a product of $2$ reflections, you can also conclude that any affine isometry in an affine space of dimension $n$ is the product of affine reflections,at most $n+2$.
Comment: Two comments. First any isometry in a finite dimensional Euclidean space is linear. See https://math.stackexchange.com/questions/570139/should-isometries-be-linear. Second, the notion of affine reflection exists. Finally it is easy to write a translation as a produçt of two affine reflections with parallel hyperplanes.
Comment: Please try to be a bit flexible... An isometry of an "Euclidean affine space" induces an isometry of the underlying Euclidean vector space. This one is linear as proved in the link I provided. Hence the first place isometry of the affine space is affine. And so on...
Comment: https://math.stackexchange.com/questions/138020/proving-that-every-isometry-of-mathbbrn-is-of-the-form-of-a-composition-of
Comment: I was going to edit my question a bit, but you answered before I could do that :-) anyway, I think your answer will answer also my edited question, with just a minor edit.
Comment: Hmmm, sorry but you say that "any affine isometry in an affine space of dimension n is the product of affine reflections,at most n+2". I was talking about (bijective) isometries in general, not just affine ones. Of course, if all bijective isometrics are affine, then your answer is fine, but could you either prove your assertion or provide a reference to the proof?
Comment: Finally, "a translation is a product of 2 reflections". This seems weird - reflections are linear transformations,since they  can
be  represented by an  orthogonal matrix whose determinant is -1. If a translation is
the product  of two reflections (product $\equiv$ composition here, right??) then this would mean that all affine transformations are linear...it sounds wrong
Comment: Well, no. The question you link to seems to explicitly say that any isometry onto $\mathbb{E}^n$ is _affine_, not linear. As a matter of fact, both the OP corrected the text of her/his question ("Is it true that $T$ is linear affine?" a dashed line covers "linear" in the original), and also [the accepted answer](https://math.stackexchange.com/q/570139/169160) starts by saying that "I suppose you meant that $T$ is *affine* rather than merely *linear*."
Comment: Concerning the fact that a translation can be produced by composing two reflections, I thought about it and it's obviously true in the plane (just let the reflections be around two parallel axes). I'll trust you on the fact that it's true $\forall n$.
Comment: Ok, I did clarify in my original question that with Euclidean space I was referring to an _affine_ space, but I didn't know that by extension you could call also the underlying vector space Euclidean. My bad. Also, in meantime I found [this](https://math.stackexchange.com/questions/1817077/distance-preserving-function-on-hilbert-space) and, from there, [this](https://en.m.wikipedia.org/wiki/Mazur%E2%80%93Ulam_theorem), which I didn't know about. This again confirms what you said, i.e., any isometry is affine (even for infinite-dimensional spaces).
Comment: Ps just for clarity, I didn't downvote your answer: I upvoted it. But I'd really appreciate if you could expand the proof a little bit, or provide a reference: I'm not sure how "stabilization" (meaning that all points on the line, say, are transformed into themselves, right)? is sufficient to say that any isometry is a product of reflections. I like your answer, but I would just be sure I understood it properly. Apologies for being a bother.
Comment: I'll read that in more detail, but if I understand correctly, your link proves that any isometry on a affine space of dimension $n$ is the product of $n+1$ reflections. Correct?
Comment: Read it please!
Comment: Ok! I wasn't going to skip it, it's just that it will take me time (the question and the answers are quite long), so don't expect me to reply soon. In meantime, if you want, you may have a look [at my other question](https://math.stackexchange.com/questions/2821401/are-non-bijective-isometries-necessarily-affine-functions).


openvpn: match client certificate to specific ldap user

Question: I'm in the process of re-configuring a openvpn setup to make it more secure.
Currently it uses only ldap authentication. I'm planning to change it to also require a client certificate.
My understanding though, is that as long as you have a valid client certificate, it won't really matter who's certificate it is.
Is there a way to have it verify that the certificate matches the ldap user?
This way, if a hacker managed to obtain a certificate, it wouldn't really be any use to them without also having the login for the matching ldap user. Likewise, if they compromised an ldap user password, they would need one specific matching certificate, not just any valid client certificate.
At this stage I'm planning to use easy-rsa to generate the CA, Server, and Client certificates, just because I know it works, and I know how to do it.
Setting up Microsoft AD (our ldap provider) as a CA, seemed a bit complicated to attempt at this stage.
Comment: You can do any checks you want in the `client-connect` script. Just return failure (exit code other than 0) and server will drop the connection.
Comment: Thanks @NikitaKipriyanov This suggestion helped me get a working solution.
Answer: Just in case it's helpful to anyone else. This is the client-connect script I ended up using...
<code>#!/usr/bin/bash

echo "client-connect: common_name=$common_name, username=$username"

if [ "$common_name" == "$username" ]; then
  echo "Client certificate common_name matches username. Allow connection."
  exit 0
else
  echo "Client certificate common_name does not match username. Reject connection."
  exit 1
fi
</code>
common_name and username are environment variables automatically provided to the script by openvpn.
From https://openvpn.net/community-resources/reference-manual-for-openvpn-2-4/


The difference between rand(1e9...1e10).to_i and rand.to_s[2..11].to_i

Question: In Ruby 2.3.0, when I do rand.to_s[2..11].to_i to insert something multiple times in a loop in the ActiveRecord, which validates length: {maximum: 10, minimum: 10}. I got an error after some iteration that the length validation failed! And when I do rand(1e9...1e10).to_i in the same scenario, it solves the problem. I want to know what's the difference between these two.
Comment: What are you trying to achieve? A string containing 10 random digits?
Answer: Kernel#rand without an argument returns a pseudo-random Float in the interval [0.0, 1.0). So, it could return such values as 0.0, or 0.123, their string representation is less than 12 characters, so, if you take "0.123"[2..11] it returns "123".
rand(range) returns a pseudo-random Integer in the given Range, so rand(1e9...1e10) always returns a number which whole part has 10 digits.
Comment: @ndn: I think you are both correct. `0`s could come from both sides of the string : 1 `0` is enough from the left, but 7 consecutive `0`s are needed from the right. So ndn's explanation is more probable, but MikDiet should play a role every 10 million rands.
Comment: @EricDuminil, you guys got lucky! (: Anyways, using `#to_i`, I got length 8 with just 100 numbers. One is significantly more likely though. But I do agree that both are possible and together they give the actual complete answer.
Comment: @EricDuminil, this assumes that all numbers with 16 digits after the decimal dot can be represented as float numbers, which is not the case. So the actual probability is slightly higher. (:
Comment: I doubt this is the issue OP is experiencing: `10_000_000.times.map { rand.to_s[2..11] }.min_by(&:length).length # => 10`.
Comment: @EricDuminil, makes sense.
Comment: @ndn: `100_000_000.times.map { rand.to_s }.min_by(&:length) #=> "0.684995671"` So it does happen sometimes.
Comment: @ndn: Note that I use `100_000_000`. So there was a `1-(1-1E-7)**1E8` chance of getting at least one random number with 7 trailing `0`s. That's approximately `0.9999546` probability. I tried it again, and even got `"0.97585113"`
Answer: #rand without arguments generates a random number between 0 and 1.
This can be 0.01111111111111. to_s[2..12] then takes the first 10 digits after the decimal dot.
'01'.to_i # => 1

Hence you might get length less than 10.
Comment: @reznov9185: Note that this answer is half of the story, or let's say 99.999% of the story. For the rest, see the other answer.


Find descendants of a specific node with Xpath

Question: Say I have an xml document and want to get the "NodeID" and "Description" element out of each Node, without necessarily knowing whether they're a sub-element of some child, or just a child.
<Stuff>
    <data>
        <Nodes>
            <NodeID>1</NodeID>
            <Name>thingA</Name>
            <MoreInfo>
                <Description>Scooter</Description>
            </MoreInfo>
        </Nodes>
        <Nodes>
            <NodeID>2</NodeID>
            <Name>thingB</Name>
            <MoreInfo>
                <Description>Bike</Description>
            </MoreInfo>
        </Nodes>
    </data>
</Stuff>

So I'd ideally end up with: 1, Scooter; 2, Bike.
NodeList nodes = (NodeList) xpath.evaluate("Nodes", resultXml, XPathConstants.NODESET);

for (int i = 0; i < nodes.getLength(); i++) {
    Node node = nodes.item(i);
    logger.info("Evaluating: " + node.toString());

    String nodeID = xpath.evaluate("//NodeID", node);
    String description = xpath.evaluate("//DescriptionID", node);

    System.out.println("Found: " + nodeID + ", " + description);
}

The output is as follows:
Evaluating: <NodeID>1</NodeID><Name>thingA</Name><MoreInfo><Description>Scooter</Description></MoreInfo>
Found: 1, Scooter
Evaluating: <NodeID>2</NodeID><Name>thingB</Name><MoreInfo><Description>Bike</Description></MoreInfo>
Found: 1, Scooter

The node is correct, but it keeps "finding" things as if it were looking from the beginning of the xml document. How do I search just for decendents of the node in question? I would think that the node being the source of the evaluation would do the trick...
Answer: //NodeID and //Description are absolute paths which will start searching from the root of the document that contains the context node.  The expressions in this case will match all nodes named NodeID (respectively Description) in the whole document, and when you evaluate such an expression as a string this will give you the string value of the first matching node in document order.
To look just within the context node itself you need to use relative paths by adding a leading dot, for example .//NodeID.


Conditional probability Poisson process and dependent random variables

Question: This is for homework, but I'm stuck cause I can't find something like this on my book, or in the internet for that matter.
Say I have a restaurant that's open the 24 hours, and we know that costumers enter the establishment according a Poisson process at a rate of 5 costumers/hour.
1. Given that 6 costumers arrived from 1:00 am and 2:30 am, what's the
    probability that less than 3 customers arrive from 2:30 am to 4:00
    am?
I have modeled this question like this: P(C<3|X=6), being C the amount of customers that could arrive and X the customers that have already arrived. so I use the definition of conditional probability:
$$\frac{P(C<3\bigcap X=6)}{P(X=6)}$$
I ended up using this property that seems shady:
$$P(A\bigcap B)=P(A) P(B)$$
Make the math and I end up with
 $$\frac{0.1367 \cdot 0.059}{0.1367}=0.059$$
which seems reasonable, but I don't think is right. So help me here if it's not ok.
2. If we know that 30 customers arrived from 10:00 pm to 4:00 am, what's the probability that 20 customers arrived from 1:30 am to 3:15 am?
This questions has me stumped. I think I should break the times in something like:
$$[10:00, 1:30)\bigcup [1:30, 3:15]\bigcup (3:15, 4:00]$$
And then do the probabilities separate, and add them up at the end, but that seems wrong, and I don't know what to do...
Please help
Answer: The textbook on Poisson processes you use surely mentions that the number of points falling in some disjoint time sets are independent. Re 1., this tells you that the conditional probability you are after is the same as the (unconditional) probability that less than 3 customers arrive from 2:30am to 4:00am. This is the probability that a Poisson random variable with parameter 5 per hour $\times$ 1.5 hour $=$ 7.5, is less than 3.
Re 2., the same textbook surely explains that once you know $n$ points fall in an interval [a,b], their locations in [a,b] are $n$ i.i.d. random variables distributed according to the density of the Poisson process. Here the Poisson process is homogeneous hence the number of customers who arrived from 1:30am to 3:15am is binomial $(n,p)$ where $n=30$ and $p$ denotes the proportion of the overall time the time span of interests represents, that is $p=\frac{1.75}6=\frac7{24}$.
Answer: There is nothing shady about the property you used in part a). In a Poisson process, events occurring in one time frame are independent of events occurring in another, disjoint, time frame.
Note that in your answer to a), the probabilities $P[X=6]$ cancel, as they should,
leaving you with
$P[C<3\mid X=6]=P[C<3]$. No suprise here, as this is a direct consequence of the independence  property. (I did not check that you computed $P[C<3]$ correctly.)

Although Didier Piau's approach to part b) is a better approach to the problem;
I think it's worthwhile to compute the probability directly. In fact, this computation may help you see why the property mentioned by Didier is true.
So I offer:
Let  $\lambda=5$ and let:
$\ \ \ A$ be the number of events occurring in the 4.25 hour time period from 10-1:30 and 3:15-4.
$\ \ \ B$ be the number of events occurring in the 1.75 hour time period from 1:30-3:15
and
$\ \ \ C$ be the number of events occurring in the 6 hour time period from 10-4.
Note $A$ has Poisson distribution with parameter $4.25\lambda$, $B$ has Poisson distribution with parameter $1.75\lambda$, and $C$ has Poisson distribution with parameter $6\lambda$.
In your problem, you wish to find $P[\,B=20\mid C=30\,]$.
We have:
$$\eqalign{
P[\,B=20\mid C=30\,]&={P\bigl[\,(B=20 )\cap (C=30)\,\bigr]\over P[C=30] } \cr
&={P[ (A=10)\cap (B=20)]\over P[C=30] } \cr
&={P[ A=10]P[B=20]\over P[C=30] } \cr
&={{\textstyle(4.25\lambda)^{10} e^{-4.25\lambda}\over\textstyle 10!}  \cdot {\textstyle(1.75\lambda)^{20} e^{-1.75\lambda}\over \textstyle20!}    \over{\textstyle(6\lambda)^{30} e^{-6\lambda}\over\textstyle 30!}   } \cr
&={{(4.25 )^{10} \lambda^{10} \over 10!} \cdot  {(1.75 )^{20}\lambda^{20} e^{-6\lambda}\over 20!}   \cdot {30!\over 6 ^{10}6^{20}\lambda^{30} e^{-6\lambda} }   } \cr
&=\Bigl({4.25\over 6}\Bigr)^{10} \Bigl({1.75\over 6}\Bigr)^{20} {30!\over 10! 20!}.
}
$$


Bottleneck building block in Residual learning networks

Question: I am wondering about how 1x1 convolution can be used to change the dimensionality of feature maps in a
residual learning network.
Here the top 1x1 convolution changes the feature map size from 256 to 64. How is this possible?
In a previous post explaining 1x1 convolution in neural net, it is mentioned that, if a layer having $n_1$ feature maps is subjected to 1x1 convolution with $n_2$ filters then number of feature map changes to $n_2$. Shouldn't it be $n_1$$n_2$ since each of the $n_2$ filters produce one output  corresponding to each of the $n_1$ inputs.
Also how does one generate 256 feature maps from 64, as done in the bottom layer.
Answer: There's only one parameter for each input map in a 1*1 filter, actually the 1*1 convolution is multiplying the every element of an input map by the same scalar.
So it is similar to getting 265 linear combinations out of 64 variables, the $n$-th feature map $y_n$ is like,
$$y_n=f(w_{n,1}x_1+w_{n,2}x_2+...+w_{n,64}x_{64})$$
so actually we can get any number of output feature maps as we want. Of course if the output dimension is greater than the input dimension, the output would be redundant.
Comment: @Newstein yes by weights I mean the single parameter in a 1x1 filter, and there're not in total 64 filters, there are 64 output channels (or feature maps), which means there'll be 64*256 filters (you apply a different filter to each input channel, add up the results and apply an activation function to get the output of one channel)
Comment: I really think this answer is misleading. There a 1x1 convolutional filter is "1x1" just in the spatial dimension, but it is a 1x1xC convolution, where C is the number of input feature maps or channels (256 for the first layer in the OPs example example). Therefore, you actually have C parameters for a 1x1 convolution. Since you want to output 64 different feature maps, you have 1x1xCx64 parameters for the whole layer.
Comment: @facuq yea it is, I've made some edit
Comment: What is $w^n_i$ ?. Is it just $w_i$
Comment: @Newstein It's meant to be in accordance with the subscript of $y_n$, so you have a different set of weights for different output channels.
Comment: @Newstein I made that up, it's not in the paper. :P
Comment: How about reducing the dimensionality from 256 to 64 in the top layer?
Comment: @Newstein it the same, only you have n=1, 2,...,64 instead of 256.
Comment: Are you saying that 64 $x_i$s are randomly selected from 256 input maps and their weighted combination is taken using the 64 weights ($w_i$) corresponding to the 64 1x1 filters
Comment: @Newstein no I mean the input of course consists 256$x_i$s, and we have 256 weights per output channel, and we'll have 64 channels.
Comment: How can we have 256 weights when there are only 64 filters? I am assuming that by weights you mean the single parameter in a 1x1 filter


$\mathbf{P}(S_{n(i)}=i \mid S_1=i),\mathbf{P}(S_{n(i)+1}=i\mid S_1=i),......$ are zeros?

Question: $\left\{\xi_{n}\right\}_{n\in\ \mathbb{N}_{+}}$ is a sequence of independently and identically distributed random variables, each taking a finite number of integer values.$\mathbf{E}(\xi_1)\ne 0,$ For any $n\in \mathbb{N}_{+},$define $S_{n}:=\sum_{i=1}^{n}\xi_{i}.$ Show that $\left\{S_{n}\right\}_{n\in\ \mathbb{N}_{+}}$ is a Markov chain, with each state being transient.

My question is how to prove whose each state is transient.I attempt to demonstrate $\sum_{n=1}^{\infty}\mathbf{P}(S_{n}=i\mid S_1=i)<\infty $ for each $i$ in the state space $S$, by Kolmogorov's strong law of large numbers and  Borel–Cantelli lemma. Maybe we can use this equivalence
$$\frac{S_n}{n} \xrightarrow[]{a.s.}\mathbf{E}(\xi_1)\Longleftrightarrow \displaystyle \lim_{ n\to \infty}\mathbf{P}\left(\bigcup_{k=n}^{\infty}\left\{\left|\frac{S_k}{k} -\mathbf{E}(\xi_1)\right|\ge\varepsilon\right\}\right)=0, \forall \varepsilon>0.$$to create a paradox to support that starting from some positive integer $n(i)$, all subsequent terms in $\sum_{n=1}^{\infty}\mathbf{P}(S_{n}=i\mid S_1=i)$ i.e. $\mathbf{P}(S_{n(i)}=i\mid S_1=i),\mathbf{P}(S_{{n(i)+1}}=i\mid S_1=i),......$ are zeros.

$\left\{S_{n}\right\}_{n\in\ \mathbb{N}_{+}}$ is a time-homogeneous Markov chain.
Someone have objections, and the reasons are as follows:
From Chapman-Kolmogorov equation, $k+m$-step transition probability \begin{align}
   \mathbf{P}(S_{n+k+m}=j\mid S_{n}=j)&=:\mathbf{P}^{(k+m)}(j,j)\\ &=\sum_{r\in S}\mathbf{P}^{(k)}(j,r)\cdot\mathbf{P}^{(m)}(r,j) \\
&\ge\mathbf{P}^{(k)}(j,j)\cdot\mathbf{P}^{(m)}(j,j).\end{align}
If both $\mathbf{P}^{(k)}(j,j)>0$ and $\mathbf{P}^{(m)}(j,j)>0$, then $\mathbf{P}^{(k+m)}(j,j)>0.$ That will lead to $\mathbf{P}(S_{n+k+m}=j)>0,$ further steps will result in infinitely many $\mathbf{P}(S_{n}=j)>0.$What is your opinion on this?
I thought from the definition of time-homogeneous, $\mathbf{P}(S_{n+1}=j|S_{n}=i)=\mathbf{P}(S_{m+1}=j|S_{m}=i),\forall i,j\in S \& \forall n,m\in \mathbb{N},$ both $\mathbf{P}(S_{n})>0$ and $\mathbf{P}(S_{m})>0$ seem to be needed.
Answer: $S_n/n$ converges to some nonzero quantity a.s., so $|S_n|$ converges to infinity a.s., so for any fixed finite $a$, $|S_n|=|a|$ a finite number of times a.s, so $a$ is transient for any $a$.
Note that when the mean is $0$, the above approach doesn't work, and it might not be transient everywhere, so you have to be a little careful in generalizing this approach.


Understanding similarity sensitive hashing algorithm in AdaBoost

Question: I'm a CS major and don't quite understand the mathematics behind a optimization problem coming from a machine learning algorithm. The algorithm is in Section 5 of the paper http://visl.technion.ac.il/bron/publications/BroBroOvsGuiTOG10.pdf
Problem Statement
given $P$ pairs of examples $(f_p, {f_p}')$ labeled by {1,-1}, where $f_p$ is a v-dimensional feature vector and +1 (reps., -1) indicates similar (reps., dissimilar) pairs, the goal is to find the $s$X$v$ matrix A and $v$X1 vector b such that $d_{A,b}$ reflects the desired similarity of the training examples. The distance $d_{A,b}$ is defined as:
$d_{A,b}(x, x')=d_H(\mbox{sign}(Af+b), \mbox{sign}(Af'+b))$,
where $d_H(y,y')=\frac{s}{2}-\frac{1}{2}\sum_{i=1}^s\mbox{sign}(y_i{y_i}')$ is the Hamming metric in the s-dimensional Hamming space of binary sequences of length s.
Ideally, we would like to achieve $d_{A,b}(f,f')\leq d_0$ for similar pairs (P) and $d_{A,b} > d_0$ for dissimilar ones (N), where $d_0$ is some threshold. However, in practice ,there always exists false positives and false negatives. Thus, optimal A, b should miminize:
$min\frac{1}{P}\sum_{(f,f')\in P}e^{\mbox{sign}(d_{A,b}(f,f')-d_0)}+\frac{1}{N}\sum_{(f,f')\in N}e^{\mbox{sign}(d_0-d_{A,b}(f,f')}$
Algorithm
The learning of the optimal parameters A and b is posed as a boosted binary classification problem, where $\mbox{sign}(Af+b)$ acts as a strong binary classifier and each dimension of the linear projection $\mbox{sign}(A_ix+b_i)$ is a weak classifier. This way, the AdaBoost algorithm (http://www.cs.rochester.edu/users/faculty/stefanko/Teaching/09CS446/Boosting2.pdf)
can be used to progressively construct A and b. Intuitively, this algorithm increases the weights of incorrectly classified examples so that the learner is forced to focus on the hard examples in the training set.
Input: P pairs of examples $(f_p, {f_p}')$ labeled by $s_p$

Initialize ${w_p}^i=1/P$
For i = 1, ..., d do
Set the ith row of A and b by solving the optimization problem
    $(A_i, b_i)=\min \sum_{p=1}^P {w_p}^is_p(2-\mbox{sign}(A_if_p+b_i))(2-\mbox{sign}(A_i{f_p}'+b_i))$
Update weights
 ${w_p}^{i+1}={w_p}^ie^{-s_p\mbox{sign}(A_if_p+b_i)\mbox{sign}(A_i{f_p}'+b_i)}$

My Question
The optimization in the above algorithm is difficult, so the author tried to solve a simpler problem by setting
$A_i=\arg \max \frac{{A_i}^TC_NA_i}{{A_i}^TC_PA_i}$
where $C_P$ and $C_N$ are covariance matrices of the positive and negative example pairs
The author says that $A_i$ maximizing the above formula is the largest generalized eigenvector of ${C_P}^{-1/2}{C_N}^{1/2}$. Since this do not coincide exactly with the original optimization problem, the author select a subspace spanned by the largest 10 eigenvectors, out which direction and as well as the threshold parameter b minimizing the exponential loss are selected. I don't understand how the selection is done. Can anyone explain to me? Thanks a lot.
Comment: I placed a question at local Meta: http://meta.stats.stackexchange.com/questions/1010/is-this-the-correct-place-for-some-questions-that-confused-nearly-everyone-on-mo
Comment: user11869, I was just checking in to see how this was going. In particular, despite appearances, I wonder whether you will get an answer here. I will see if I can post on the Meta here, there does seem to be a reputation lower bound for that. From my count of numbers of questions with related tags, certainly there are those who are able to answer you. Will, http://mathoverflow.net/users/3324/will-jagy and http://math.stackexchange.com/users/10400/will-jagy
Comment: I really appreciate your help. Thanks! BTW, what is the Meta you mentioned above?
Comment: stats.stackexchange has its own discussion page for questions about policy. If you just click on http://meta.stats.stackexchange.com/questions/1010/is-this-the-correct-place-for-some-questions-that-confused-nearly-everyone-on-mo  you will be sent to my question. If you then click on the word QUESTIONS it will show you all the most recent discussions. At the bottom of the page, you may then click to any of the total of 19 pages of discussions to see more.
Answer: It seems like the exponential loss they refer to is:
$\sum_{f,f' \in N}\exp^{−sign(A_{i}f_p+b_i)sign(A_if'_p+b_i)}$
so I would imagine they simply take the argmin over $A \in \mathcal{A}$ and $b \in \mathcal{B}$ where $\mathcal{A}$ is the set of $A$ represented by the top ten eigenvectors, and $\mathcal{B}$ is some range (e.g. $b \in \left[-1,1\right]$, since $\left| A \right| \leq 1$ from the constrained eigendecomposition)
$\arg\min_{A \in \mathcal{A}, b \in \mathcal{B}} \sum_{f,f' \in N}\exp^{−sign(A_{i}f_p+b_i)sign(A_if'_p+b_i)}$
Comment: Thanks. Is b the eigenvalue associated with the eigenvector A?
Comment: Is the goal to minimize the exponential loss or maximize it? The paper said A and b should be chosen such that the exponential loss is minimized, while your goal is to maximize it. But anyway, the point here is to increase the weights of mis-classified examples, am I correct?
Comment: Sorry yes that should have been argmin - I've updated. And yes that is the point
Comment: And no, b doesn't enter into the eigenvalue decomposition. It's not quite clear they minimise over $b$ - I've updated the answer to give one idea how. Perhaps you could email the authors?
Comment: I actually did contact the author before I post my question here. But I haven't heard anything back yet


Decrementar um valor Mysql

Question: Existe algum atributo MySQL para Decrementar dia , mes , ano ou hora no próprio banco que seria automaticamente.
Ex: coluna-> valor inicial = 15, depois de um dia seria 14
O banco mysql faz isso ???
Comment: Realmente é um Problema XY, sua pergunta se prendeu no jeito que você achou que ia conseguir resolver, e não na súa dúvida real, que é "como bloquear um sistema após um mês de teste" (cuja resposta provavelmente nem passa pelo que está na sua pergunta, bastaria um `IF data > prazo de teste`). A solução realmente seria refazer totalmente a pergunta com sua dúvida real.
Comment: Melhor [edit] a postagem e explicar o que quer ao menos com um exemplo claro. Da forma como está, não dá para ter muita certeza do que está procurando. Inclusive por não fazer muito sentido uma operação de data ser contrária a um auto increment, sendo que a natureza do dado é bem diferente. Me parece que sua pergunta remete a um [Problema XY](http://meta.pt.stackoverflow.com/q/499)
Comment: Ai cara da para entender agora
Comment: @Bacco seria um exemplo um sistema pago depois de um mês de teste  é bloqueada a conta do usuário ate que ele pague.
Answer: De modo automático não tem, mas existe algumas abordagens para fazer isso.
A mais simples é implementar na própria query que faz o update, segue um exemplo:
UPDATE tabela SET
    campo1 = 'Teste',
    campoData = DATE_ADD(campoData, INTERVAL -1 DAY)
WHERE
    (id = 1);

Answer: Você pode criar uma trigger ou evento (Trigger só ocorre quando uma operação é acionada, tipo insert, delete, update) para executar esta operação diariamente, por exemplo:
CREATE EVENT event1
ON SCHEDULE EVERY '1' DAY
DO
-- sua operação aqui, por exemplo:
   UPDATE TABELA SET CAMPODATA = str_to_date( date_format(now(), '%Y%m%d 0200'), '%Y%m%d %H%i' ) - INTERVAL 1 DAY
END

Decrementar um valor Mysql


between ex4 and XFS which is better for large small files

Question: Between EXT4 and XFS which file system is better when an application uses multiple threads to read/write large amount of small files on a SSD.
From what I read

In general, Ext3 or Ext4 is better if an application uses a single
read/write thread and small files, while XFS shines when an
application uses multiple read/write threads and bigger files
Comment: Does this answer your question? [Can you help me with my capacity planning?](https://serverfault.com/questions/384686/can-you-help-me-with-my-capacity-planning)
Answer: Both EXT4 and XFS are very fast in small file handling. EXT4 is faster for single thread operations, while XFS scales better when using multiple threads/cores.
You can read more in RedHat documentation.


Trying to prove that the sequence: 3, 3, sin(1), 3, 3, sin(2), 3, 3, sin(3), 3, 3, sin(4), $\ldots$ does not converge to 3

Question: $\textbf{Proof}$:
I need to show that $\exists \epsilon > 0$, such that, $\forall N \in \mathbb{N},\ \exists n \ge N$, such that, $\ \left | a_n - 3 \right | \ge \epsilon$
Let $\epsilon = 2$, then if you take $n = 30$, we have that $a_{30} = sin(10) \approx -0.544 \ldots$
So we have that $ \left | sin(10) - 3 \right | \ge 2$
Therefore I have showed that, $\exists \epsilon > 0$, such that, $\forall N \in \mathbb{N},\ \exists n \ge N$, such that, $\ \left | a_n - 3 \right | \ge \epsilon$, where $\epsilon = 2$ and $n = 30$.
I am wondering if this proof is right.
Comment: You don't need to calculate (even approximately) values of sine. Just use the fact that $\sin x \le 1$ for all $x$.
Comment: I will remove the approximations, they do look rather ugly. I was wondering if general argument I've made is correct though?
Comment: Ok Andre, I see my mistake.
Comment: You didn't make a general argument yet. You chose correct $\varepsilon$, but for any $N$ you must show that the inequality is true for some $n > N$.
Answer: You have to find $\varepsilon>0$ such that, for all $N\in\mathbb N$, there exists $n\geq N$ such that $|a_n-3|\geq \varepsilon$. What you did will only work for $N\leq 30$, but, if for example $N=31$, you have to find an $n\geq 31$ such that $|a_n-3|\geq\varepsilon$.
Instead, you could do the following: given $N\in\mathbb N$, there exists $n\geq N$ such that $a_n=\sin k$ for some $k\in\mathbb N$. So, for that $n$, $|a_n-3|=|\sin k-3|\geq\left||3|-|\sin k|\right|\geq 3-|\sin k|\geq 2$.
Comment: Ok I see my mistake, and your argument makes sense. I was jumping gun and finding specific n's, when it should be for a given n, as you've said.
Answer: It suffices you show $\langle \sin n\rangle $ cannot converge to $3$. That is, if the sequence converged to $3$, then the subsequence $\sin 1,\sin 2,\ldots$ would converge to $3$. But for any $n$, we have that $\sin n \leqslant 1$. Thus, this is impossible. Note that if we replaced $3$ with $1$; things'd get much more delicate!
Comment: Oh ok, this is very clever. Thanks for your response Pedro.
Comment: @Michael No problem.


Laravel on VestaCP throws error 500. What file should I update to solve this problem?

Question: When i try visiting my site I get this error, when I look domain.com.error.log:
PHP Fatal error: require(): Failed opening required '/home/<user/web/<domain>/public_html/../vendor/autoload.php

Files from my laravel project are placed into domain folder, files from public folder are inserted into /public_html folder. I've tried composer update, generate key, and all the other composer commands that people adviced here before.
I've set all my directories permissions to 777, so this error cannot be because of lacking permissions.
If you need any additional information, please ask, trying to solve this problem for 4 days already.
Edit: Solved this problem, it appeared because php didn't have access to files that are not inside public_html. So, this can be solved by either putting all your project into public_html folder, or updating your server rules to allow php access the folder with the project.
Comment: don't give 777 permission for folder it `755` and for file it `644`
Answer: You need to install composer from www.getcomposer.org
composer dump-autoload
composer install --no-dev --no-interaction --prefer-dist --optimize-autoloader
php artisan cache:clear
php artisan route:cache
php artisan view:clear
php artisan migrate --force

Comment: its Working Dude?
Comment: In progress, a little problem with migration, but site still throws 500
Comment: Solved the problem with migration, but 500 error is still there(


How can I get a JSON status that my server is busy?

Question: My JSON Code:
<code>Response.Listener<String> responseListener = new Response.Listener<String>() {
                    @Override
                    public void onResponse(String response) {
                        try {
                            JSONObject jsonResponse = new JSONObject(response);
                            int success = jsonResponse.getInt("success");

                            if (success == 0) {

                            }
                            if (success == 1) {

                            }
                        } catch (JSONException e) {
                            e.printStackTrace();
                        }
                    }
                };
                nRequest nRequest = new nRequest(edtTextReg.getText().toString(), responseListener);
                RequestQueue queue = Volley.newRequestQueue(login.this);
                queue.add(nRequest);
</code>
How can I find a way that JSON give me the status "server is busy"? Busy means for example: thousands of people use my server.
Comment: That definition of busy is basically useless. How does your code know how many people are using the server?
Comment: This is my question. How can I find a way that JSON give me the status "server is busy"
Comment: Why do you need the status? What are your criteria?
Comment: I need this for a login request. The user need a Toast, that the Server is busy
Comment: If your server's really slammed you may need to do this at the HTTP server level and have a JSON "error document" configured.
Answer: Http status code for server overload is 503.
503 - Service Unavailable
A 503 status code is most often seen on extremely busy servers, and it indicates that the server was unable to complete the request due to a server overload.

so your code should be something like this:

<code>                @Override

                public void onResponse(String response) {
                    try {
                        JSONObject jsonResponse = new JSONObject(response);

                 int status = jsonResponse.getStatusLine().getStatusCode();
                      int success = jsonResponse.getInt("success");
                        if (status == 503) {
                          //
                        }

                        if (success == 0) {

                        }
                        if (success == 1) {

                        }
                    } catch (JSONException e) {
                        e.printStackTrace();
                    }
                }
            };
</code>
Comment: I get an error: "Cannot resolve Method 'getStatusLine()'"
Comment: For getting the status code correctly you could refer to this url: https://stackoverflow.com/questions/22948006/http-status-code-in-android-volley-when-error-networkresponse-is-null
Answer: If your server is busy you don't want it to spend time rendering a JSON, moreover, if server load is in your mind you might end up putting your application behind a load-balancer and those usually knows nothing about JSON.
The proper way to handle this situation is to parse the correct HTTP status code, usually 429, see this answer
To access the status code your onResponse should look like this:

<code>@Override
public void onSuccess(int statusCode, String content) {
}
</code>


Random walk on the edges of a square

Question: A bug is at one corner of a square. What's the
expectation of the number of steps it takes, to reach the opposite corner? Each step takes it to an
adjacent corner, with either corner equally likely.
Comment: See [this](https://stats.stackexchange.com/search?q=random+walk+cube) for ideas
Answer: What you describe is a discrete circular random walk with four points.  (The fact that it is arranged as a square instead of a circle does not affect the behaviour.)  Let $E_1$ and $E_2$ denote the expected number of steps to get to a point that is one or two points away from the starting point respectively.  Drawing the four points, and using the law of total probability conditioning on the first move clearly gives:
$$\begin{equation} \begin{aligned}
E_1 &= \frac{1}{2} \times 1 + \frac{1}{2} \times (1+E_2), \\[8pt]
E_2 &= \frac{1}{2} \times (1+E_1) + \frac{1}{2} (1+E_1).
\end{aligned} \end{equation}$$
Have a think about why these equations should hold, having regard to the two directions the bug could move on the first step.  Once you understand this, it should be quite simple to solve these two simultaneous equations to find the expected number of steps $E_2$.
Comment: Good point - that gets you $E_2$ very easily.
Comment: +1.  A little more thought suggests working two steps at a time, because the bug's position after an even number of steps can be only where it started or at the opposite corner, resulting in a geometric distribution for *half* the number of steps needed to reach the opposite corner.

Random walk on the edges of a square


How to loop through json object with specific key and value in javascript?

Question: I am looping through a json object with the structure below:
var my_products = {
    "products": [
        {
            "title": "Product 1",
            "id": 001,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 2",
            "id": 002,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 3",
            "id": 003,
            "stock_quantity": 1,
            "in_stock": true
        },
        {
            "title": "Product 4",
            "id": 004,
            "stock_quantity": 1,
            "in_stock": true
        }
    ]
};

I can loop through the object with the code below:
for(var i=0; i<my_products.products.length; i++) {
    var product = my_products.products[i];
    console.log(product);
    console.log('in_stock:',product.in_stock);
}

Do you know how can I loop through each record where Products that have a "stock_quantity": 1 and "in_stock": true properties?
In the example above there are 2 product records that have a "stock_quantity": 1 property(not zero quantity)
Answer: You can filter the array first:

var my_products = {
    "products": [
        {
            "title": "Product 1",
            "id": 001,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 2",
            "id": 002,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 3",
            "id": 003,
            "stock_quantity": 1,
            "in_stock": true
        },
        {
            "title": "Product 4",
            "id": 004,
            "stock_quantity": 1,
            "in_stock": true
        }
    ]
};
console.log(
  my_products.products.filter(({ stock_quantity, in_stock }) => (
    in_stock && stock_quantity === 1
  ))
);
Answer: Simply you can iterate and filter data with if condition.

var my_products = {
    "products": [
        {
            "title": "Product 1",
            "id": 001,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 2",
            "id": 002,
            "stock_quantity": 0,
            "in_stock": false
        },
        {
            "title": "Product 3",
            "id": 003,
            "stock_quantity": 1,
            "in_stock": true
        },
        {
            "title": "Product 4",
            "id": 004,
            "stock_quantity": 1,
            "in_stock": true
        }
    ]
};
var filterData = [];
$.each(my_products.products, function(){
    if (this.in_stock && this.stock_quantity === 1) {
        filterData.push(this);
    }
});
console.log(filterData);


Class imbalance in clustering

Question: Is there is a problem for clustering if the dataset is highly imbalanced?
I have a clustering task and it looks like that there is a realy huge peak whose tail covers other clusters.
Are there any techniques to deal with that?
Comment: if  you do not have label (unsupervised setting) how do you know the class is imbalanced?
Comment: I estimate density and then see that in one point there is a huge peak that by couple of orders of magnitudes greater than the rest of the density. If I draw a heatmap then I see just a red dot at this point and everything else is completely blue.
Comment: can you try to fit a mixture of Gaussian, by assigning prior with the class ratio?
Comment: The problem that i dont know exact class ratio. I even dont know the number of clusters. The only thing that I observe is that due to that peak we dont see other clusters since they are not so prominent.
Comment: I think you need to tell us more about your data. Given the imbalance, what is driving or underlying that? For instance, is the imbalance due to a lack of information that would aid in differentiating these entities? Or something else?
Answer: In general: yes, this could very well be problematic. Imagine you have a number of clusters of unknown, but different classes. Clustering is usually done using a distance measure between samples. Many approaches thereby implicitly assume that the clusters share certain properties, at least within certain boundaries - like distances between clusters only diverging within a certain maximum, or more likely the scale of cluster spread only diverging within a certain maximum. This can be problematic in case you e.g. have one prominent, largely and unequally scattered class, that to some extent shadows other, less prominent, and as well unequally scattered classes - which further have very different distances between clusters. This could lead to your not-so-prominent clusters to not be found at all, as they e.g. get just pushed away from the prominent cluster (e.g. K-means), or could end up at just some slightly-above-average area of the prominent class (e.g. SOM, to some extent). But if this is the case with your problem, clustering will pretty likely be quite difficult with any clustering approach.
Two thoughts about possible approaches:

If you don't have any idea about the class prevalence, changing the data/data weight (e.g. subsampling using the density observed in your data) might defeat the purpose of clustering (imagine the extreme scenario of flattening out the whole feature space, which means discarding the information you would need for building clusters). But it could be that there are scenarios where this makes sense.
If you have a rough idea of class prevalence, as @hxd1011 mentioned, using some weight for your clusters/distributions could be helpful. I guess that adapting the prevalence, using sampling techniques, the estimated prevalence, and the density observed in your data might be possible too (but keep in mind that when you use a mixed, observed density of different classes, your assumptions and simplifications might not be completely true, as mentioned in the first part of the answer).
Answer: https://www.researchgate.net/post/Can_anyone_recommend_algorithms_to_deal_with_unbalanced_clusters_for_classification
I think "try agglomerative ones (single link, complete link, Ward, etc.)" may be a good answer

Class imbalance in clustering


BASH Scripting, su to www-data for single command

Question: I am working on automating the creation of subversion repositories and associated websites as described in this blog post I wrote. I am running into issues right around the part where I su to the www-data user to run the following command:
svnadmin create /svn/repository

There is a check at the beginning of the script that makes sure it is running as root or sudo, and everything after that one command needs to be run as root. Is there a good way to run that one command as www-data and then switch back to root to finish up?
Answer: With 'su' is probably that request a password, and www-data doesn't have password. I recommend the command sudo:
 sudo -u www-data command

The condition is that your user must be root, or configurated in the sudoers file
Comment: Wow, why isn't this the selected answer?  Took me way too long to find this.
Comment: Yes, this is the easiest approach to running a command as www-data user.  Thanks!
Answer: Just use su - www-data -c 'svnadmin create /svn/repository' in your script run by root. So that only this command is run by www-data user.

Update for future viewers:
In case you receive a "This account is currently not available" error, consider using:
su - www-data -s /bin/bash -c 'svnadmin create /svn/repository'
( @Petr 's valuable mention about -s flag to accommodate www-data user's no login policy )
Comment: Use `su - www-data -s /bin/bash -c 'your_command'` to make this work. User www-data has shell `/usr/sbin/nologin` so without the `-s` parameter it leads to the error message.
Comment: Trying this, I get the error `This account is currently not available.`
Comment: Yeah, it appears I forgot a basic rule of scripting... RTFM. Thanks!
Comment: I also get this error, but futbolsalas15's answer below works fine.
Answer: 2 Possible approaches:

1) sudo command:
In most cases you will have access to the sudo command and so the solution is simply:
sudo -u target_user target_command

2) su command (if sudo not installed. Ex. alpine-linux images):
su - target_user -c 'target_command'
In case you receive a 'This account is currently not available' error, the user has a no login (shell access) policy in effect. If so, consider using:
su - target_user -s /bin/bash -c 'target_command'
(Based on @Petr 's valuable comment about -s flag to accommodate www-data user's no login policy)
Comment: the `sudo -u www-data target_command` version worked for me, thanks!
Answer: Use su:
   su [options] [username]

   The options which apply to the su command are:

   -c, --command COMMAND
       Specify a command that will be invoked by the shell using its -c.

Comment: It should be noted that su commands don't work with accounts that have disabled logins (such as www-data). You'll have to use sudo.


Drupal Commerce PayPal Express Checkout review and confirm step is skipped

Question: User is not seeing Confirm Order Page (Express Checkout review and confirm) during PayPal Express checkout.

The EC module defines a new checkout page called Confirm order between the payment redirect and completion pages and a checkout pane called Express Checkout review and confirm that should only be used on this special page.

I am using 7.x-2.3 release of Commerce PayPal module. Testing with PayPal Sandbox accounts.
After logging into PayPal customer sees "continue" button and a message *"You're almost done. You will confirm your payment on **'s Test Store."*. After clicking "continue" customer is brought right to the Order Complete page and the payment is registered in the system. Which is not the intended behavior of PayPal EC module.
These are Order history messages that have been recorded during checkout with PayPal express:
Status has been set to Checkout: Complete (previously: Checkout: Confirm order).
Status has been set to Checkout: Confirm order (previously: Checkout: Payment).
Payment of $62.00 submitted via PayPal Express Checkout.
Order has been paid in full.
Status has been set to Checkout: Payment (previously: Checkout: Review).
Status has been set to Checkout: Review (previously: Checkout: Checkout).
Status has been set to Checkout: Checkout (previously: Shopping cart).
Product sample2 added to the cart.
Order has been created.
QUESTION: What could be the reason of why Express checkout review and confirm page is skipped?
Answer: The only code that would trigger a skip of the order confirmation page is in the checkout router hook implementation commerce_paypal_ec_commerce_checkout_router(). If you take a look at that, you'll see that it skips the confirmation page if the order either doesn't have a record in its data array that Express Checkout was used or if the customer used EC but did not use the EC flow (as opposed to the Mark flow, which is when the customer chooses PayPal from the payment method checkout pane and is then redirected to PayPal):
<?php
  // If the current page is the Express Checkout page but the current order did
  // not use the Express Checkout flow...
  if ($checkout_page['page_id'] == 'paypal_ec' &&
    (empty($order->data['commerce_paypal_ec']['flow']) ||
     $order->data['commerce_paypal_ec']['flow'] != 'ec')) {
    // Update the order status to the next checkout page.
    // ...
  }
?>

This means if it's being skipped for you, you should first ensure it's being skipped when the EC button is being used from the cart form (that's the EC flow) as opposed to when the customer redirects from inside the checkout form.
The second place to look will be the order's data array to figure it if / why the order doesn't have a record that it was paid for via EC.
If the module itself is to blame, please open a bug report explaining how to reproduce the issue. Last I used it, it was all working fine.


groups, levels and denominator dof in mixed effect models

Question: I am trying very hard (I am not a statistician) to understand the concepts of "groups" and "levels" in mixed effect models. In particular, I am trying to understand this in the context of the denominator degrees of freedom of the fixed effect terms of a model. I know that denDF is an open question in mixed effect models, so for the sake of the current discussion I would refer to the estimated denDFs based on the inner-outer rule proposed by Pinheiro and Bates (pag. 91) and summarized here.
First off, can anyone define what "levels" and "groups" are in mixed effect model? My understanding is that groups are defined by the structure of the random effects (and therefore by the model). However, I am confused with the definition of levels. Also, is there any relationship between levels and groups? In other words, can I modify the number of groups (by changing the structure of the random effects) independently on the levels. Are levels inherently defined by the structure of the data, or they emerge from the definition of the model?
Regarding the definition of denDF. Pinheiro and Bates define:
$denDF_i = m_i -(m_{i-1}+p_i), i=1,...,Q+1$
where $m_i$ denotes the total number of groups in level $i$, $p_i$ denotes the number of degrees of freedom corresponding to the terms estimated at level $i$, and $Q$ is the number of grouping factors. My questions are:

This definition is highly dependent on the definition of levels. How do I know what groups belong to what level?
According to this definition, there seems to be $Q+1$ denDFs. But I can change the number of grouping factors by changing the structure of the random effects, while I always need as many denDFs as the number of fixed effect terms to be estimated plus 1 (the intercept). For example, in the Machines example from the book, I could define random=~1|Worker (Q=1, hence 2 denDFs) or random=~1|Worker\Machines (Q=2, hence 3 denDFs). Wouldn't I need only two denDFs in both models, Intercept and Machine? Why does this depend on the definition of the grouping?
I cannot understand the logic of this formula. Especially, why do I need to subtract $p_i$?

Finally, is there a rule of a thumb to figure out how to define groups (and therefore random effect structure) in a model? This seems to be a very important design decision: the definition of the groups influences the denDF, and therefore the results of the hypothesis tests on the fixed effect terms.
I do apologize for my naive questions, but I am really trying to understand this. If this is too basic for this forum, can you give me specific references where I can read about these issues? Thanks a lot for your help.
Answer: This is a great question and I am not surprised about the confusion.
I am writing this as an answer, as it is too long for a comment. I am only going to address the first question here. Perhaps the remaining questions will be redundant if it makes sense....

First off, can anyone define what "levels" and "groups" are in mixed effect model?

In short, there are "levels" of the grouping factor (the "groups") and there are "levels" of nesting (0,1,2 etc), which are completely different.
In a bit more detail, "groups" are the unique items within a "grouping factor" which is generally a categorical variable. For example, if we had observations on patients within a ward of a hospital, the "grouping factor" (just a variable name in the dataset) would be something like wardID. "groups" refers to the individual wards, so if there were 20 wards in the sample, then there would be 20 groups.
Things start to get confusing when talking about "levels". In Pinheiro and Bates they are discussing models with nested grouping factors, and when they talk about levels they are talking about the relative level of nesting. Sticking with the patients in wards scenario. If we now have multiple hospitals, then wards are nested within hospitals - so any individual ward "belongs" to one and only one hospital. By convention, the lower level (wards in this case) is 1. So the hospital level is level 2, if there was a further level of nesting, say City, that would be level 3. Hopefully that makes sense. The source of confusion is that within any particular grouping factor, the individual entities (eg individual hospitals) are also often referred to as "levels". So in some contexts, "groups" and "levels" can be used interchangeably. So, in R, if we refer back to the Machines example in nlme, and look at the structure of the Machine grouping variable we obtain:
> str(Machines$Machine)
 Factor w/ 3 levels "A","B","C": 1 1 1 1 1 1 1 1 1 1 ...

So it is necessary to understand the context of what they are talking about in the book. For instance when they say:

grouping level at which the term is estimated

they are talking about level 1 or level 2 of the nesting structure (wards or hospitals in my example of worker and machine in theirs). On the other hand, when they say:

A term is outer to a grouping factor if its value does not changes within levels of the grouping factor

they are talking about the individual levels (individual wards or hospitals) within a grouping factor.
I hope this helps !
Comment: This helps a lot. Thanks!!! I am still a bit confused, but I guess I need to digest this. The book says "A term is said to be estimated at level i, if it is inner to the i-1st grouping factor and outer to the i-th grouping factor". Here, they are using "grouping factor" as `wardID` in your example, and I understand the concept of inner and outer. But what is the i-th grouping factor? Are grouping factors ordered so I can identify the i-th? Also, the term Machine is said to be estimated at level 2. Shouldn't it be level 1 according to your convention of start counting from the lower level?
Comment: Now I am confused too ! This is partly why I focussed on groups vs levels an  didn't try to answer the other points!! Maybe someone such as @BenBolker can clarify the other points !


Listview.builder causes RenderFlex overflow

Question: I have the code below that I'm using to generate a list view
class PostListsWidget extends StatefulWidget {
  const PostListsWidget({Key? key}) : super(key: key);

  @override
  _PostListsWidget createState() => _PostListsWidget();

}

class _PostListsWidget extends State<PostListsWidget> {
  static const List<PostListModel> _latestPosts = [
    PostListModel(post: 'this is a test as.', votes: 453, like: 1, iconID: 0xe908),
    PostListModel(post: 'this is a test as', votes: 2, like: 2, iconID: 0xe904),
    PostListModel(post: 'this is a test a', votes: 324, like: 1, iconID: 0xe908),
    PostListModel(post: 'this is a test s', votes: 435, like: 3, iconID: 0xe90d),
  ];

  Widget _buildListItem(BuildContext context, PostListModel postList){
    return Container(
      margin: const EdgeInsets.all(20),
      padding: const EdgeInsets.all(10),
      decoration: BoxDecoration(
        color: Colors.white,
        borderRadius: BorderRadius.circular(8)
      ),
      child: Column(
        mainAxisSize: MainAxisSize.min,
        children: [
         Flexible(

          child: Container(
          padding: EdgeInsets.only(bottom: 10),
            child: Row(
              crossAxisAlignment: CrossAxisAlignment.start,
              children:  <Widget> [
                PentEmotionWidget(iconid:  postList.iconID),
                Expanded(child:   Text(
                    postList.post,
                    overflow: TextOverflow.clip,
                  )    )

              ],
            )  ,
          )
         ),
          Flexible(

            child: Container(
              child: Row(

                children:  <Widget> [
                 Expanded(child:
                   PentLikesCountWidget(votes: postList.votes),
                 ),
                  PentLikesActionWidget(like: postList.like)
                ],
                ),
            )
          )

        ],
     ),
    );
  }

  @override
  Widget build(BuildContext context) {
    return ListView.builder(
      itemCount: _latestPosts.length,
      scrollDirection: Axis.vertical,
      shrinkWrap: true,
      itemBuilder: (context, index) =>
        _buildListItem(context, _latestPosts[index]),
      );
  }
}

The code works but the last tile is cut

I've tried playing around with Expanded and Flexible but that doesn't make a difference. Seems like the problem is the List cannot scroll therefore it cuts the content overflow. Can anybody help?
Have to add random text here because SO won't let me post without adding more 'details'
Comment: Hi @mr_j  add Flex to flexible!
Comment: @PradipD. Tried that. Doesn't make a difference. I think I'll have to rewrite the way I've done it
Answer: If your layout is correct, then you only need to wrap you ListView.builder inside a Container/SizedBox and give it to some height
SizedBox(
  height: 200,
  child: ListView.builder(
   ...
  )
)

If that's not the case, maybe try not using expanded or flexible.
Comment: I've figured out what the problem is. I've put the Listview.builder widget inside a column in the main.dart file. The column cannot expand hence it causes the issue
Comment: Yeah, I thought so. cool.


Let $V$ be finite dimensional v.s. and $0 \ne T\in \mathscr L(V)$ , then $\exists$ $S \in \mathscr L(V)$ such that $0 \ne T \circ S$ is idempotent

Question: If $V$ is a finite dimensional vector space and $T \ne0$ is a linear operator on $V$ , then how may we prove that there is a linear operator $S$ on $V$ such that $T\circ S$ is non-zero and idempotent?
Answer: One can easily ensure $T\circ S\circ T=T$, from which $T\circ S\neq0$ and $(T\circ S)^2=T\circ S$ follow. Here's  how.
Choose a basis $b_1,\cdots,b_k$ of $\ker T$, and extend by $b_{k+1},\cdots,b_n$ to a basis of $V$. Then put $e_i=T(b_{i+k})$ for $i=1,\ldots,n-k$, which are linearly independent, since if some linear combination of those $e_i$ is zero, then the corresponding linear combination of the $b_{i+k}$ is in $\ker T$, which given the choices made is only possible for the trivial linear combination. One can then extend by $e_{n-k+1},\ldots,e_n$ to a basis of$~V$. Now define $S$ on the basis $e_1,\ldots,e_n$ by $S(e_i)=b_{i+k}$ for $i\leq n-k$, and $S(e_i)$ arbitrary (for instance zero) for $i>n-k$. One immediately checks $(T\circ S\circ T)(b_i)=T(b_i)$ for all $i$, whence $T\circ S\circ T=T$ by linearity.
Note that this also works for any linear $T:V\to W$ giving $S:W\to V$; in the argument one just needs to adjust the number of basis vectors $e_i$ to the dimension of$~W$ rather than$~V$.
Answer: Hint: Suppose $(c,v)$ is an eigenpair of $T$. Consider $T \circ P_v$, where $P_v$ is the orthogonal projector onto the span of $v$. When $c=1$ or $c=0$, I claim this is idempotent. How do you fix it otherwise?
Comment: Not at all. As I remarked in my answer it applies to maps between different vector spaces as well as to endomorphisms; there is no relation at all with eigenvalue problems.
Comment: It was not given that the vector space is over the complex numbers.
Comment: @MarcvanLeeuwen While you are right, I think this construction exposes the basic idea of your construction.
Comment: @MarcvanLeeuwen I disagree. Ultimately we are both setting up the problem so that $TS$ is the identity on a subspace $A \subset W$ and the constant zero map on the subspace complement of $A$. My setup is less general, but gives a more straightforward geometric way to choose $A$.
Comment: Have it as you like. But solving eigenvalue equations is really not necessary to find $S$.


The constructor Service(URL, QName, WebServiceFeature[]) is undefined

Question: Estou consumindo um WebService em minha aplicação Java e estou encontrando o seguinte erro:

The constructor Service(URL, QName, WebServiceFeature[]) is undefined

public AnaliseCredito(URL wsdlLocation, WebServiceFeature ... features) {
    super(wsdlLocation, SERVICE, features);
}

O problema é que estou utilizando o JAX-WS 2.2 que, segundo a documentação, possui o construtor em questão.
Inclusão da dependência no meu pom.xml:
<dependency>
    <groupId>javax.xml.ws</groupId>
    <artifactId>jaxws-api</artifactId>
    <version>2.2</version>
</dependency>

Comment: sua classe estende a classe  javax.xml.ws.Service?
Comment: Sim, @adelmo00.
Comment: Já tive exatamente o mesmo problema. Qual a versão da JDK que vc esta utilizando?
Comment: JDK8, @AlexandreStrevenski.
Comment: Verifica no java build path se o projeto esta utilizando a JDK 8, pois eu tive este problema, quando estava utilizando a JDK 1.6_15 e troquei para 1.7, gerei novamente os stubs e funcionou, depois pesquisei na internet e descobri que a causa do problema é a versão do JAX-WS, estava utilizando 2.0 porém os stubs precisavam de classes que existem na 2.1 (que não estava implementada no JDK que eu utilizava).
Comment: Detectei que o projeto está usando a `jaxws-api 2.1` que tem incluída na `jaxws-rt 2.2.10`, vou testar fazer um ``.
Comment: @EduardoSilva avisa se deu certo, mas acredito que seja versão do JDK.
Comment: Vamos [continuar esta discussão no chat](http://chat.stackexchange.com/rooms/30818/discussion-between-alexandre-strevenski-and-eduardo-silva).
Answer: O problema era que estou utilizando Apache CXF,  que registra a si mesmo como o provider JAX-WS na JVM, dessa forma mesmo que eu tenha o jaxws-rt : 2.2 em meu classpath/pom ele não será utilizado, a correção foi criar o arquivo
/src/main/resources/META-INF/services/javax.xml.ws.spi.Provider

com o seguinte conteúdo
org.apache.cxf.jaxws.spi.ProviderImpl

Veja também: http://docs.oracle.com/javaee/7/api/javax/xml/ws/spi/Provider.html#provider()


Como somar intervalos específicos de várias linhas?

Question: Tenho um arquivo com várias linhas e desejo somar determinado intervalo considerando todas as linhas. Para simplificar, tomei como exemplo:
AAA0000011111000090011
BBB0000011111000080011
CCC0000011111000070011

São várias linhas a mais e campos também, mas conheço o intervalo que desejo somar. Caso queira somar "00009", "00008" e "00007" que iniciam na posição 14 e terminam na posição 18 qual é a forma mais eficiente? Também desejo somar "0011", "0011" e "0011". Quero escrever em um novo arquivo os campos.
O resultado seria um novo arquivo com as somas:
24
33

Pensei da seguinte maneira:
file_name = teste.GS3
arquivo = open(file_name, "r")
TOTAL_CHARGEABLE_UNITS = 0
DATA_VOLUME_OUTGOING = 0

i = True
for line in arquivo:
    if i: # para pular a primeira linha
        i = False
        continue
    TOTAL_CHARGEABLE_UNITS = TOTAL_CHARGEABLE_UNITS + sum(line[14:18] + ...)
    DATA_VOLUME_OUTGOING = DATA_VOLUME_OUTGOING + sum(line[19:22] + ...)
arquivo.close()

arquivo = open("/dados/cdrs-roaming/resultado.txt", "w")
arquivo.writelines([TOTAL_CHARGEABLE_UNITS],[DATA_VOLUME_OUTGOING])
arquivo.close

TOTAL_CHARGEABLE_UNITS e DATA_VOLUME_OUTGOING são os campos desejados nesse caso. Sou iniciante então não tive ideia de como criar essa soma. Alguma ideia?
Detalhe: acredito que eu deva converter esses campos/intervalos para float, mas não sei como fazer isso sem converter o arquivo inteiro.
Comment: Um negócio que pod te ajudar é, a partir de cada linha, criar um objeto de alto nível. Aí, com uma lista desses objetos você opera nas propriedades deles.  Não é  a melhor opção se é algo que você precisa fazer uma vez só e nunca mais olhar, mas, se esses dados são a base do um projeto maior, aí pode valer a pena
Dê uma olhada nessa resposta: https://pt.stackoverflow.com/questions/399778/como-extrair-as-informa%c3%a7%c3%b5es-de-um-arquivo-cnab-usando-python/400033#400033
Answer: Como você está percorrendo uma linha de cada vez, você terá apenas um valor a cada iteração, então não faz sentido usar sum. Apenas atualize as variáveis a cada iteração:
total_chargeable_units = 0
data_volume_outgoing = 0
with open('arq.txt') as arq:
    # se quiser pular a primeira linha
    next(arq)

    for linha in arq:
        total_chargeable_units += int(linha[13:18])
        data_volume_outgoing += int(linha[18:22])

Se quiser pular a primeira linha, basta chamar next no arquivo antes do for (e não usar o retorno para nada, assim a primeira linha será ignorada).
Eu converti os valores para int pois são números inteiros (não vi necessidade de usar float). E dentro do loop eu atualizo as somas. Repare que no slice eu coloquei 13:18 (pois a primeira posição é zero, então o décimo quarto caractere estará na posição 13, e a última posição - no caso, 18 - não é inclusa).
Depois, bastas escrever no arquivo de resultados:
with open('resultado.txt', 'w') as out:
    out.write(f'{total_chargeable_units}\n{data_volume_outgoing}')

Tanto para ler quanto para escrever os arquivos eu usei with, que garante que o arquivo é fechado ao final (assim você não precisa chamar close).

Se quiser, também pode generalizar, criando um dicionário com os nomes das variáveis e suas respectivas posições:
# guarda as posições inicial e final de cada variável
posicoes = {
    'total_chargeable_units': (13, 18),
    'data_volume_outgoing': (18, 22)
}

results = {}
with open('arq.txt') as arq:
    # se quiser pular a primeira linha
    next(arq)

    for linha in arq:
        # lê todas as posições e atualiza o valor das variáveis
        for variavel, (inicio, fim) in posicoes.items():
            try:
                results[variavel] = results.get(variavel, 0) + int(linha[inicio:fim])
            except ValueError:
                # se não tiver um número, mostra mensagem de erro
                print(f'Valor nas posições [{inicio}:{fim}] não é um número')

# grava tudo no arquivo
with open('resultado.txt', 'w') as out:
    for qtd in results.values():
        out.write(f'{qtd}\n')

Também incluí uma validação, caso o arquivo não tenha um número nas referidas posições.
O resultado será outro dicionário, contendo os nomes das variáveis e seus respectivos totais.
Comment: Ajudou bastante! Ainda não alterei conforme sua segunda sugestão, mas a partir da sua ideia surgiu o código que deixei nas respostas
Answer: Desenvolvido a partir da resposta do colega hkotsubo:
Ajudou bastante! Ainda não alterei conforme sua segunda sugestão, mas a partir da sua ideia surgiu:
sequencia = ""
imsi = ""
msisdn = ""
operadora = ""
total_chargeable_units = 0
data_volume_outgoing = 0
gprs_apn_point = ""
with open('teste.GS3') as arq:
    # se quiser pular a primeira linha
    next(arq)

    for linha in arq:
        if imsi == linha[3:18]:
            msisdn = linha[188:209]
            operadora = linha[25:30]
            total_chargeable_units += int(linha[147:159])
            data_volume_outgoing += int(linha[159:171])
            gprs_apn_point = linha[237:300]
        else:
            sequencia = sequencia + imsi + ' ' + msisdn + ' ' + operadora + ' ' + str(total_chargeable_units) + ' ' + str(data_volume_outgoing) + ' ' + gprs_apn_point + '\n'
            imsi = linha[3:18]
            msisdn = linha[188:209]
            operadora = linha[25:30]
            total_chargeable_units = 0
            data_volume_outgoing = 0
            gprs_apn_point = linha[237:300]

    sequencia = sequencia + imsi + ' ' + msisdn + ' ' + operadora + ' ' + str(total_chargeable_units) + ' ' + str(data_volume_outgoing) + ' ' + gprs_apn_point + '\n'

with open('resultado.txt', 'w') as out:
    out.write(sequencia)


How do I sort by node authored on date?

Question: In Drupal 7, I have created a view to show a list of nodes I need to sort by node authored on date.
Does anybody know how to do this?
Comment: Will the "Post date" field work for you? Every node has this date automatically added when first saved with this date by default.
Answer: In your view, go to section: SORT CRITERIA.
Add field "Content: Post date"
Sort ascending or Sort descending.
Answer: [!This is a picture of Views UI]
Edit the View and go to (SORT CRITERIA) and Checked (Content: Post date)
[1]: https://i.stack.imgur.com/5xq2F.png

How do I sort by node authored on date?


References containing arguments against null hypothesis significance testing?

Question: In the last few years I've read a number of papers arguing against the use of null hypothesis significance testing in science, but didn't think to keep a persistent list. A colleague recently asked me for such a list, so I thought I'd ask everyone here to help build it. To start things off, here's what I have so far:

Johansson (2011) "Hail the impossible: p-values, evidence, and likelihood."
Haller & Kraus (2002) "Misinterpretation of significance: A problem students share with their teachers."
Wagenmakers (2007) "A practical solution to the pervasive problem of p-values."
Rodgers (2010) "The epistemology of mathematical and statistical modeling: A quiet methodological revolution."
Dixon (1998) "Why scientists value p-values."
Glover & Dixon (2004) "Likelihood ratios: a simple and flexible statistic for empirical psychologists."
Comment: It's not 100% relevant, so I'm not posting this as an answer. But, J. P. A. Ioannidis, [Why most published research findings are false](http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124), *PLoS Med* **2** (8): e124, 2005. is worth a read.
Comment: A famous book among economists is Ziliak, Stephen T., and Deirdre Nansen Mccloskey. The cult of statistical significance: How the standard error costs us jobs, justice and lives. University of Michigan Press, 2008.
Comment: What's the objective here?
Comment: *A lot* of references are listed on this page (including comments): http://lesswrong.com/lw/g13/against_nhst. More references in Glen_b's answer here: http://stats.stackexchange.com/questions/142533.
Comment: In March 2016, the ASA issued an official ["Statement on p-Values"](http://amstat.tandfonline.com/doi/pdf/10.1080/00031305.2016.1154108).
Answer: Chris Fraley has taught a whole course on the history of the debate (the link seems to be broken, even though it's still on his official site; here is a copy in Internet Archive).  His summary/conclusion is here (again, archived copy). According to Fraley's homepage, the last time he taught this course was in 2003.
He prefaces this list with an "Instructor's bias":

Although my goal is to facilitate lively, deep, and fair discussions on the issues at hand, I believe that it is necessary to make my bias explicit from the outset. Paul Meehl once stated that "Sir Ronald [Fisher] has befuddled us, mesmerized us, and led us down the primrose path. I believe that the almost universal reliance on merely refuting the null hypothesis as the standard method for corroborating substantive theories in the soft areas is a terrible mistake, is basically unsound, poor scientific strategy, and one of the worst things that ever happened in the history of psychology." I echo Meehl's sentiment. One of my goals in this seminar is to make it clear why I believe this to be the case. Furthermore, I expect you, by the time you have completed this seminar, to be able to articulate and defend your stance on the NHST debate, regardless of what that stance is.

I'll copy in the reading list in case the course page ever disappears:

Week 1. Introduction: What is a Null Hypothesis Significance Test? Facts, Myths, and the State of Our Science

Lyken, D. L. (1991). What's wrong with psychology? In D. Cicchetti & W.M. Grove (eds.), Thinking Clearly about Psychology, vol. 1: Matters of Public Interest, Essays in honor of Paul E. Meehl (pp. 3 – 39). Minneapolis, MN: University of Minnesota Press.

Week 2. Early Criticisms of NHST

Meehl, P. E. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806-834.
Rozeboom, W. W. (1960). The fallacy of the null hypothesis significance test. Psychological Bulletin, 57, 416-428.
Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437. [optional]

Week 3. Contemporary Criticisms of NHST

Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-1003.
Gigerenzer, G. (1993). The superego, the ego, and the id in statistical reasoning. In G. Keren & C. Lewis (Eds.), A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 311-339). Hillsdale, NJ: Lawrence Erlbaum Associates.
Schmidt, F. L. & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In Lisa A. Harlow, Stanley A. Mulaik, and James H. Steiger (Eds.) What if there were no significance tests? (pp. 37-64). Mahwah, NJ: Lawrence Erlbaum Associates.
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley. (Chapter 2 [A Critique of Significance Tests]) [optional]

Week 4. Rebuttal: Advocates of NHST Come to Its Defense

Frick, R. W. (1996). The appropriate use of null hypothesis testing. Psychological Methods, 1, 379-390.
Hagen, R. L. (1997). In praise of the null hypothesis statistical test. American Psychologist, 52, 15-24.
Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
Wainer, H. (1999). One cheer for null hypothesis significance testing. Psychological Methods, 6, 212-213.
Mulaik, S. A., Raju, N. S., & Harshman, R. A. (1997). There is a time and place for significance testing. In Lisa A. Harlow, Stanley A. Mulaik, and James H. Steiger , Eds. What if there were no significance tests? (pp. 65-116). Mahwah, NJ: Lawrence Erlbaum Associates. [optional]

Week 5. Rebuttal: Advocates of NHST Come to Its Defense

Abelson, R. P. (1997). On the surprising longevity of flogged horses: Why there is a case for the significance test. Psychological Science, 8, 12-15.
Krueger, J. (2001). Null hypothesis significance testing: On the survival of a flawed method. American Psychologist, 56, 16-26.
Scarr, S. (1997). Rules of evidence: A larger context for the statistical debate. Psychological Science, 8, 16-17.
Greenwald, A. G., Gonzalez, R., Harris, R. J., & Guthrie, D. (1996). Effect sizes and p values: What should be reported and what should be replicated? Psychophysiology, 33, 175-183.
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301. [optional]
Harris, R. J. (1997). Significance tests have their place. Psychological Science, 8, 8-11. [optional]

Week 6. Effect Size

Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage. [Ch. 2, Defining Research Results]
Chow, S. L. (1988). Significance test or effect size? Psychological Bulletin, 103, 105-110.
Abelson, R. P. (1985). A variance explanation paradox: When a little is a lot. Psychological Bulletin, 97, 129-133. [optional]

Week 7. Statistical Power

Hallahan, M., & Rosenthal, R. (1996). Statistical power: Concepts, procedures, and applications. Behaviour Research and Therapy, 34, 489-499.
Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153. [optional]
Maddock, J. E., Rossi, J. S. (2001). Statistical power of articles published in three health-psychology related journals. Health Psychology, 20, 76-78. [optional]
Thomas, L. & Juanes, F. (1996). The importance of statistical power analysis: An example from Animal Behaviour. Animal Behaviour, 52, 856-859. [optional]
Rossi, J. S. (1990). Statistical power of psychological research: What have we gained in 20 years? Journal of Consulting and Clinical Psychology, 58, 646-656. [optional]
Tukey, J. W. (1969). Analyzing data: Sanctification or detective work? American Psychologist, 24, 83-91. [optional]

Week 8. Confidence Intervals and Significance testing

Gardner, M. J., & D. G. Altman. 1986. Confidence intervals rather than P values: Estimation rather than hypothesis testing. British Medical Journal, 292, 746-750.
Cumming, G., & Finch, S. (2001). A primer on understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61, 532-574.
Loftus, G. R., & Masson, M.E.J. (1994). Using confidence intervals in within-subject designs. Psychonomic Bulletin and Review, 1, 476-490.

Week 9 [note: we are skipping this section]. Theoretical Modeling: Developing Formal Models of Natural Phenomena

Haefner, J. W. (1996). Modeling biological systems: Principles and applications. New York: International Thomson Publishing. (Chapters 1 [Models of Systems] & 2 [The Modeling Process])
Loehlin, J. C. (1992). Latent variable models: An introduction to factor, path, and structural analysis. Hillsdale, NJ: Lawrence Erlbaum Associates. (Chapter 1 [Path models in factor, path and structural analysis], p. 1-18]
Grant, D. A. (1962). Testing the null hypothesis and the strategy of investigating theoretical models. Psychological Review, 69, 54-61. [optional]
Binder, A. (1963). Further considerations on testing the null hypothesis and the strategy and tactics of investigating theoretical models. Psychological Review, 70, 107-115. [optional]
Edwards, W. (1965). Tactical note on the relations between scientific and statistical hypotheses. Psychological Bulletin, 63, 400-402. [optional]

Week 10. What is the Meaning of Probability? Controversy Concerning Relative Frequency and Subjective Probability

Salsburg, D. (2001). The lady tasting tea: How statistics revolutionized science in the twentieth century. New York: W. H. Freeman. (Chapters 10, 11, & 12)
Oakes, M. (1986). Statistical inference: A commentary for the social and behavioral sciences. New York: Wiley. (Chapters 4, 5, & 6)
Pruzek, R. M. (1997). An introduction to Bayesian inference and its applications. In Lisa A. Harlow, Stanley A. Mulaik, and James H. Steiger , Eds. What if there were no significance tests? (pp. 287-318). Mahwah, NJ: Lawrence Erlbaum Associates.
Rindskoph, D. M. (1997). Testing "small," not null, hypothesis: Classical and Bayesian Approaches. In Lisa A. Harlow, Stanley A. Mulaik, and James H. Steiger (Eds). What if there were no significance tests? (pp. 319-332). Mahwah, NJ: Lawrence Erlbaum Associates.
Edwards, W., Lindman, H., Savage, L. J. (1963). Bayesian statistical inference for psychological research. Psychological Review, 70, 193-242. [optional]

Week 11. Theory Appraisal: Philosophy of Science and the Testing and Amending of Theories

Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1, 108-141.
Roberts, S. & Pashler, H. (2000). How persuasive is a good fit? A comment on theory testing. Psychological Review, 107, 358-367.

Week 12. Theory Appraisal: Philosophy of Science and the Testing and Amending of Theories

Urbach, P. (1974). Progress and degeneration in the "IQ debate" (I). British Journal of Philosophy of Science, 25, 99-125.
Serlin, R. C. & Lapsley, D. K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist, 40, 73-83.
Dar, R. (1987). Another look at Meehl, Lakatos, and the scientific practices of psychologists. American Psychologist, 42, 145-151.
Gholson, B. & Barker, P. (1985). Kuhn, Lakatos, & Laudan: Applications in the history of physics and psychology. American Psychologist, 40, 755-769. [optional]
Faust, D., & Meehl, P. E. (1992). Using scientific methods to resolve questions in the history and philosophy of science: Some illustrations. Behavior Therapy, 23, 195-211. [optional]
Urbach, P. (1974). Progress and degeneration in the "IQ debate" (II). British Journal of Philosophy of Science, 25, 235-259. [optional]
Salmon, W. C. (1973, May). Confirmation. Scientific American, 228, 75-83. [optional]
Meehl, P. E. (1993). Philosophy of science: Help or hindrance? Psychological Reports, 72, 707-733. [optional]
  Manicas. P. T., & Secord, P. F. (1983). Implications for psychology of the new philosophy of science. American Psychologist, 38, 399-413. [optional]

Week 13. Has the NHST Tradition Undermined a Non-Biased, Cumulative Knowledge Base in Psychology?

Cooper, H., DeNeve, K., & Charlton, K. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2, 447-452.
Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129.
Greenwald, A. G. (1975). Consequences of prejudice against the null hypothesis. Psychological Bulletin, 82, 1-20.
Berger, J. O. & Berry, D. A. (1988). Statistical analysis and illusion of objectivity. American Scientist, 76, 159-165.

Week 14. Replication and Scientific Integrity

Smith, N. C. (1970). Replication studies: A neglected aspect of psychological research. American Psychologist, 25, 970-975.
Sohn, D. (1998). Statistical significance and replicability: Why the former does not presage the latter. Theory and Psychology, 8, 291-311.
Meehl, P. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66, 195-244.
Platt, J. R. (1964). Strong Inference. Science, 146, 347-353.
Feynman, R. L. (1997). Surely you're joking, Mr. Feynman! New York: W. W. Norton. (Chapter: Cargo-cult science).
Rorer, L. G. (1991). Some myths of science in psychology. In D. Cicchetti & W.M. Grove (eds.), Thinking Clearly about Psychology, vol. 1: Matters of Public Interest, Essays in honor of Paul E. Meehl (pp. 61 – 87). Minneapolis, MN: University of Minnesota Press. [optional]
Lindsay, R. M. & Ehrenberg, A. S. C. (1993). The design of replicated studies. The American Statistician, 47, 217-228. [optional]

Week 15. Quantitative Thinking: Why We Need Mathematics (and not NHST per se) in Psychological Science

Aiken, L. S., West, S. G., Sechrest, L., & Reno, R. R. (1990). Graduate training in statistics, methodology, and measurement in psychology: A survey of Ph.D. programs in North America. American Psychologist, 45, 721-734.
Meehl, P. E. (1998, May). The power of quantitative thinking. Invited address as recipient of the James McKeen Cattell Award at the annual meeting of the American Psychological Society, Washington, DC.
Comment: +1.  It is interesting--and quite telling--that the readings start and end with Meehl.
Comment: You've done a real service in sharing this.
Comment: The links to the course and to the summary seem to be broken. You can find the info through the Archive project : https://web.archive.org/web/20151023151618/http://www.uic.edu/classes/psych/psych548/fraley/NHSTsummary.htm
Answer: These are excellent references.  I have a perhaps useful handout at http://hbiostat.org/bayes
Answer: 402 Citations Questioning the Indiscriminate Use of
Null Hypothesis Significance Tests in Observational Studies:
http://warnercnr.colostate.edu/~anderson/thompson1.html


Using function context/variables without function arguments

Question: I'm trying to implement my own template literals functionality (in educational purposes), but I can't understand how it works.
My idea is to extend String.prototype with function, that will eval every ${} sequence inside the string. The problem is my new function don't know context (variables inside the string). Here's how I think it should work:
String.prototype.smart_eval = function() { /* find all ${} and eval them */ }

function some_function() {
  let a = 1;
  let b = 2;
  return 'A is ${a}, B is ${b}, the sum is ${a + b}'.smart_eval()
}

This will cause Uncaught ReferenceError: a is not defined since a and b belong to some_function(), not smart_eval(). Is there any elegant way to solve this without using function arguments or .call() / .apply()?
Answer: So... It seems I want something, that is not possible. My first idea to solve this as smooth as possible is Python-like function format. I don't qualify this piece of code as 100% working solution, but maybe somebody will find it usable.
String.prototype.format=function(values) {
  let s = this;
  let r = '';
  for (let i = 0; i < s.length; i++) {
    if (s[i] == '{') {
      for (let t = i+1; t < s.length; t++) {
        if (s[t] == '}') {
          let d = s.substring(i + 1, t);
          try { d = parseInt(d) }
          catch(err) { d = false }
          finally {
            if (d !== false) {
              if (values.length >= d + 1) { r += values[d]; i = t; break; }
              else { throw new ReferenceError('Literal is out of values range.') }
            }
          }
        }
      }
    } else {
      r += s[i] || ''
    }
  }
  return r
}

Examples:
'Hello {0}!'.format(['there']) // output: 'Hello there!'
'G{0}n{0}ral K{0}n{1}.'.format(['e', 'obi']) // output: 'General Kenobi.'

If you don't need IE11 support and some other browsers you can use rest parameters in function definition:
String.prototype.format=function(...values) {

And call it in more convenient way (without wrapping parameters into array):
'{1} wanna {0}? Let\'s {0}!'.format('play', 'You') // output: 'You wanna play? Let's play!'

Comment: code is pretty complicated... simple reg exp is how most people would have solved it. https://stackoverflow.com/questions/36994853/javascript-to-replace-variable-in-a-string


Little minor parts motion exercise (Exercise 1 from chapter 4 Piston)

Question: I'm studying some minor harmony and I've done this little exercise with all chords in root position

My question is: since In a minor setting you have multiple choices about triads (whether you are considering melodic/harmonic/natural minor), are you basically free to choose which triad to use (from a specific scale?)
For example in 3.4 I start with C minor triad, then I go for Eb augmented triad (considering the melodic minor ascending) and then I go for F minor considering the natural minor (having the flat sixth).
So is it possible to switch between melodic/harmonic/natural choices?
(I still have to do the 3.5)
Answer: You can certainly switch between melodic and harmonic.  In example 3.4 you even seem to be switching between C minor and C major!  Though I suspect you've just forgotten to notate a few flats.
I like the melody line C, B♮, A♭  with its astringent augmented 2nd leap, but in the context of a harmony exercise I fear it might be considered a fault!
You realise that restricting yourself to diatonic notes, those in the various forms of the C minor scale, is a very artificial situation?  Music just doesn't DO that.  There's no 'rule' (breakable or otherwise) about staying diatonic.   Perhaps there are some about voice leading and resolution of harmonic tensions.
Comment: You've got to know the rules first, in order to break them later.
Comment: But it's a good place to start. One bite at a time?
Comment: Thank you so much, and yes I forgot to notate some flats! Lol, also, thank you for noticing me an augmented second leap, yes it is considered a mistake in a traditional harmony exercise, as said in Piston. Gonna fix it in a bit. Thank you very much for the feedback !