Eigenvalues of correlation matrices exhibit exponential decay

Question: I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential decaying function. That is, there is a good fit linear of from $i=1..N$ to $\log|\lambda_i|$. Moreover, for several data sets I found that the exponent of the decay is fairly constant.
Is this a well-known fact or just one's tendency for finding patterns?
Obviously from PCA / SVD it tells me something about the ability to approximate the data using low-dimensional matrix.
Are there any solid mathematical results on the size of the exponent of this decay?
Comment: To support what @random said, consider the fact that you can begin with literally *any* non-increasing sequence of $N$ non-negative values, form an $N$ by $N$ matrix with their square roots on its diagonal, and right- and left-multiply it by suitable orthogonal matrices to produce an $N$ by $M$ data matrix whose PCA has the original sequence for its eigenvalues. Thus there cannot be a universal law describing the decay for *all* datasets. In the spirit of Benford's Law, one might nevertheless contemplate the possibility that many actual datasets could exhibit such behavior.
Comment: @whuber: and they do! I observed it many times myself, and have always assumed that for a random covariance (i.e. positive semi-definite) matrix the spectrum will show exponential decay (maybe in the limit of large $N$?), but I am not aware of precise mathematical results about it. Would be very interesting to know.
Comment: @Amoeba and Uri: We need to be careful about is meant by "exponential decay." The PCA results will order the eigenvalues by decreasing magnitude, so of course the resulting scree plot will look initially like *some* kind of decay. It would be of interest to formally test the exponential model of decay for goodness of fit. I'm pretty sure that in most applications of PCA the goodness of fit will be poor overall, bringing us to a crucial issue: if you are allowed to "eyeball" the plot and perform the fit only for the first "few" eigenvalues, exponential decay might be a self-fulfilling prophecy.
Comment: It may also be worth mentioning that the physicists have been interested in questions about the distributions of eigenvalues of random symmetric (or hermitean) matrices and have obtained quantitative (asymptotic) results.  Google "distribution eigenvalues random matrices" or "Wigner surmise" for more.
Comment: @whuber: The first few eigenvalues actually tend to deviate from the overall exponential decay, it is the "bulk" of the spectrum that fits to it very well. Right now I generated a random matrix (each value from standard Gaussian) of 5000*500 size and computed the spectrum of its covariance matrix. [Here is the result](http://i.imgur.com/IczLZ80.png). So I strongly suspect that the reason we see this behaviour in real datasets is simply due to noise.
Comment: @amoeba That is exactly right--but whether it is exponential decay or something else depends on the nature of the noise.  I suspect it will come closest to being exponential in the tail provided the columns of the data matrix are first standardized: in other words, you have to do the PCA based on correlation rather than covariance.
Comment: Indeed the experiment of @amoeba seems to be described by the general [Marchenko–Pastur distribution](http://en.wikipedia.org/wiki/Marchenko%E2%80%93Pastur_distribution), and for $\lambda=1$ (in the wikipedia notation, this is not an eigenvalue) we get the following measure on $x\in[0,4]$: $\mu(x)=\sqrt{4x-x^2}/(2\pi*x)$ and it is exponential-like...
Comment: This [answer](http://stats.stackexchange.com/a/7870/1036) discusses a Wachter plot - which is just a QQ plot of the eigenvalues you mention against a [Marchenko–Pastur distribution](http://en.wikipedia.org/wiki/Marchenko%E2%80%93Pastur_distribution).
Comment: I don't think there is something like a law of exponential decreasing eigenvalues for PCA. The eigenvectors are orthogonal to each other and the eigenvector corresponding to the largest eigenvalue just gives you the "main direction" of your point cloud. Therefore, I think it depends on the data. If there is no "strong main direction" in the data one can get probably linearly decreasing eigenvalues for the first two or three eigenvectors (principal components).
Comment: I see the point of @whuber that arbitrary matrix will not have this phenomena, but it does seem like a repeating tendency in real, rich datasets. Thus the question stand...
Comment: Indeed, I verified for covariance matrix of a random matrix 1000*1000 and the Marchenko–Pastur formula $\mu(x)=\sqrt{4x-x^2}/(2\pi x)$ describes the distribution (histogram) of eigenvalues very well. Funny that it's not exponential at all! However, this is an expression for the probability distribution of eigenvalues, and I am not sure how to get from it an approximation for the spectrum itself (which is a function on ${1,2,...,1000}$), it does not seem to be easy. It should be something like expected values of 1/1000-quantiles of $\mu(x)$?? Maybe @whuber can give us a further hint?
Comment: @amoeba, I think you can sample N=1000 values from the distribution by calculating the cdf of $mu(x)$ and then inverting the mapping from $x\to\mu(x)\to cdf(mu)$:
x=1:dt:4; y=mu(x); cdf_mu = cumsum(y)/sum(y); samples = interp(cdf_mu, x, rand(1000); sorted = sort(samples)
Comment: @UriCohen: Sure I can sample, but I was wondering how to get an *analytical* formula for the spectrum.
Comment: @amoeba, thinking about it again I think my answer provide this as well: the cdf is the integral of $mu$ and then the spectrum is just the inverse of the cdf. We just need to integrate and inverse $\sqrt{4x-x^2}/(2\pi x)$ :)
Comment: @UriCohen: Hmmm, the cdf $M(x)$ is a function that monotonically grows and $M(0)=0$, so for its inverse we also have $M^{-1}(0)=0$, hence $M^{-1}$ cannot be the spectrum as it should be monotonically decreasing. Maybe $(1-M)^{-1}$?..
Comment: @UriCohen did you mention the size of your P and N dimensions? This is crucial in determining the statistical properties of your sample eigenvalues (most answers here use random matrix theory that assumes that the number of observations N is not much bigger than the number of variables).
Answer: Everything has already been pretty much figured out in the comments, thanks to @AndyW, @whuber, and @UriCohen, but I would still like to write it up as a coherent answer.
First, let me illustrate the original question. Here is the eigenspectrum of some actual real data (neural recordings) that I happen to work with right now. First few (~20-30) PCs obviously carry some signal, but after that the eigenvalues start slowly decreasing in what does seem like an exponential fashion: note that the middle part of the spectrum is almost a straight line on this log-plot. I am not showing the last part of the spectrum, because there the eigenvalues decrease pretty much to 0, due to some temporal smoothing that I used before PCA.

Question is: why exponential decay?
The answer is, I believe, that any high-dimensional real data are highly contaminated by noise, so the bulk of the eigenspectrum shows the spectral behaviour of pure noise. What is the spectrum of a random covariance matrix? Turns out, there is a nice asymptotic result given by Marchenko–Pastur distribution, see the pdf of  the original 1967 paper in Russian if you like.
Marchenko and Pastur tell us to consider a random data matrix of $N\times D$ size filled with independent Gaussian random values from $\mathcal{N}(0,\sigma^2)$. If $\sigma^2=1$ and $N=D$, then in the limit $N \to \infty$ the distribution of eigenvalues of its covariance matrix is given by $$\mu(x)=\frac{\sqrt{4x-x^2}}{2\pi x}.$$
Let us verify. I generated a random matrix of the $1000 \times 1000$ size, computed its covariance matrix and then calculated the eigenspectrum. The first subplot below shows the covariance matrix. The second shows the distribution (histogram) of the eigenvalues and the Marchenko-Pastur function given above. It does fit perfectly.

But we are interested not so much in the distribution of eigenvalues, but in the eigenspectrum itself. If we draw 1000 values from the Marchenko-Pastur distribution (forming the spectrum) and sort them in decreasing order, then the resulting function will be given by $S(x)=(1-M(x))^{-1}$ rescaled to $[1, 1000]$, where $M(x)$ is the Marchenko-Pastur cumulative distribution function, i.e. $M(x) = \int_0^x \mu(t) dt$. The third subplot on the figure above shows the empirical spectrum vs Marchenko-Pastur fit.
It is quite a mess to compute $M(x)$, here is Wolfram Alpha's attempt. But we can note that $\mu(x)$ in the middle of its domain (around $x\approx 2$) is very well approximated by a straight line. This means that $M(x)$ will be approximately quadratic, and so its inverse $S(x) \sim \mathrm{const}-\sqrt{x}$.
In other words, the decay is not exponential at all, it is a square-root decay! However, funny enough, it is close enough to the exponential shape so that on the log-plot (see the fourth subplot above) the middle part of the spectrum looks pretty straight.
Comment: +1 Very nice answer. The part of your contribution--which is informative, well explained, and well illustrated--seems to boil down to the fact that any reasonably smooth function (such as the plot of eigenvalues *versus* their rank for a large correlation matrix) will, by the very definition of "smooth," look close to linear when one focuses on any short section of it. You have established that the linearity does not hold throughout, thereby putting an important qualification on the O.P.'s observation, and have characterized the general behavior very well.
Comment: Hi your first figure does not look exponential decay to me. It looks much more like power decay. You should try plotting it in log-log, I bet you will have a straight line !
Comment: @whuber: Thank you, but I am not sure I fully agree with your statement (or maybe I misunderstood it). Note that the plot of eigenvalues vs. rank (I call this "spectrum") is certainly smooth, but decidedly *not* linear: it forms a nice curl (see the low-left subplot on my main figure). What *does* look linear though, is the middle section of its logarithm (shown on the low-right subplot).
Comment: The definition of "smooth" means at least *differentiable* and that, in turn, means the graph is very close to linear when looked at over small enough sections.  That's all I'm saying--and that's the entire content of the OP's observation, when all is said and done.
Comment: @whuber: But the main point of the OP's observation is that log(spectrum) is *much* more "linear" than raw spectrum, even though both are "smooth" and so your reasoning could be applied to either of them.
Comment: You're right: my reasoning can and does apply to both observations.  Its conclusion is that the OP's observation is practically without content in either case.  It was only when you provided a clear *quantitative* assessment of the situation that any meaningful conclusions could be derived.


Busca avançada PHP

Question: Tenho uma tabela chamada veiculo_cadastrado com os seguintes campos:
id_cadastro,
           id_user,
           status,
           estado_veiculo,
           tipo_cadastro,
           fabricante_veiculo,
           modelo_veiculo,
           ano_fabricacao,
           ano_modelo,
           cor_veiculo,
           combutivel_veiculo,
           portas_veiculo,
           cambio_veiculo,
           kilometragem_veiculo,
           dono_veiculo,
           cidade_veiculo,
           valor_veiculo,
           descricao_veiculo,
           placa_veiculo,
           renavam_veiculo,
           nome_documento,
           alienado_veiculo,
           certificado_veiculo,
           garantia_veiculo,
           ipva_veiculo,
           licenciado_veiculo,
           revisoes_veiculo,
           revisoes_concessionaria
Minha busca avançada vai gira em torno dos campos fabricante_veiculo, modelo_veiculo, ano_fabricacao, ano_modelo, valor_veiculo(valor mínimo e valor máximo) e teria um campo de texto livre para o usuário digitar uma busca. Eu não tenho ideia de como começa alguém tem algum exemplo?
Comment: Da uma pesquisada meu caro. Isso envolve WHERES,BETWEENS, LIKE e etc. Da uma pesquisa sobre isso
Comment: A minha maior dúvida é q como os campos n vão ser sempre preenchidos eu devo fazer um If de verificação pra cada input ou tem outro jeito?
Answer: Primeiro você precisa criar o formulário de busca com os campos que citou, isso pode ser com tags  para alguns mas para o valor acho que vai precisar criar um "range", tipo valor entre X e Y.
Exemplo do form:
<form method="get" action="busca.php">

Fabricante:
<select name="fabricante" id="fabricante">
    <option value="">Selecione</option>
    <option value="11">Honda</option>
    <option value="22">Suzuki</option>
</select>

Faixa de preços:
<select name="preco" id="preco">
    <option value="">Selecione</option>
    <option value="0-10000">até R$ 10.000</option>
    <option value="10000-20000">Entre R$ 10.000 e R$ 20.000</option>
    <option value="20000-30000">Entre R$ 20.000 e R$ 30.000</option>
    <option value="30000-50000">Entre R$ 30.000 e R$ 50.000</option>
    <option value="50000-0">Acima de R$ 50.000</option>
</select>

Termo:
<input type="text" name="busca" id="busca" placeholder="Digite um termo para pesquisa">

<input type="submit" value="Buscar">
</form>

No PHP você precisa verificar quais campos o usuário definiu para montar a query.
Exemplo do PHP
<?php
$sql = "SELECT * FROM veiculo_cadastrado WHERE (1=1)";

if ($_GET['fabricante'] != "") {
    $sql .= " AND fabricante_veiculo = '{$_GET['fabricante']}'";
}

if ($_GET['preco'] != "") {

    switch ($_GET['preco']) {
        case '0-10000':
            $sql .= " AND valor_veiculo < 10000";
            break;
        case '10000-20000':
            $sql .= " AND valor_veiculo BETWEEN 10000 AND 20000";
            break;
        case '20000-30000':
            $sql .= " AND valor_veiculo BETWEEN 20000 AND 30000";
            break;
        case '30000-50000':
            $sql .= " AND valor_veiculo BETWEEN 30000 AND 50000";
            break;
        case '50000-0':
            $sql .= " AND valor_veiculo >= 10000";
            break;
    }

    $sql .= " AND fabricante_veiculo = '{$_GET['fabricante']}'";
}

if ($_GET['busca'] != "") {
    $sql .= " AND modelo_veiculo LIKE '%{$_GET['busca']}%'";
}

//Executa a consulta com a query pronta
mysql_query($sql);

Você vai precisar adaptar o código para suas necessidades, mas acho que o básico para montar uma busca está aí.
Basicamente você define um WHERE (1=1) e depois vai concatenando o restante da string com os campos que o usuário definiu. Pode ser uma boa escolha usar uma função de filtro como mysql_escape_string() para os campos livres para preenchimento do usuário. Ou melhor ainda se você fizer as consultas usando PDO, mas isso já é outro assunto..

Busca avançada PHP


How to search a table tr based on two values

Question: I have a table and a search input .
When User Types something in the input , i need to show relevant information related to the value  entered inside the search input
User can search either  by company name or ceo name
 // if user enters TCS or Chandran ,i need to show TCS table row
    $("#search").keyup(function(){
        _this = this;
        $.each($(".portlet"), function() {
            if($(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1)
               $(this).hide();
            else
               $(this).show();
        });
    });

http://jsfiddle.net/cod7ceho/173/
Could you please let me know how to fix this
Comment: in the fiddle, there's no table, just divs and lists. There are no `tr` elements anywhere. Nor are there any elements with a `data-compname` attribute. Did you write this code? It bears absolutely no relation to the example HTML. Not surprising it doesn't work.
Comment: There is  a main table which starts with
<table id="results" and under it the the first parent div consists of data-compname="TCS" data-ceo="Chandran"
Comment: yes, there's a table. but there are no `tr`s which is what you're trying to search for. And the only valid element within `tbody` is `tr`. You shouldn't put divs in there. Your HTML is invalid.
Comment: okay , please check the updated fiddle , how to filter based on two conditions right now its working only for company name , how to include ceo name also ?
Comment: simplest way is just to add an "and" clause to the if statement. If the search phrase does not appear in either of the fields, then hide the div.
Comment: you mean this way http://jsfiddle.net/cod7ceho/175/ which isn't working for me
Answer: Added one more condition:
if((($(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase())) === 0)||($(this).data('ceo').toLowerCase().indexOf($(_this).val().toLowerCase())===0))

Check out this fiddle:
http://jsfiddle.net/cod7ceho/177/
Comment: Pleasure. Cheers!!
Answer: You only want to hide 'rows' if you can't find the search string in both compname AND ceo, so you should add
&& ($(this).data('ceo').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1) to your if
like so:
if(($(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1) && ($(this).data('ceo').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1))
this gets a bit hard to read so I would also reformat to:
if(
    ($(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1)
    && ($(this).data('ceo').toLowerCase().indexOf($(_this).val().toLowerCase()) === -1)
){
    $(this).hide();
} else ...

Comment: Thanks but  getting Uncaught SyntaxError: Unexpected token &&
Comment: that's probably a bracket issue, your `if` may now look like
`if (yourstatement) && (mystatement)`
when it should be
`if ((yourstatement) && (mystatement))`
Answer: You could use JQuery filter(), for example:
$("#search").keyup(function() {
  _this = this;
  inputVal = $(this).val().toLowerCase();
  console.log(inputVal);
  if (inputVal === "") {
    $('.portlet').fadeIn();
  } else {
    $('.portlet').filter(function() {
      return $(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) !== -1;
    }).fadeIn();
    $('.portlet').filter(function() {
      return $(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) == -1;
    }).fadeOut();
  }
});

Fiddle: http://jsfiddle.net/cod7ceho/182/
Comment: what do you mean? I've tested and it seems fine... it filters the company name,but you can also modify to filter by the CEO aswell.
Comment: Could you please provide a sample fiddle
Comment: [http://jsfiddle.net/DouglasCamargo/cod7ceho/176/](http://jsfiddle.net/DouglasCamargo/cod7ceho/176/)
Comment: yes, but as I said, if you want to filter by the CEO as well you need to change the code a bit... but it shouldn't be a problem for you  ;)
Comment: but, of course, this is not the complete code, it's just a possibility... you can refine it to suit your needs...
Answer: I went following way:
$("#search").keyup(function(){
   _found = false;
   _this = this;
    $.each($(".portlet"), function() {
        if($(this).data('compname').toLowerCase().indexOf($(_this).val().toLowerCase()) == -1) {
           $(this).hide();
           _found = false;
       } else {
           $(this).show();
           _found = true;
       }

       if($(this).data('ceo').toLowerCase().indexOf($(_this).val().toLowerCase()) == -1 && _found == false)
           $(this).hide();
        else
           $(this).show();
    });
});

I added the variable _found as a state holder and additional condition. Properbly not the best way, it's just working.


Sum of geometric series from discrete maths

Question: My discrete maths book Gives proof for The sum of geometric series as:
$$\ S = \sum_{j=0}^n ar^j = \frac{(ar^{n+1}-a)}{r-1}$$
When r != 1,
And the next line reads:
Clearly, when r = 1, then, the sum is:
<code>                       (n+1)a
</code>
I don't understand the clearly part, What I could clearly see was only then when r = 1, the expression took an indeterminate form.
How was the (n+1)a expression obtained?
Answer: For $r = 1$, we have $r^j = 1$, and thus we just add $a$ $(n+1)$ times (from $j=0$ to $j=n$).
Answer: When $r=1$ you have $$S = \sum_{j=0}^n a\times 1^j = \sum_{j=0}^n a = a+a+\cdots +a = (n+1)a$$


Changing order of double integral

Question: I have a double integral with the integral with respect to x on the inside between 0 and y^2 and the outer integral with respect to y between 0 and 1. If i change the order of the integrals what would the new limits be for x and y?
Comment: hint: draw the region of integration
Answer: On the first case, you integrate on
$$\int_0^1 \int_0^{y^2} f(x,y) dxdy$$
with
$$0 < x < y^2 < 1$$
It's the same as
$$ 0 < \sqrt{x} < y < 1$$
Hence you want to integrate
$$\int_0^1 \int_{\sqrt{x}}^1 f(x,y) dydx$$
Answer: From your question you are integrating a function $f(x,y)$ on the set
$$S=\{(x,y)\in\mathbb{R}^2 : x \in (0, y^2) \wedge y \in (0,1)\}.$$
In order to change the limits of integration, let's work on $x \in (0, y^2)$. $x > 0$, easy. Then $\sqrt{x}$ exists and:
$$x < y^2 \Rightarrow (y < -\sqrt{x} \vee y > \sqrt{x})$$
Since $y \in (0,1)$, then we have:
$$y > \sqrt{x}.$$
Now, when $y=0$, then $\sqrt{x} = 0$ iff $x = 0$. When $y=1$, then $\sqrt{x} = 1$ iff $x = 1$.
At this point your domain $S$ can be rewritten as follow:
$$S=\{(x,y)\in\mathbb{R}^2 : y \in (\sqrt{x}, 1) \wedge x \in (0,1)\}.$$
You can conclude that:
$$\int_0^1 \int_{0}^{y^2} f(x,y) dxdy = \int_0^1 \int_{\sqrt{x}}^1 f(x,y)dydx$$

Changing order of double integral


tf.gather_nd to get values of 2d array from 1d array of indices

Question: I have a 2D array of values and a 1D array of indices. I want to get the value of each index at each element of the 2D array using tf.gather_nd or some other numpy or tensorflow command like so:
2D_array = np.arange(27).reshape((9, 3))
array([[ 0,  1,  2],
       [ 3,  4,  5],
       [ 6,  7,  8],
       [ 9, 10, 11],
       [12, 13, 14],
       [15, 16, 17],
       [18, 19, 20],
       [21, 22, 23],
       [24, 25, 26]])

1D_array = np.random.randint(low=0, high=3, size=9)
array([2, 2, 0, 2, 0, 1, 0, 1, 0])

output
[2, 5, 6, 11, 12, 16, 18, 22, 24]

What I currently have is this mess:
output = tf.gather_nd(a, list(zip(np.arange(len(b)), b)))
<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 2,  5,  6, 11, 12, 16, 18, 22, 24])>

A cleaner solution that avoids using zip and list would be greatly appreciated, as I don't think I'm currently utilizing tf.gather_nd completely correctly.
Answer: The 1D_array in your code should be made a 2d array with
inds = np.random.randint(low=0, high=3, size=9)
inds = np.dstack((np.arange(len(inds)),inds))

with this 2d array of indices, you can directly apply tf.gather_nd
import tensorflow as tf
import numpy as np

tmp = np.arange(27).reshape((9, 3))
inds = np.random.randint(low=0, high=3, size=9)
inds = np.dstack((np.arange(len(inds)),inds))

output = tf.gather_nd(tmp,inds)

with tf.Session() as sess:
    output_val = sess.run(output)


Squared fractional sum inequality

Question: Let $n$ be a positive integer. Is it true that $$\sum_{k=n+1}^\infty\dfrac{1}{k^2}<\dfrac1n?$$
Answer: Yes, it is true. This is because
$$k^2 > k(k-1) \implies \dfrac1{k^2} < \dfrac1{k(k-1)} = \dfrac1{k-1} - \dfrac1k$$
Hence,
$$\sum_{k=n+1}^{\infty} \dfrac1{k^2} < \sum_{k=n+1}^{\infty} \left(\dfrac1{k-1} - \dfrac1k \right) = \dfrac1n$$
Answer: Yes.  We can show this by noting that $f(t) = 1/t^2$ is a decreasing function, and that its definite integrals are therefore underestimated by the corresponding right-sided Riemann sum.  Thus, we have
$$
\sum_{k=n+1}^ \infty \frac 1{k^2} <
\sum_{k=n+1}^ \infty \int_{k-1}^k \frac 1{t^2}dt = \int_n^\infty \frac 1{t^2}dt=\frac 1n
$$

Squared fractional sum inequality


How to add extra legend to a map plot in ggplot2 based on column entries?

Question: Data: Data
Code:
palette = brewer.pal(11,"RdYlGn")    # ColorBrewewr.org spectral palette, 11 colors
ggmap_byscen   = ggplot(wmap_byscen.df[wmap_byscen.df$variable !=c("AVG") &
                                     wmap_byscen.df$ID_1 !=c("0"),], aes(x=long, y=lat, group=group))
ggmap_byscen   = ggmap_byscen + geom_polygon(aes(fill=value)) + facet_wrap(~ variable)
ggmap_byscen   = ggmap_byscen + geom_path(colour="grey50", size=.1)
ggmap_byscen   = ggmap_byscen + geom_text(aes(x=c.long, y=c.lat, label=ID_1),size=5)
ggmap_byscen   = ggmap_byscen + scale_fill_gradientn(name="% Change",colours=palette)
ggmap_byscen   = ggmap_byscen + coord_fixed(xlim = longlimits, ylim = latlimits)
ggmap_byscen   = ggmap_byscen + scale_y_continuous(breaks=seq(-60,90,30), labels=c("60ºS","30ºS","0º","30ºN","60ºN","90ºN"))
ggmap_byscen   = ggmap_byscen + scale_x_continuous(breaks=seq(-180,180,45), labels=c("180ºW","135ºW","90ºW","45ºW","0º","45ºE","90ºE","135ºE","180ºE"))
ggmap_byscen   = ggmap_byscen + labs(x="",y="",title="Average yield impacts across all crops across\nby climate scenarios (% change)")
ggmap_byscen   = ggmap_byscen + theme(plot.title=element_text(size=rel(2), hjust=0.5, vjust=1.5, face="bold"),
                                  legend.text=element_text(size=17),
                                  legend.position="left",legend.text=element_text(size=rel(1.3)),
                                  legend.title=element_text(size=rel(1.4), hjust=0.5, vjust=1),
                                  panel.background = element_rect(fill = "white", colour = "gray95"),
                                  strip.text = element_text(size=18),
                                  axis.text.x = element_text(size=16),
                                  axis.text.y = element_text(size=16))
ggmap_byscen

Result:
Question: I am looking to add an additional legend defined by the column "label" in the dataframe to identify the region on the map. Preferably, I'd like the legend to be below the faceted map. I have seen examples where one can add a table entry as a separate plot and then merge the two. I could not figure out how to make it for my case.
Any help would be great, thanks.
Comment: You don't seem to define `longlimits` and `latlimits`.
Answer: As @jlhoward mentioned, longlimits and latlimits are not defined. I, therefore, decided to leave  coord_fixed(xlim = longlimits, ylim = latlimits) part from this answer. My workaround works, but I am sure there are better ways to work on this. The challenge was to create another legend in a way it can present the data well. If you use colour in geom_text, you can create another legend, but you end up seeing the alphabet, a in the grey boxes in the legend. So, I decided to use geom_point with alpha = 0 as well as colour in aes. In this way, you have a new legend with ID names, but you do not see any points on the maps. Then, I used annotate to assign the numbers on the maps. Thanks to @jlhoward, I created a small data frame which is necessary for annotate(). If you use the original data frame, R tries to write the texts 4000 times or so. In the theme part, I added legend.key = element_rect(fill = NA) in order to remove grey squares in the legend. I made the height and width of the figure pretty small so that I could post it here. So it does not look that great. But if you specify large numbers, the figure will look better.
library(dplyr)
library(ggplot2)

wmap_byscen.df <- read.csv("mydata.csv", header = T)

mydf <- wmap_byscen.df[wmap_byscen.df$variable != c("AVG") &
        wmap_byscen.df$ID_1 != c("0"),]

### This is for annotate()

mydf2 <- select(mydf, c.long, c.lat, ID_1, ID_name) %>%
         distinct()

### Color setting

palette = brewer.pal(11,"RdYlGn")

ggplot(mydf, aes(x = long, y = lat, group = group)) +
geom_polygon(aes(fill = value)) +
facet_wrap(~ variable) +
geom_path(colour = "grey50", size = .1) +
geom_point(aes(x = c.long, y = c.lat, color=factor(ID_name, levels=unique(ID_name)), label = ID_1), size = 1, alpha = 0) +
annotate("text", x = mydf2$c.long, y = mydf2$c.lat, label = mydf2$ID_1) +
scale_fill_gradientn(name = "% Change",colours = palette) +
scale_color_discrete(name = "Regions") +
#coord_fixed(xlim = longlimits, ylim = latlimits) +
scale_y_continuous(breaks = seq(-60,90,30), labels = c("60ºS","30ºS","0º","30ºN","60ºN","90ºN")) +
scale_x_continuous(breaks = seq(-180,180,45), labels = c("180ºW","135ºW","90ºW","45ºW","0º","45ºE","90ºE","135ºE","180ºE")) +
labs(x = "",y = "",title = "Average yield impacts across all crops across\nby climate scenarios (% change)") +
theme(plot.title = element_text(size = rel(2), hjust = 0.5, vjust = 1.5, face = "bold"),
      legend.text = element_text(size = 8),
      legend.position = "bottom",
      legend.text = element_text(size = rel(1.3)),
      legend.title = element_text(size = rel(1.4), hjust = 0.5, vjust = 1),
      panel.background = element_rect(fill = "white", colour = "gray95"),
      strip.text = element_text(size = 18),
      axis.text.x = element_text(size = 16),
      axis.text.y = element_text(size = 16),
      legend.key = element_rect(fill = NA)) +
guides(col = guide_legend(nrow = 3, byrow = TRUE))

Comment: If you set `alpha=0` in the call to `geom_point(...)` the points disappear and it doesn'l matter what `size=...`.
Comment: Also, when you use `annotate(...)` this way, you are drawing a label *for every row of `mydf`*, so each label is drawn about 4000 times at the same location. This is why this runs so slowly. OP's use of `geom_text(...)` has the same problem. You should really create a separate data frame with just the 16 labels and the coordinates.
Comment: Finally, note the legend is ordered alphabetically, not numerically (e.g., 1, 10, 11, 12, ... instead of 1, 2, 3, ...) this is because of the way `ggplot` manages factors. You can fix that using `color=factor(ID_name, levels=unique(ID_name))` in the call to `geom_point(aes(...))` and by using `byrow=TRUE` in the call to `guide_legend(...)` at the bottom.
Comment: @jlhoward Thank you very much for the further notes. Yes, I noticed that the code was so slow, but I did not think why that was happening. As for the order of legend, I just left it since it would be fixed later using factor. But it seems that this operation was more than what I thought. Thank you for the advice. Given all of these, I have some good amount of revise today. :)
Comment: @iouraich I thought about that. I found a post on SO, but that seems to make things a bit complicated. Have a look of [**this post**](http://stackoverflow.com/questions/13143894/how-do-i-position-two-legends-independently-in-ggplot)
Comment: @iouraich I revised my answer. This is much faster thanks to jlhoward.
Comment: @iouraich Ah good to hear that. Your question gave me a good chance to know more about ggplot!
Comment: Is there a way to control manually the position of each legend?
Comment: @jlhoward Thank you very much for your message. That is absolutely right. I totally forgot that when I was trying to solve this question. Let me play with the code today and revise my suggestion. Thank you for your support.
Comment: @iouraich I am not sure if I am reading your message right. But, the legend is taking up space at the bottom, so you definitely have smaller maps. The other thought I have is that you may want to specify width and height in `ggsave` or `png`. Have you specified these?
Comment: @jazzurro I figure it out. It was just a matter of choosing the right image resolution while saving the plot as `png` file. Thank you for your work. It helped a lot.


Group By select

Question: Tenho uma tabela Funcionario com os campos Nome, Idade, Sexo e Empresa.
E cada funcionário pode ou não estar em mais de uma empresa.
Com isso fiz o seguinte select nessa tabela.
select Nome, Empresa from funcionario group by Nome, Empresa

Só que obviamente quando tenho um mesmo funcionário para mais de uma empresa, é duplicado o registro, e não queria que isso acontecesse,
sendo assim, fiz uma tratativa no back-end para resolver esse problema pegando apenas um dos registros.
var listaFuncionarios = funcionarios.GroupBy(e => e.Nome).Select(e => e.FirstOrDefault()).ToList();

Pergunta: Como resolver isso na própria query sem precisar alterar o back-end da forma que fiz ?
Comment: Seu comando SQL faz exatamente o que você disse que obteve. Se deseja que o Nome apareça uma única vez pode utilizar, por exemplo: `select Nome, MAX(Empresa) from funcionario group by Nome`.
Comment: A ideia é usar apenas a primeira empresa?
Comment: @LINQ Sim, não pode trazer mais de uma empresa para o funcionário, somente uma, independente de qual empresa seja.
Comment: Tentou usar LISTAGG ? pode ser uma solução de visualização https://docs.oracle.com/cd/E11882_01/server.112/e41084/functions089.htm#SQLRF30030
Answer: Sua tabela funcionario não está normalizada, porque o campo nome pode repetir de valor para diferentes empresas, para o mesmo funcionário,
nome                 | empresa
Mário da Silva Sauro | Toaldo Túlio Ltda.
Mário da Silva Sauro | João Falarz Ltda.

^^^^^^^^^^^^^^^^^^^^
desnormalização

Você pode fazer exatamente o que você fez no LINQ: agrupar por nome e pegar o primeiro registro que coincidir o nome,
SELECT f1.nome, f3.empresa
FROM   (SELECT nome
        FROM   funcionario
        GROUP BY nome) f1
CROSS APPLY (SELECT f2.empresa
             FROM funcionario f2
             WHERE f1.nome = f2.nome
             ORDER BY f2.empresa
             OFFSET 0 ROWS FETCH NEXT 1 ROWS ONLY) f3;

O comentário do usuário @anonimo também serve:
SELECT nome, MAX(empresa)
FROM funcionario
GROUP BY nome

Veja também no fiddle, aqui.

Group By select


Python sub-classes

Question: I have this class which works with fractions (e.g. (1,2), (3,4) etc):
class Fraction(object):

    def __init__(self, num=0, denom=1):
        ''' Creates a new Fraction with numberator num and denominator denom'''
        self.numerator = num
        if denom != 0:
            self.denominator = denom
        else:
            raise ZeroDivisionError
    def __str__(self):
        '''Returns the string numerator/denominator '''
        return "{0}/{1}".format(self.numerator, self.denominator)
    def __repr__(self):
        """blah"""
        return Fraction(self.numerator, self.denominator)

And I want to have a sub-class Mixed number which can accept both a whole number and a fraction (e.g. 2 Fraction(1,2), 3 Fraction (3,4) etc). I'm not sure how to do this however, Any help as to how to do this would be appreciated
class MixedNumber(Fraction):

Comment: I don't think it makes sense for `MixedNumber` to inherit from `Fraction`. It would make more sense to *compose* a `Fraction` into the `MixedNumber` class.
Comment: Subclass always means a 'is a' relationship. So `MixedNumber` seems not suitable to inherit from `Fraction`.
Comment: I agree with @HunterMcMillen, but just for the record, the syntax you use at the end of your post is exactly right for inheriting in Python. Maybe I don't understand what you're asking or what problem you're facing; could you clarify?
Answer: There are two ways that immediately come to mind when thinking of implementing a MixedNumber class with a Fraction class provided.  One way is as @HunterMcMillen suggests, is to have the class use composition, which is to utilize distinct objects to represent the whole number, and fraction parts, respectively.  This gives you access to each objects specialized methods inside your class, which can be handy if the components you're building from have distinct behaviors each encapsulates.
class MixedNumber(object):
   def __init__(self, whole_number, fraction):
        self.whole_number = whole_number
        self.fraction = fraction

   def __str__(self):
        if self.fraction.numerator == 0:
            return str(self.whole_number)
        elif self.whole_number == 0:
            return str(self.fraction)
        else:
            return '{} {}'.format(self.whole_number,self.fraction)

   def __repr__(self):
        return 'MixedNumber({},{!r})'.format(self.whole_number,self.fraction)

The other possibility is inheritance, if your Fraction class supports improper fractions, then you can use a subclass to appropriately handle initialization and string rendering in mixed number format, as follows:
class MixedNumber(Fraction):
    def __init__(self, whole_number, fraction):
        num = whole_number * fraction.denominator + fraction.numerator
        super(MixedNumber,self).__init__(num,  fraction.denominator)

    def __str__(self):
        # assume positive fractions for demonstration only
        q, r = divmod(self.numerator,self.denominator)
        if r == 0:
            return str(q)
        elif q == 0:
            return super(MixedNumber,self).__str__()
        else:
            return '{} {}/{}'.format(q,r,self.denominator)

    def __repr__(self):
        q, r = divmod(self.numerator, self.denominator)
        return 'MixedNumber({},Fraction({},{}))'.format(q,r,self.denominator)

Your Fraction.__repr__ method should return a string that when passed into eval should instantiate something equal to the source object, i.e. eval(frac) == frac.  See this stack overflow question for more details on the distinction between __str__ and __eval__.
Answer: I recently finished writing class Mixed that inherits from the standard library Fraction class.  It would probably be a good guide on how to use existing functionality of the parent class.  The source is HERE.  It accepts as input anything that the library Fraction class accepts as well as string representations of mixed numbers such as Mixed('3 4/5') or rational whole_number/numerator/denominator sets as in Mixed(3,4,5).  All of the functionality of Fraction has been preserved.  __str__ and __repr__ would yield 3 4/5 and Mixed(3,4,5) respectively.  It makes conversion to float or standard Fraction as easy as float(m) or Fraction(m).  Cheers

Python sub-classes


Response body of a Post Request can't be displayed

Question: I wonder if anyone can help. I am trying to send a post request to create an item and I get the correct  the status code back, but not the response body. I am fairly new to APIs',so, bit confused by this. Googling hasn't helped either. Here's my code:-
public void createGoods() {
    RequestSpecification httpRequest = RestAssured.given();
    RestAssured.baseURI = "http://192.168.0.31:8081/transit-api/clothes?_dc=159053130885&productId=1";
    JSONObject requestParams = new JSONObject();
    requestParams.put("productId", "1");
    requestParams.put("productDeploymentId", "1");
    requestParams.put("displayableId", goodsDisplayable);
    requestParams.put("createdUserName", "null");
    requestParams.put("product", "null");
    requestParams.put("productDescription", "");
    requestParams.put("createdDate", "null");
    requestParams.put("lastEventDate", "null");
    requestParams.put("progressDescription", "");
    requestParams.put("progressDate", "null");
    requestParams.put("active", true);
    requestParams.put("treatmentMilestones", "null");
    requestParams.put("status", "null");
    requestParams.put("stopComment", "");
    requestParams.put("primaryIdentifierString", "");
    requestParams.put("isPrimaryIdentifierAvailable", false);
    httpRequest
            .header("Accept", "application/json")
            .header("Accept-Language", "en-US,en;q=0.9")
            .header("Connection", "keep-alive")
            .header("Accept-Encoding", "gzip, deflate, br")
            .contentType(ContentType.JSON)
            .header("X-Requested-With", "XMLHttpRequest")
            .sessionId(sessionId);

    httpRequest.body(requestParams.toJSONString());
    Response response = httpRequest.request(Method.POST);

    System.out.println("RESPONSE body is--->>>>" + response.getBody().asString());
    System.out.println("Status code is----------------->>>" + response.getStatusCode());

}

}
The response I get is below:-
>>>>

RESPONSE body is---createTreatment----------->>>>*io.restassured.internal.RestAssuredResponseImpl@55259aa7*

Status code is ----------------->>>200

Step failed
java.lang.IllegalStateException: Cannot invoke the path method because no content-type was present in the response and no default parser has been set.
You can specify a default parser using e.g.:
RestAssured.defaultParser = Parser.JSON;

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:60)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:235)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:247)
    at io.restassured.internal.RestAssuredResponseOptionsGroovyImpl$_path_closure5.doCall(RestAssuredResponseOptionsGroovyImpl.groovy:392)

Comment: Does this answer your question? [How to pass default parser in REST Assured?](https://stackoverflow.com/questions/41234485/how-to-pass-default-parser-in-rest-assured)
Answer: parsing is a way to read data from your response. Set the below as your default parser.
RestAssured.defaultParser = Parser.JSON;

result read!

Comment: Thanks for the reply. I have noticed it doesn't have Content-Type in it's response headers when I post it from my method in Java. However, the same request does have Content-Type in its response when I post from Postman and it works fine. I tried adding the the parsor too but it does still work. From the mothod:-HEADERS----->Server=Apache-Coyote/1.1 X-Frame-Options=SAMEORIGIN X-Content-Type-Options=nosniff Content-Length=0 Date=Sun, 14 Jun 2020 13:39:34 GMT.
Answer: Try setting a default parser for REST Assured:
RestAssured.defaultParser = Parser.JSON;

See an example here: https://github.com/rest-assured/rest-assured/blob/rest-assured-4.3.0/examples/rest-assured-itest-java/src/test/java/io/restassured/itest/java/DefaultParserITest.java#L44-L52
Alternatively, make sure that the service you're testing returns a valid Content-Type HTTP response header.
Comment: Thanks for the reply. I have noticed it doesn't have Content-Type in it's response headers when I post it from my method in Java. However, the same request does have Content-Type in its response when I post from Postman and it works fine. I tried adding the the parsor too but it does still work. From the mothod:-HEADERS----->Server=Apache-Coyote/1.1
X-Frame-Options=SAMEORIGIN
X-Content-Type-Options=nosniff
Content-Length=0
Date=Sun, 14 Jun 2020 13:39:34 GMT.
Comment: This hints to an empty response body which doesn't need a `Content-Type`, see the `Content-Length` HTTP response header.


Covariant differential on p forms

Question: On Peter Li's book  Geometric Analysis on page 19 (http://www.im.ufrj.br/andrew/GR14-2/Lecture%20Notes%20on%20Geometric%20Analysis.pdf) I can't understand the following line
$\begin{array}{l}
da_{i_1\cdots i_p-1,j}\wedge w_j+a_{i_1\cdots i_p-1,k}w_{kj}\wedge w_j\\
=a_{i_1\cdots i_p-1,jk}w_k\wedge w_j-a_{i_1\cdots k_a\cdots i_p-1,j} w_{k_ai_a}\wedge w_j
\end{array}$
In fact my main doubt is about the second term on the RIGHT hand side, that I can't understand how it appears.
For more explanation  about the notation, the book is found on http://www.im.ufrj.br/andrew/GR14-2/Lecture%20Notes%20on%20Geometric%20Analysis.pdf, page 19. Thanks for help.
Comment: you probably want $d a_{i_1\cdots i_{p-1},j}\wedge \omega_j+...$ on the l.h.s.
Comment: what happened with the term $a_{i_1\cdots i_p-1,k}w_{kj}\wedge w_j$?
Answer: In the formula at pag. 19 after "Exterior differentiating this, we have" the author used the Cartan structural equation
$$d\omega_k = \omega_{kj}\wedge \omega_j $$
which defines the $\omega_{kj}$'s, given the basis $\{\omega_k\}$ of the cotangent space $T^*_xM$, dual to the basis $\{e_k\}$ on $T_xM$, for all $x\in M$.
In particular
$$ d(a_{i_1\cdots i_{p-1},j}\wedge w_j)=da_{i_1\cdots i_{p-1},j}\wedge \omega_j+ a_{i_1\cdots i_{p-1},j}d \omega_j = da_{i_1\cdots i_{p-1},j}\wedge \omega_j + a_{i_1\cdots i_{p-1},k}d \omega_k = da_{i_1\cdots i_{p-1},j}\wedge \omega_j + a_{i_1\cdots i_{p-1},k} \omega_{kj}\wedge \omega_j $$
EDIT
After a change in the OP, I update my answer. We know that
$$d(a_{i_1\cdots i_{p-1},j}\wedge w_j) = da_{i_1\cdots i_{p-1},j}\wedge \omega_j + a_{i_1\cdots i_{p-1},k} \omega_{kj}\wedge \omega_j; $$
we begin by proving that
$$d(a_{i_1\cdots i_{p-1},j}\wedge w_j) =  da_{i_1\cdots i_{p-1}} \wedge \omega_{j_a i_a} +
a_{i_1\cdots j_a\cdots i_{p-1}} \wedge \omega_{j_a k} \omega_{ki_a} + \\
\frac{1}{2} a_{i_1\cdots j_a\cdots i_{p-1}}\mathcal R_{j_a i_a k l} \omega_l \wedge \omega_k,  $$
i.e. the second equation at pag. 19 in the reference.
By definition of the covariant derivative (first definition at pag. 19), we have
$$ d(a_{i_1\cdots i_{p-1},j}\wedge \omega_j) = d^2 a_{i_1\cdots i_{p-1}} +
da_{i_1\cdots j_a \cdots i_{p-1}} \omega_{j_a i_a} +
a_{i_1\cdots j_a \cdots i_{p-1}}d \omega_{j_a i_a} = da_{i_1\cdots j_a \cdots i_{p-1}} \omega_{j_a i_a} + a_{i_1\cdots j_a \cdots i_{p-1}}d \omega_{j_a i_a}.~~(*) $$
All we need is to apply the Cartan second structural equation (pag. 16)
$$d \omega_{j_a i_a}= \omega_{j_a k}\wedge \omega_{k i_a} + \frac{1}{2}\mathcal R_{j_a i_a k l} \omega_l \wedge \omega_k $$
to the r.h.s. of (*) and arrive at the result.
We can move to the original equation in the OP. The second term on the r.h.s. in such equation follows from the use of the definition of covariant derivative on $d a_{i_1\cdots i_{p-1},j}$.In  fact
$$ a_{i_1\cdots i_{p-1},jk}\omega_k =d a_{i_1\cdots i_{p-1},j} +
 a_{i_1\cdots k_a\cdots i_{p-1},j}\omega_{k_a i_a}. $$
The above equation is the definition of the covariant derivative of the symbols
$a_{i_1\cdots i_{p-1},j}$. Reversing it we arrive at
$$ d a_{i_1\cdots i_{p-1},j} = a_{i_1\cdots i_{p-1},jk}\omega_k -
 a_{i_1\cdots k_a\cdots i_{p-1},j}\omega_{k_a i_a}, $$
as desired.
Comment: I see that, my question is about the next equation$$
da_{i_1\cdots i_p-1,j}\wedge w_j+a_{i_1\cdots i_p-1,k}w_{kj}\wedge w_j\\
=a_{i_1\cdots i_p-1,jk}w_k\wedge w_j-a_{i_1\cdots k_a\cdots i_p-1,j} w_{k_ai_a}\wedge w_j$$
I can't understand this equation. I can understand the first term on the right hand side, but I can't understand the second term on the right hand side. do you see?
Comment: Actually the equation you write in the comment is the one in the OP, once you apply the exterior derivative to $a_{...}$ as suggested in my comment above. I am a bit confused now: in the OP you stated that the 2nd term in the l.h.s. gives you problems, and not the second on the r.h.s.:my whole answer is based on the l.h.s. :-/
Comment: I am so sorry, that was my mistake, my doubt is about the right hand side. I got  a mistake, I changed the sides. Sorry.
Comment: No problem, I will update my answer.
Comment: I added some computations; I hope it helps. In case of doubts / remarks, please write me a comment, ok?
Comment: What happened to the term $a_{i_1,⋯,i_p−1,k}w_{kj}∧w_j$?

Covariant differential on p forms


Como retornar o valor ordenado do arsort para variáveis?

Question: Bom, minha duvida é como retornar o valor contido dentro do "arsort($array);" em variáveis que possam ser utilizadas em outras partes do código. mas as mesmas estando ordenadas como o resultado que aparece no "var_dump" tipo: $1 = 5, $2 = 10, $3 = 26, $4 = 34. Já tentei de varias maneiras, mas como sou novato no php, posso estar fazendo errado! Desde já agradeço a ajuda!
<code>$num1 = 10;

$num2 = 5;

$num3 = 34;

$num4 = 26;

$array = array($num1, $num2, $num3, $num4);

arsort($array);

var_dump($array);

//Código que aparece quando chamado o var_dump

array(4) {

[2]=> string(1) "5"

[1]=> string(2) "10"

[4]=> string(2) "26"

[3]=> string(2) "34"

}
</code>
Comment: Você tentou fazer um foreach para percorrer a array??
Comment: foreach($array as $valor){
echo $valor."";
}
Comment: Sim, utilizei. Mas tem como retornar o valor individual de cada resultado para uma variável diferente? ou como modificar o foreach para as variáveis? Provavelmente não estou sabendo fazer
Comment: sim você pode numerar as variaveis, usando um $i++, ou então você pode criar um array para cada variavel...
Answer: Você pode fazer assim:
<code>$num1 = 10;

$num2 = 5;

$num3 = 34;

$num4 = 26;

$array = array($num1, $num2, $num3, $num4);

$sort = arsort($array);

$nova_array = array();
$i = 0

foreach($sort as $value) {
    $nova_array[$i] = $value;
    $i++;
}

//Agora você pode usar assim:

echo $nova_array[0]; // imprime 5

echo $nova_array[1]; // imprime 10
</code>


Closed form for a binomial identity another solution

Question: Is this true?
$$\sum_{j=0}^n{j \cdot \displaystyle\binom{j}{r}} =\displaystyle\frac{(n+1)(r+1)-1}{r+2}\displaystyle\binom{n+1}{r+1}$$
Comment: @N.F.Taussig You can easily leave out $j=0$ and get the same result. What is the value of $j\binom{j}{r}$ when $j=0$, after all?
Comment: Should the lower index on the summation be $j = 0$?
Comment: Yes, sorry is over j
Comment: Maple says $${\frac { \left(  \left( n+1 \right) r+n \right)  \left( n+1-r \right)
 \left( n+1 \right) !}{ \left( {r}^{2}+3\,r+2 \right) r!\, \left( n+1-
r \right) !}}-{\frac {r \left( -r+1 \right) }{ \left( {r}^{2}+3\,r+2
 \right) r!\, \left( -r+1 \right) !}}
$$
Comment: Vielen Dank ! I have no maple.
Comment: @ThomasAndrews  True.  The reason I asked the question was that the original statement of the problem had $i = 0$ rather than $j = 0$.
Answer: Yes, it's true. See formula 4.35 here (p. 26)
We use
$$ \sum_{k=a}^n \sum_{i=a}^k C_{i,k}=\sum_{i=a}^n \sum_{k=i}^n C_{i,k}\tag{1}$$
Then
$$S=\sum_{k=1}^{n} \binom{k}{m}k=\sum_{k=1}^{n} \binom{k}{m}(\sum_{i=1}^k 1)=
\sum_{k=1}^{n} \sum_{i=1}^k \binom{k}{m}=
\sum_{i=1}^{n} \sum_{k=i}^n \binom{k}{m} \tag{2}
$$
But $$\sum_{k=0}^n \binom{k}{c}=\binom{n+1}{c+1} \tag{3}$$
Hence the inner sum in $(2)$ is:
$$\sum_{k=i}^n \binom{k}{m}=\sum_{k=0}^n \binom{k}{m}-\sum_{k=0}^{i-1} \binom{k}{m}=\binom{n+1}{m+1}-\binom{i}{m+1} \tag{4}$$
Further, using $(3)$ once more: $$ \sum_{i=1}^n \binom{i}{m+1}=\binom{n+1}{m+2}=\binom{n+1}{m+1}\frac{n-m}{m+2} \tag{5}$$
Then, putting all together:
$$ S=n \binom{n+1}{m+1}-\frac{n-m}{m+2}\binom{n+1}{m+1}=\frac{nm+n+m}{m+2}\binom{n+1}{m+1}$$
Comment: Where can I search if this was an IMO problem and who solved it???
Answer: Notice:
$$\sum_{n=a}^{m}n\binom{n}{r}=\frac{\frac{\left(m+r+mr\right)\Gamma\left(2+m\right)}{\Gamma\left(1+m-r\right)}-\frac{a\left(a+ar-1\right)\Gamma\left(a\right)}{\Gamma\left(a-r\right)}}{\Gamma\left(3+r\right)}$$
When, $a=0$:
$$\sum_{n=0}^{m}n\binom{n}{r}=\frac{\frac{\left(m+r+mr\right)\Gamma\left(2+m\right)}{\Gamma\left(1+m-r\right)}+\frac{1}{\Gamma\left(-r\right)}}{\Gamma\left(3+r\right)}$$
So, yes you're right!
Comment: @frank you're welcome!


Simple question about derivatives of integrals

Question: Assume that $f$ is differentiable in $\mathbb{R}$ and $f'$ is continuous in $\mathbb{R}$.
Set $F(x)= \int_0^1f(x-t)\,dt$.
Prove that $F'(x) = \int_0^1f'(x-t)\,dt.$
This is the question.
I thought this is just about the basic theorem about the derivatives of integrals, and according to the theorem, isn't the condition given ($f$ and $f'$ being continuous in $\mathbb{R}$) sufficient enough to prove the following equation?
Or is there more sophisticated method to prove this equation?
Answer: One approach would be to use the the rule for differentiating under the integral sign (there's an entire wikipedia article about this); if  $f$ and $f'$ are continuous, then the hypotheses of that theorem are certainly satisfied, and the proof will be a one-liner.
A more elementary, but necessarily longer, solution would be to change variables, for example, let $t = x-\alpha$. Then,
\begin{align}
F(x) &= \int_0^1 f(t) \, dt = \int_x^{x-1} f(\alpha) (- \, d\alpha) = \int_{x-1}^x f(\alpha) \, d \alpha.
\end{align}
Now, you can directly apply the standard fundamental theorem of calculus, to calculate the derivative, and then once again use the FTC to re-express that answer back as an integral involving $f'$.
Answer: Notice that the hypothesis you have are exactly the ones necessary to interchange the integral with the derivative. Hence, you can write $$
F'(x)=\dfrac{d}{dx}\int_0^1f(x-t)dt=\int_0^1\dfrac{d}{dx}f(x-t)dt=\int_0^1f'(x-t)dt=-f(x-1)+f(x),
$$
where in the second equality we have used the fact that $f$ is continuously differentiable in the whole space $\mathbb{R}$ and the fact that you are integrating in the compact set $[0,1]$ (so that you can interchange derivative and integral without problem).


Getting Faulting application error in application log

Question: We are using some legacy foxpro pages in our asp.net application, we are using an .exe to handle the foxpro page requests and also using WC.dll. It was working fine, but suddenly getting the following error
<code>Faulting application danamateweb.exe, version 0.0.0.0, stamp 47139f24, faulting module ntdll.dll, version 5.2.3790.4455, stamp 49900d60, debug? 0, fault address 0x0002cac2.
</code>
I do not have any idea, how to resolve this issue, our production is down, and DEV and Test box have no problem
Answer: WC.dll is a part of webconnect  (http://www.west-wind.com/).
I suggest restarting IIS or the server if that doesn't work.
Depending on how it is installed these might get you an error log:
/admin/wc.wc?wwmaint~showlog~Error
/admin/wc.wc?wwMaint~wcDLLErrorLog


How does the test error pattern depend on the regularizer function?

Question: This question is regarding the role of regularizer in an objective function.
Given a loss function $f(x)$, a regularizer function $r(x)$, and $\lambda$ being a trade-off function, our aim is to $\min f(x) + \lambda*r(x)$. I'm able to understand that as the value of $\lambda$ increases our training error increases.
I wanted to know how the test error behaves as $\lambda$ increases. From "Pattern Recognition and Machine Learning" by Bishop, I came to know that "the test error initially decreases and then starts increasing". Consequently, we choose the $\lambda$ which has the lowest validation error.
Does the same pattern (decrease and then increase) repeat for any given regularizer function? Or, does this pattern depends on the regularizer function which we use?
Answer: Yes-dependent on where the optimal cross-validated value of $\lambda$ lies-which is dependent on the true joint distribution that generates the features and the response (interpret as problem dependent) and the class of functions $\Omega$ such that $f(.) \subseteq \Omega$ (interpret as the kind of model that is being used to fit).  As you move from underfitting to the optimal point, the test-error and training-error continue to decrease. Then you reach the optimal $\lambda^*$, after which the Test-error begins to increase, while the training-error continues to decrease- indicating an over-fitting - that is not generalizing the model enough, so as to perform optimally on out-of-sample (unseen) data.


In any triangle is $\sin A+\sin B+\sin C=\frac{3\sqrt3}{2}$ always

Question: Well, I came with an interesting proof. But I just want to verify it
$$\begin{array}{l}
{\text { Applied at function } y=\sin x,} \\
 {\text { We have, } \sin \left(\frac{A+B+C}{3}\right) \geq \frac{\sin A+\sin B+\sin C}{3}}
\end{array}$$
From here we will get $$\sin A+\sin B+\sin C\leq \frac{3\sqrt3}{2}.$$
$$\begin{array}{l}
{\text { Also by A.M. } \geq \text { G.M. in an acute angled triangle }} \\
{\frac{\sin A+\sin B+\sin C}{3} \geq \sqrt[3]{\sin A \sin B \sin C}} \\
{\Rightarrow \sin A+\sin B+\sin C \geq 3(\sqrt[3]{\sin A \sin B \sin C})} \\
{\Rightarrow \sin A+\sin B+\sin C \geq 3\left(\frac{\sqrt{3}}{2}\right)=\frac{3 \sqrt{3}}{2}>2}
\end{array}$$
and from this I get $$\sin A+\sin B+\sin C\geq \frac{3\sqrt3}{2}.$$
Now the equation to be satisfied,
only equality condition should hold.
So in an acute angled triangle  $$\sin A+\sin B+\sin C=\frac{3\sqrt3}{2}.$$
Is there any fallacy in this convention.
Comment: Take half a square. Then $$\sin A + \sin B + \sin C = 1 + \frac{1}{\sqrt{2}} + \frac{1}{\sqrt{2}} \neq \frac{3\sqrt{3}}{2}$$
Comment: To echo Theo's comment, don't start looking for a general proof before you've tried a few specific examples.
Answer: There's an implied claim in the second block, that $\sqrt[3]{\sin A\sin B\sin C}\ge \frac{\sqrt{3}}{2}$. That claim is false. In fact, $\sqrt[3]{\sin A\sin B\sin C}\le \frac{\sqrt{3}}{2}$ with equality only when $A=B=C=60^\circ$. In an acute triangle, that quantity can get arbitrarily close to zero - consider a triangle with angles $\epsilon, 90^\circ-\frac{\epsilon}{2}, 90^\circ-\frac{\epsilon}{2}$. The product of sines in that triangle is less than $\sin\epsilon$, which goes to zero as $\epsilon\to 0$.
Naturally, the arguments that follow from that don't work.
I take it (from the last inequality claimed in the second block) you were asked to prove that $\sin A+\sin B+\sin C > 2$ in an acute triangle? That's true, and it's as strong as we can possibly have. Equality there is approached by the triangle I mentioned, in which the sines approach $0,1,1$.
Answer: In an acute angled triangle, it should be noted that $$\sqrt[3]{ sinA sinB sinC} \geq \ {\sqrt{3}\over2}$$ implies that the angles are $ \geq 60$. In this case, the sum of the angles are going to be $\geq 180$. So, such a triangle will not exist (except for the case of an equilateral triangle, where your proposition seems right).


How to make WPF ComboBox default selection to blank

Question: I have an enum like:
enum Gender
{
   Male = 1,
   Female = 2
}

Then I have a Combobox binding to a list of Gender.
<ComboBox ItemSource = 'Binding GenderList, Mode=TwoWay,
                        UpdateSourceTrigger=PropertyChanged'
          SelectItem = 'Binding SelectedGender' />

When user opens the view, how can I make ComboBox selection default to blank? Then after user made a selection, the blank selection is removed from the ComboBox source.
Thanks

Thanks all the replies. I fixed my problem by manully assigning the selected item to null.
Comment: Please make sure your XAML works to start with - there are no such properties as `ItemSource` or `SelectItem`.
Answer: What you are describing is the default behavior of the ComboBox.  You don't need to do anything (except for correct the typos in your XAML.  It should look like:
<ComboBox ItemsSource="{Binding GenderList}" SelectedItem="{Binding Path=SelectedGender}" ></ComboBox>

Comment: Hi Greg. thanks for your reply. I tried your code, but it still displays the first item in the list instead of blank item.
Comment: @Jing: Try removing the `SelectedItem` attribute.


Generate single random number from Pygame Sprite Collision, currently multiple numbers generated

Question: I am hacking the space invaders game from Python Crash Course. I'm using Python 3.5 and Pygame 1.9.2a0, groupcollide()
In the original, when bullets collide with spaceship both sprites are removed from screen.
In my version, I want the removal to me more random so that not all hits are successful.I have done this using the random module and making collisions successful if the random number is below a certain threshold (n). I have used the function in the code below, but it doesn't work as I want.
Using print(num) I have found that that random numbers are generated until    the num <=n is reached. I just want one number generated per collision.
I think this is happening because multiple collisions are detected as the bullet moves through the ships sprite rect. Testing this theory, if I remove bullet at collision then only 1 number is generated.
Q1. How can I keep bullet but only generate one number per sprite collision?
Q2. Do you have a better suggestion than using random numbers?
Any direction much appreciated, thanks
def check_bullet_alien_collisions(ai_settings, screen, stats, sb, ship, aliens, bullets):

    """Respond to bullet-alien collisions."""

    #I swapped True, True to False, False below
    collisions = pygame.sprite.groupcollide(bullets, aliens, False, False)

    #I added this if statement

    if collissions:
        num = randrange(100)
        print (num) # lots of numbers generated just want 1
        if num <= 30:
            collissions = pygame.sprite.groupcollide(bullets, aliens, True, True)

    #below in original code

    if len(aliens) == 0:
        # Destroy existing bullets, and create new fleet.
        bullets.empty()
        create_fleet(ai_settings, screen, ship, aliens)
        create_fleet(mi_settings, screen, decayingNucleus, ships)

Answer: You could give your Bullet class a self.live attribute and set it to True or False depending on the random number during the instantiation. Then iterate over the collided sprites and only call enemy.kill() if the bullet's live attribute is True.
import random
import pygame as pg

pg.init()

BG_COLOR = pg.Color('gray12')
ENEMY_IMG = pg.Surface((50, 30))
ENEMY_IMG.fill(pg.Color('darkorange1'))
BULLET_IMG = pg.Surface((9, 15))
BULLET_IMG.fill(pg.Color('aquamarine2'))

class Enemy(pg.sprite.Sprite):

    def __init__(self, pos):
        super().__init__()
        self.image = ENEMY_IMG
        self.rect = self.image.get_rect(center=pos)

class Bullet(pg.sprite.Sprite):

    def __init__(self, pos):
        super().__init__()
        self.image = BULLET_IMG
        self.rect = self.image.get_rect(center=pos)
        self.pos = pg.math.Vector2(pos)
        self.vel = pg.math.Vector2(0, -450)
        # If `live` is True, the bullet is able to destroy an enemy.
        self.live = True if random.randrange(100) < 70 else False

    def update(self, dt):
        # Add the velocity to the position vector to move the sprite.
        self.pos += self.vel * dt
        self.rect.center = self.pos  # Update the rect pos.
        if self.rect.bottom <= 0:  # Outside of the screen.
            self.kill()

class Game:

    def __init__(self):
        self.clock = pg.time.Clock()
        self.screen = pg.display.set_mode((800, 600))

        self.all_sprites = pg.sprite.Group()
        self.enemies = pg.sprite.Group()
        self.bullets = pg.sprite.Group()

        for i in range(15):
            pos = (random.randrange(30, 750), random.randrange(500))
            enemy = Enemy(pos)
            self.enemies.add(enemy)
            self.all_sprites.add(enemy)

        self.bullet_timer = .1
        self.done = False

    def run(self):
        while not self.done:
            # dt = time since last tick in milliseconds.
            dt = self.clock.tick(60) / 1000
            self.handle_events()
            self.run_logic(dt)
            self.draw()

    def handle_events(self):
        for event in pg.event.get():
            if event.type == pg.QUIT:
                self.done = True
            elif event.type == pg.MOUSEBUTTONDOWN:
                bullet = Bullet(pg.mouse.get_pos())
                self.bullets.add(bullet)
                self.all_sprites.add(bullet)
                pg.display.set_caption('This bullet is live: {}'.format(bullet.live))

    def run_logic(self, dt):
        mouse_pressed = pg.mouse.get_pressed()
        self.all_sprites.update(dt)

        # hits is a dict. The enemies are the keys and bullets the values.
        hits = pg.sprite.groupcollide(self.enemies, self.bullets, False, True)
        for enemy, bullet_list in hits.items():
            for bullet in bullet_list:
                # Only kill the enemy sprite if the bullet is live.
                if bullet.live:
                    enemy.kill()

    def draw(self):
        self.screen.fill(BG_COLOR)
        self.all_sprites.draw(self.screen)
        pg.display.flip()

if __name__ == '__main__':
    Game().run()
    pg.quit()

Comment: thanks for this, I will try to incorporate over the next week and let you know how it goes, too busy to do it now unfortunately as its my weekend project, wish it was the other way around haha


How do I use Lua hooks to find out the name of the currently running function?

Question: I am currently running a Lua script with a hook attached using LUA_MASKCALL and the hook function implemented as follows:
void LuaHook(lua_State *L, lua_Debug *ar) {
    switch(ar->event) {
        case LUA_HOOKCALL:
            lua_getinfo(L, ">n", ar);
            //breakpoint here...
        } break;
        default:
            break;
    }
}

I would like to get the name of the called function, but am not sure I am doing it correctly since it never seems to give me anything (the breakpoint is hit at the appropriate times though). The functions that are called are bound C functions that do have names so that seems weird. Overall lua_getinfo is a complete mystery to me and the documentation does not clear things up either so any help would be greatly appreciated on this one.
Comment: Dunno, but in the Lua Reference, they call lua_getinfo(L, ">n", &ar), the ampersand is missing in your code, or am I wrong?
Comment: They do so because they have defined ar to be 'lua_Debug ar;' (not a pointer). It would not have compiled if I had done such an error.
Answer: lua_getstack did the trick so never mind


Google Drive API - JavaScript Insert File - Create native google doc

Question: I have been working with the JavaScript Google Drive API in order to create (insert) a new file.  It works great, except that the document it creates is not a google doc, but some kind of document that can be converted in to a google doc.
I would like an actual google doc.  I tried "convert:true", but it does not seem to convert it to a doc.
insertFile: function (filename, content, callback) {
  const boundary = '-------314159265358979323846';
  const delimiter = "\r\n--" + boundary + "\r\n";
  const close_delim = "\r\n--" + boundary + "--";

  {
    var contentType = 'application/vnd.google-apps.document';
    var metadata = {
      'title': filename,
      'mimeType': 'text/html',
      'description': 'Created'
    };

    // TODO: replace this with a library - https://github.com/beatgammit/base64-js/blob/master/lib/b64.js
    var base64Data = window.btoa(unescape(encodeURIComponent(content)));
    var multipartRequestBody =
        delimiter +
        'Content-Type: application/json\r\n\r\n' +
        JSON.stringify(metadata) +
        delimiter +
        'Content-Type: ' + contentType + '\r\n' +
        'Content-Transfer-Encoding: base64\r\n' +
        '\r\n' +
        base64Data +
        close_delim;

    var request = gapi.client.request({
      'path': '/upload/drive/v2/files',
      'method': 'POST',
      'params': {'uploadType': 'multipart', 'convert': true },
      'headers': {
        'Content-Type': 'multipart/mixed; boundary="' + boundary + '"'
      },
      'body': multipartRequestBody
    });
    if (!callback) {
      callback = function (file) {
        console.log(file)
      };
    }
    request.execute(callback);
  }
}

After I insert the file, when I go to the link I get:
Answer: I think I found your error. Try migrating to v3, according to v3 migration:

Other changes
Imports to Google Docs formats are now requested by setting the appropriate target mimeType in the resource body, rather than specifying ?convert=true.
Import operations will return a 400 error if the format is not supported.

And here is the link for the list of supported mimeType.


Switch back to one workbook after activating another workbook

Question: My problem is as follows:
I have two files opened - file1.xlsm and file2.xlsm. The first one is active. I want to switch back to the first workbook, every time I activate the second one. I put
Private Sub Workbook_Activate()

    Windows("file1.xlsm").Activate

End Sub

in a macro module of the second workbook. It doesn't work. Is there a way to do it? Thanks for any help
Comment: Oh, I made it calling a macro with the same statement (Windows("file1.xlsm").Activate). Nevertheless, I don't understand why it doesn't work while putting it in workbook macro module
Answer: You have to place
Private Sub Workbook_Activate()
    Workbooks("file1.xlsm").Activate
End Sub

in the ThisWorkbook code window of file2.xlsm (not a macro module).
Answer: Try
Call Workbooks("filename.xlsm").Activate


Somar valores no Javascript

Question: No seguinte código, ao clicar no input name "goleiro" ele altera outro input com o valor desejado.
O problema é que eu gostaria de colocar outro campo que calcula automaticamente o valor total (valor goleiro + valor técnico), mas não consigo. Alguém poderia me ajudar?
Script

Html:
<input name="goleiro" onclick="alterarValor('valor_goleiro', '5');" value="5" type="radio" id="gol" />
<input name="tecnico" onclick="alterarValor('valor_tecnico', '10');" value="10" type="radio" id="tec" />

Valor do seu goleiro: <input type="text" id="valor_goleiro" value="0" disabled />

Valor do seu Técnico: <input type="text" id="valor_tecnico" value="0" disabled />

Valor Total: <input type="text" id="valor_total" value="0" disabled />

Comment: Posta como resposta, vai ajudar quem passar por um problema de somas semelhante ao seu.
Comment: tira as 'aspas' do numero para evitar chamar a função parseInt(), economiza recursos
Comment: Buscando na internet eu já consegui. O que faço? removo a postagem ou devo postar a solução mesmo assim?
Answer: A solução foi simples, eu apenas criei mais uma função, e chamei as duas funções no onclick.
Observe:

E nos inputs, chameis as 2 no onclick:
<input name="goleiro" onclick="alterarValor('valor_goleiro', '5');somar();" value="5" type="radio" id="gol" />

Como disse, mais simples do que eu esperava!

Somar valores no Javascript


Convolutional Neural Network to detect watermark images

Question: I have two different sets of images , one we have water mark that we want to keep and other set of images we have water mark but we dont want to keep that
how can i train i model to detect the watermark images that i want to keep and to detect the images that i dont want to keep since all images has water mark
Regards
Prince Bhatia
Comment: Sounds like straight-forward supervised classification. Start by trying transfer learning on a pre-trained CNN.
Comment: Could you clarify your question a bit? The title says that you want to detect images with water marks, but the question seems to imply that you have two categories of images which you want to binarily classify.
Comment: Yes i have two categories of images, one with watermark that i want to keep and others are also with watermark but i dont want to keep, how can i train my model to detect images that i want to keep and also that i dont want to keep since all the images have watermark.

Apologies for title ,  i am not so good in english and nothing came in mind so wrote that title. I will change it
Answer: This is a classical classification problem. There are many frameworks like TensorFlow, Keras, Caffe which support this out of the box.
All you've to do is to create the required directory structures with images.
The link below explains about building an image classifier using keras.
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html


A chord to follow B6 for use with melody?

Question: I am trying to find a chord to follow a B6 chord. The melody I am writing sounds really nice, and I have noticed the first part follows the E6 chord, then the B6 chord, so my thought is that if I can find another chord to fit this progression, it will help me finish my melody. I guess that would mean my "chord progression" so far is E6 then B6...
I know enough music theory to get around (at least for electronic music), but I am not perfect at it. I would really appreciate any input as to the direction I can take to resolve this melody nicely! (I am aiming for a bittersweet melody. Happy, but with a sad undertone if that means anything)
Thank you so much for your time! I normally don't post on stackoverflow for music (I am a computer programmer so I use the programming section quite a bit though), so please let me know if I am missing anything!
EDIT:
Ultimately I am writing a melody. Just thought I would clarify.
EDIT 2:
The key is B major! (or G# minor I guess. The tonal center of this particular melody is hard for me to judge.)
Comment: Of great importance is 'what key is it in?' Other factor is 'what notes are after the B6 part?'
Comment: I can't believe I forgot to mention the key! It is B major! (or its relative minor, G# minor) The notes after the B6 part is what I am trying to figure out. I would include an audio file so you can listen to the melody I have so far, but I don't think stackoverflow lets you attach audio files.
Comment: If the melody begins in E6 (really a colorful variety of E major) then I'm guessing the whole melody is in that key.  If it is, then the B6 is a V chord, and the normal way to finish would be by going back to the E6.  However, if it really IS in B, the it starte on the IV chord (rather rare) and has already resolved to the tonic, so I would have no idea where to go from there!
Comment: I guess it really could be in the key of E major. I had not thought of that. All the notes that I have actually used in the melody are both in B major and E major. After playing around with the melody/harmony a little bit, I am beginning to think that using the key of E major is a good idea, however I would not like to resolve the melody right away. If I go straight back to an E6 chord, it seems to end the melody too abruptly. Thank you for pointing this out! I will play around with some more of the options in E major and see if I can find anything!
Comment: If you want to extend the melody and not resolve it too soon, you have several options-- go to IV, back to V, then to I; or stick a vi in there somewhere, perhaps between the IV and the second V.  If you're into classical at all, listen to the slow movement of the Rachmaninoff Symphony #2, and see how he extends the opening seemingly forever, merely by shifting between related chords without quite ever going back to the tonic, except at the very end.  Since you've used the 6-flavor so far, you'll want for the sake of consistency to keep doing that most of the rest of the way too.
Comment: As far as the tonal center is concerned, you'll probably know by the end.  (Most melodies end on the tonic note.)
Comment: I found the slow movement of the Rachmaninoff Symphony #2, and I can see what you are talking about! (I mostly like classical music for analyzing music theory. I can play the Moonlight Sonata by Beethoven though XD). Your comments have helped me  a ton!! @L3B I am still playing around with it, but I am coming closer to something I like!
Comment: @L3B I think I have figured out what I am going to do with it! Your comments have helped the most so far, so if you put it as an answer, I will mark it as correct! If I remember, I will come back and post a link to the song when it is done!
Comment: I've never turned a batch of comments into an answer before, but I'll try!  And thanks for the kind words.  And yes I'd definitely like to hear the melody when you're comfortable sharing it.
Answer: You could go back to the B6, then, using the G# as a common note go Bb7, Eb6...
But without hearing your melody, we're just fantasizing.
Answer: I would suggest going to say an Amajor7 chord and then to G#minor.  That sounds real nice and keeps you moving towards the key center you are shooting for (Emajor/Gminor).
Answer: If the melody begins in E6 (really a colorful variety of E major) then I'm guessing the whole melody is in that key. If it is, then the B6 is a V chord, and the normal way to finish would be by going back to the E6. However, if it really IS in B, the it starte on the IV chord (rather rare) and has already resolved to the tonic, so I would have no idea where to go from there!
In a comment you indicated that you were comfortable with the idea of it being in E, but wanted to make sure you didn't resolve it too soon.
My answer to that is this:   If you want to extend the melody and not resolve it too soon, you have several options-- go to IV, back to V, then to I; or stick a vi in there somewhere, perhaps between the IV and the second V. If you're into classical at all, listen to the slow movement of the Rachmaninoff Symphony #2, and see how he extends the opening seemingly forever, merely by shifting between related chords without quite ever going back to the tonic, except at the very end. Since you've used the 6-flavor so far, you'll want for the sake of consistency to keep doing that most of the rest of the way too.  As far as the tonal center is concerned, you'll probably know by the end. (Most melodies end on the tonic note.)
I would like to add the following to our conversation in the comment section:  The vi, ii, and iii chords are your friends in extending a melody.  Along with the V of ii and the V of iii.  You'll find that when you do these kinds of manipulations that the melody is becoming harmony-driven, rather than driving the harmony itself, but there's nothing wrong with that.  A lot of great melodies are harmony driven.
Good luck and please do post the melody when you are comfortable sharing it.


Simplifying a particular expression

Question: I have the expression
$$ \frac{x(-c z^{n+1} (-1)^k + c x^{n+1})}{(x-ze^{i \theta})(x - ze^{-i\theta})},$$
where $b$ and $c$ are non-zero complex numbers, $z = \frac{b}{\sqrt{bc}}$ and $\theta = \frac{k\pi}{n+1}$. I am confident that for any $n \in \mathbb{N}_{\geq 1}$ and $k \in \{1,2,\ldots,n\}$ this fraction can be written as a polynomial in $x$ (for non-zero denominator), since it should be equal to some generating function, but I haven't proven this conclusively yet. Therefore, I would like to prove that we can rewrite this fraction in the (slightly more specific since $y_0 = 0$) form
$$ Y(x) = y_1x + y_2x^2 + \cdots + y_nx^n, \ y_i \in \mathbb{C}.$$
For $n \in \{1,2,3\}$ Mathematica has provided $Y$, but I was hoping for something more general. Is there some way to either find $Y$ explicitly or prove such a $Y$ exists for all such $n,k$ (possibly with more help from Mathematica)?
Answer: Let $f(x):=-c z^{n+1} (-1)^k + c x^{n+1}$.
Such a $Y$ exists since we have
$$\begin{align}f(ze^{\pm i \theta})&=-c z^{n+1} (-1)^k + c(ze^{\pm i \theta})^{n+1}
\\\\&=-c z^{n+1} (-1)^k + cz^{n+1}e^{\pm i \theta\ (n+1)}
\\\\&=-c z^{n+1} (-1)^k + cz^{n+1}e^{\pm i k\pi}
\\\\&=cz^{n+1}\bigg(-(-1)^k + e^{\pm i k\pi}\bigg)
\\\\&=0\end{align}$$
Comment: Of course, I don't know why I didn't think of that. Thank you!

Simplifying a particular expression


Django: How to add query params to queryset as is done in Pagination?

Question: Our site is DRF backend, with Vue frontend.  It incorporates extensive filter options, so any endpoint could have many query parameters - like example.com?name__icontains=bob&...etc.
We have a situation on one view where we need to return a sum of one column. Instead of creating a separate endpoint for the frontend to hit, we want to add it to the results of the original view.  Like so...
{
    "pagination": {
        ...pagination stuff...
    },
    "results": [
        {
            "id": 1,
            "task": "DMSD-50",
            ...other fields...,
            "duration": 204
        },
        {
            "id": 2,
            "task": "DMSD-53",
            ...other fields...,
            "duration": 121
        },
        ...more rows...
    ],
    "totals": {
        "duration": 955
    }
}

My first attempt was to add this to the get() method of the view:
    def get(self, request, *args, **kwargs):
        response = super().get(request, *args, **kwargs)
        queryset = self.get_queryset()
        totals = queryset.aggregate(total_duration=Sum('duration'))
        response['totals'] = {
            'duration': totals['total_duration'] or 0
        }
        return response

...which worked when I did the full (unfiltered) list.  But when I added filters in the URL, the total was always the same.  That made me realize that get_queryset() doesn't add query parameters from the URL.  I then realized that the pagination class must be incorporating query parameters, so my second solution was to add this code to our custom pagination class.  That works, but it seems like a hack.
What's the best way to get the queryset with query parameters already added, or to add them myself?  I know those parameters are available in kwargs, so if (for example) my view only filtered on "name", I could access the filter via kwargs['name'] - simple.  But, as I said, there can be many filter args, and they certainly vary from one view to another.
If I loop through kwargs, I'm thinking I might be getting other things besides filter args - sort order comes to mind - but I'm guessing there may be others.  I need a fool-proof way of getting ALL filter args and ONLY filter args.
Answer: I assume you are using some generic view or are atleast inheriting from ListModelMixin. The get_queryset method does not do the filtering of the queryset, that task is done by the filter_queryset method of the view, hence when you override the get method and don't call that method there is no filtering done. Change your code to use it like so:
class YourView(generics.ListAPIView):  # Just an assumption here
    ...
    def get(self, request, *args, **kwargs):
        response = super().get(request, *args, **kwargs)
        queryset = self.filter_queryset(self.get_queryset())
        totals = queryset.aggregate(total_duration=Sum('duration'))
        response['totals'] = {
            'duration': totals['total_duration'] or 0
        }
        return response

For understanding better exactly how the builtin view works you might want to look at it's source code [GitHub].
Comment: That did the trick. I figured there must be a simple method call that would take care of it, I just couldn't find it. Instead of reading the code, I had stepped through in debug, hoping to see where it added those parameters. But I quickly got lost in all the confusing method calls that happen during a request. Thanks for the quick reply.
Answer:
Create empty context
Context = {}

Update context with each existing filter in request.GET
If request.GET.get('filter'):

   Context['filter] = request.GET.get('filter')

Add filters to queryset
Query.objects.filter(**Context )

Comment: That would still require hard-coding all the filter keys in the code. It's actually much worse than the simple example in my OP. For some of our TextChoices fields, we might have __exact or __icontains or __in suffixes on the field name. That grows the number of possibilities exponentially. I guess I'll have to do it by exception in a loop. That is, if "ordering" is the only non-filter query param, I'll skip that key value. If others come up at some point, I'll just have to add them to the "skip" list.  I just thought there must be some canned way, if pagination classes handle it.


If you drop a bouncy ball on a level surface doesn't it never stop bouncing?

Question: I realized this because of the Infinite Divisibility Theory. Which states that you can infinitely keep splitting something in half. Furthermore it would keep bouncing because there would always be a space between the surface and the ball's surface. Also the only reason why we can't see it bouncing is because it is an extremely small distance between the surface of the ball and the surface it's bouncing on.
Comment: If the time of the first **round-trip** bounce is $T$, then the total time will be $$T \times \left[1 + \frac{1}{2} + \frac{1}{4} + \cdots \right] = 2T.$$  See also [Zeno's Paradoxes](https://en.wikipedia.org/wiki/Zeno%27s_paradoxes).
Comment: Is space infinitely divisible? What happens when the distances involved are less than the width of a proton? etc.
Comment: What the comments above imply is that your physical model for the bouncy ball is no longer accurate in certain limits
Comment: What are those limits
Answer: To answer this question you have to distinguish between the physical system and the mathematical model.
If the bounces are from inelastic collisions between the ball and the immovable surface then the bounce height never decreases and the motion is periodic. In the real world that's impossible.
For elastic collisions you need to take into account the energy lost to heat when the ball and the surface momentarily contract and rebound at the bounce. Then the bounces are smaller and smaller as time goes by. I can imagine mathematical models where the magnitude decays to 0 in finite time  or in infinite time (I don't know enough physics to decide which is realistic. I suspect the former).
But any good model of the macroscopic behavior will fail at small times and distances. Then quantum mechanics takes over.
Comment: But wouldn't't there always have a space no matter how small be made
Comment: Your comment makes no sense at small distances. For an analogy, think of a fuzzy tennis ball bouncing on a shag carpet. The contact is not instantaneous. Near the time of contact you can't really tell where the ball ends and the carpet begins. At really  small distances the world is weird. Newton's mathematics no longer predicts the results of experiments in physics. The mathematics of quantum mechanics does much better. But it's not the last word. There's still physics we can't model correctly. See https://simple.wikipedia.org/wiki/Planck_length
Answer: Mathematics is a tool for modelling real world phenomena; real world phenomena is not mathematics. Math will only be as useful insofar as the asssumptions underlying the math are dually satisfied by the real world system. For instance, you reasoning suggests that space can be infinitely divided, but there is a limit to the smallest unit of space known as the Planck distance, according to the best currently accepted theory in physics.
Comment: yes, it is difficult to imagine a very small unit of indivisible space, but the standard model of physics suggests this indeed exists. Part of the problem is that you are using your mathematical intuition to deduce something about the real world. While this has the potential to lead great insights, it can also be dangerous if you are not aware the of limits of this kind of reasoning. Various branches of mathematics are predicated on fundamental sets of assumptions (axioms) we simply assume to be true. The physics of the universe also appears to obey a fundamental set of laws...
Comment: I don't know how there is not something smaller than planck distance
Comment: but the fundamental laws of the universe and the axioms of our mathematics are not necessarily one and the same. The latter was determined by us. The former was determined independently of us.
Answer: The ball couldn't keep jumping. As @EthanBolker said, each bounce becomes smaller. But space is actually not continuous, contrary to what most people think. It is actually discrete, and the smallest possible length where physics makes sense is known as the Planck length. If we let the ball bounce until its last bounce was of height $l\le a\le2l$, where $l$ is the Planck length, then the ball will have to stop bouncing.

If you drop a bouncy ball on a level surface doesn’t it never stop bouncing?


allow machine to access localhost website

Question: I've got some IP rules on an Azure VM, the first blocks all traffic, the second allows my laptop to access the website. Both work fine, however when I try to access a website hosted on the VM locally the traffic gets blocked.
I tried setting up an additional rule with the source IP as 'My IP Address' but that didn't seem to work, I don't have a static IP address on this.
Any ideas on how I can maintain a block on all external traffic, allow just my laptop (which does have a static IP address) but continue to allow browsing the website from the VM?
Thanks
Comment: Did you try to add "localhost" aka 127.0.0.1?
Comment: Yep, when I tried to save it, I got a message stating, that 'The loopback address (127.0.0.1) is not a valid source or destination address.'
Comment: Not entirely sure on this, but: You might want to try setting up an IPSec connection on both, your server and computer. Edit your block rule to only block if there is no secure connection. Then you can create a rule that allows traffic, if there is a secure connection.
Answer: Resolved the issue.
For the IP security rule implementing the block action, the connection tile was 'All Connections'.
These were the steps I followed to resolve the issue

Open up MMC,
'File' > 'Add/remove Snap-in'
Select 'IP Security Policies'
Select the relevant policy and click Edit,
Select the connection type tab,
change the connection type from 'All Network' connections to 'Remote Access'

Then presto I could access localhost again


Generating license keys for my python app

Question: this is my first time ever making full python app with tkinter and wanting to distribute it. I've been searching the net, but I cant quite figure out how to make a license key for my application, as in I want a unique key I give to certain people which allows them to run the application, and also I want to able to deactivate that key and make new ones. Note this is an application for window computers, Thanks for any help in advance.
Comment: Generating a key is a piece of cake. Protecting your app especially for an interpreted language is going to be the hard part.
Comment: If you want to desactivate remotely, it seems you need a central server that allows the app to run by checking its key everytime it's launched. What is your question ? What have you tried ?
Comment: Yeah I just wanna know how to generate the keys and authenticate it, I have zeroooo clue, I don't know where to start
Comment: Yeah it seems i need a Server to check it's key every time  it's launched
Answer: Check out keygen.sh - It's LaaS (Licensing as a service).
You create an account and generate license keys with or without an expiration date and number of unique devices. From your app, you then use their API to validate/activate your license.
There's an example showing device activation, license validation and deactivation.
Answer: I hope this helps :)
<code>import requests

def license():
    # The list with all keys.
    keys = requests.get("http://yourlink.com/licensekeys.txt").text
    # keys = ["key1", "key2", "key3"]

    # License key from user.
    keyfromuser = "mykey"

    for key in keys.splitlines():
        if key == keyfromuser:
            # Code when key match.
            return

    # Code if the key don't match.
    exit()

license()
</code>
Comment: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answer: you can generate using python uuid after that need to store in database from there you can authanticate, remove or add new keys.
Comment: Is there a tutorial on this somewhere ?
Comment: I don't know about that but you can make script as I said and If you want gui for this than make django admin panel.
Comment: also one thig you can also implement it like session/coockie so after every X time it'll expire.
Comment: ohk than you can use tkinter but that looks like outdated thing :)
also if you use django than as I think your development will be 20X faster and also looks good.
Comment: Isn't django used for web application? Mine is desktop application
Answer: If you want to convert it to exe then ship it with an installer, InstallForge app can generate serial codes for you. At the left panel of  InstallForge, there is a Dialog section containing 'Serial Validation.' You can generate random serial codes with it and store it in your activation server.


Revision history summary

Question: I am looking for a solution to show a summary of revision history for all nodes in one place. The out of the box solution is to show a revision history for each node apart. I want to be able to track my content admins' changes in one place (I couldn't find a module that does what I want)
Thanks!
Answer: If you don't mind to use Views module, you can configure any list of revisions. You also can add filter/sort logic, pager, configure access and much more useful things.

You always can export your solution with Views export or with Features module and reuse it on other Drupal sites.
More information about Views: http://nodeone.se/en/node/20
Answer: There are severel modules that extend revision, one of this is Revisioning.
Another way is create a view of revisions.

Revision history summary


Why when Y=-X I get negative p.d.f?

Question: Am I doing something wrong here?
Suppose we know the pdf of the random variable X  fx(x) and we wish to find the pdf of the random variable Y fy(y), where Y=-X
Then fy(y)=-fx(-y). But that means the p.d.f of Y is negative since p.d.f of X is positive.
But we can't have a negative p.d.f
What's going on here?
Thanks in advance
Answer: Let us restrict to the case where the CDF of $X$ is differentiable.
Then its derivative serves as PDF ($F_X'=f_X$) and:
$$F_{Y}\left(y\right)=P\left\{ Y\leq y\right\} =P\left\{ X\geq-y\right\} =1-F_{X}\left(-y\right)$$
so that: $$f_{Y}\left(y\right)=F_{Y}'\left(y\right)=f_{X}\left(-y\right)$$
Equality $f_{Y}\left(y\right)=f_{X}\left(-y\right)$ holds also if $F_X$ is not differentiable.
Comment: but -y=x so I thought 1-Fx(-y)=1-Fx(x) so it's derivative is       -fx(x)= -fx(-y)
Comment: Do you agree that $F_Y(y)=1-F_X(-y)$? What is the derivative of this function?
Comment: you are right I differenciated wrt to x. my mistake

Why when Y=-X I get negative p.d.f?