Agrupar dados de uma tabela

Question: tenho a seguinte instrução sql:
Select

Notas.NotaID,
Materias.MateriaNome,
Alunos.AlunoNome,
Alunos.AlunoID,
   max(case when NotaBimestre = 1 then Nota else null end) as 1bim,
   max(case when NotaBimestre = 2 then Nota else null end) as 2bim,
   max(case when NotaBimestre = 3 then Nota else null end) as 3bim,
   max(case when NotaBimestre = 4 then Nota else null end) as 4bim

From ((notas
INNER JOIN Materias
   ON Notas.MateriaID = Materias.MateriaID)
INNER JOIN Alunos
   ON Notas.AlunoID = Alunos.AlunoID)

group by (NotaID)

Resultado:

Como faço para ficar nesse padrão:
ID | ALUNO    | DISCIPLINA           | 1BIM| 2BIM | 3BIM | 4BIM
16 | NATANAEL | BANCO DE DADOS       | 8   |      |      |
17 | GUSTAVO  | PROGRAMAÇÃO PARA WEB | 9.5 | 8    | 8.5  | 10

Comment: Qual SGBD você está utilizando?
Comment: Você vai precisar consultar a tabela de notas 1 vez para cada bimestre, não percorrer ela apenas uma única vez como está na sua consulta. Recomendo usar subqueries para tal
Comment: Tire o "nota id" do GROUP BY.
Comment: @Motta não vai adiantar porque ele precisa verificar o bimestre. No caso ele terá que fazer uma tabela com os valores e depois realizar um `PIVOT`
Answer: Considerando que seu SGBD seja o SQL Server você pode reunir os dados em uma subquery e realizar o PIVOT em cima desses dados:
SELECT a.alunonome AS aluno,
       m.materianome AS disciplina,
       y.bim1,
       y.bim2,
       y.bim3,
       y.bim4,
  FROM (
    SELECT n.materiaid,
           n.alunoid,
           'bim' + CAST(n.notabimestre AS VARCHAR) as semestre,
           n.nota
      FROM notas n
  ) x
  PIVOT (AVG(x.nota) FOR semestre IN (bim1, bim2, bim3, bim4)) y
  INNER JOIN materias m ON m.materiaid = y.materiaid
  INNER JOIN alunos a ON a.alunoid = y.alunoid

Caso esteja utilizando o MySQL, você pode agrupar utilizando CASE e ajustando o GROUP BY:
SELECT a.alunonome AS aluno,
       m.materianome AS disciplina,
       AVG(CASE n.notabimestre
             WHEN 1 THEN n.nota
           END) AS bim1,
       AVG(CASE n.notabimestre
             WHEN 2 THEN n.nota
           END) AS bim2,
       AVG(CASE n.notabimestre
             WHEN 3 THEN n.nota
           END) AS bim3,
       AVG(CASE n.notabimestre
             WHEN 4 THEN n.nota
           END) AS bim4
  FROM nota n
 INNER JOIN materias m ON m.materiaid = n.materiaid
 INNER JOIN alunos a ON a.alunoid = n.alunoid
 GROUP BY n.materiaid, n.alunoid, a.alunonome, m.materianome

AVG
Returns the average value of expr... If there are no matching rows, AVG() returns NULL.

Em tradução livre:

Retorna o valor médio da expressão... Se não houverem linhas, AVG() retornará NULL.

Observação: Notei que você utilizou a função MAXpara obter as notas mas acredito que o ideal no seu caso é utilizar o AVG, já que o que interessa é a média.
Comment: Muito bom Sorack! Me ajudou aqui
Comment: @SmokeRohden disponha. Se possível adicione qual o SGBD nas tags da pergunta, por favor
Comment: Se caso eu quiser mostrar o ID da nota referente ao bimestre teria como?
Comment: @SmokeRohden Só se você concatenar, já que são 1 para vários
Answer: O seu SELECT não pode iniciar na tabela Notas, tem que iniciar em alunos, e fazer um SUB-SELECT para cada bimestre, mais ou menos dessa forma:
 SELECT b.MateriaNome, a.AlunoNome, a.AlunoID,
        (SELECT MAX(Nota) FROM Notas WHERE NotaBimestre = 1 AND MateriaId = b.MateriaId AND AlunoId = a.AlunoId) AS '1bim',
        (SELECT MAX(Nota) FROM Notas WHERE NotaBimestre = 2 AND MateriaId = b.MateriaId AND AlunoId = a.AlunoId) AS '2bim',
        (SELECT MAX(Nota) FROM Notas WHERE NotaBimestre = 3 AND MateriaId = b.MateriaId AND AlunoId = a.AlunoId) AS '3bim',
        (SELECT MAX(Nota) FROM Notas WHERE NotaBimestre = 4 AND MateriaId = b.MateriaId AND AlunoId = a.AlunoId) AS '4bim',
    FROM Alunos a
    INNER JOIN Materias b
    GROUP BY a.AlunoID, b.MateriaId;

Agrupar dados de uma tabela


How to "fit" the size of a binomial distribution

Question: I'm solving a problem where I have 15 samples of a unknown distribution. They ask me to fit somehow the parameters of the Binomial, Poisson and Normal distributions. I could use the sample mean and standard deviation to obtain the parameters in the cases of the Normal and Poisson distributions. That's pretty clear. But I'm puzzled with the binomial. The MLE for p assumes that the size N is known, which is not my case. Any idea on what I'm missing here? Can I estimate the size of a unknown binomial distribution?
Answer: Assume $X_1, ...,X_{15} \sim B(n,\pi)$. We know that $E(X) = n\pi$ and $V(X)=n\pi(1-\pi)$.
You can use moment method to get the estimate of the estimate of $n$ and $\pi$. At first get the sample mean $\bar X$ and sample variance $\hat V(X)$, then solve following you will get the estimate.
$$\bar X = \hat n \hat\pi$$
$$\hat V(X) = \hat n \hat\pi(1-\hat \pi)$$
Comment: Ok, just to make it clear. This problem requires using the moment method. There's no closed solution with maximum likelihood alone. Right?
Comment: Of course you can try MLE. But maybe more complicated.


Emberjs Complex Routing

Question: In my application there is a client side for members to select bottles and an admin section to manage members, bottles, lockers etc. that both use the same models so I figured that my my router should look like this:
App.Router.map ->
  @.route "signin" #Restuarant signs in to the application
  @.resource "member", ->
    @.route "signin" #Members sign in to see their bottles
    @.resource "member", path: ":member_id", ->
      @.resource "bottles", ->
        @.route "select"
      @.resource "transcations", ->
        @.route "create"
        @.resource "transcation", path: ":transcation_id", ->
  @.resource "admin", ->
    @.route "signin" #Restaurant signs in to manage
    @.resource "members", ->
      @.route "create"
      @.resource "member", path: ":member_id", ->
        @.resource "bottles", ->
          @.route "create"
          @.resource "bottle", path: ":bottle_id", ->
            @.route "edit"
        @.resource "transcations", ->
          @.resource "transcation", path: ":transcation_id", ->
    @.resource "lockers", ->
      @.route "create"
      @.resource "locker", path: ":locker_id", ->
      @.resource "lockertype", path: "types", ->
        @.resource "lockertype", path: ":locker_type_id", ->
          @.route "edit"
        @.route "create"

I understand that the resources under admin will replace the resources for the members that have the same name.
What would be the best pattern to solve this?
My current solution would be to namespace (kinda?) the resources like this:
    @.resource "member", ->
      @.route "m.signin" #Members sign in to see their bottles
      @.resource "m.member", path: ":member_id", ->
        @.resource "m.bottles", ->
          @.route "select"
        @.resource "m.transcations", ->
          @.route "create"
          @.resource "m.transcation", path: ":transcation_id", ->

Is there a better way? Is this way terrible? I've been looking up EMBER.NAMESPACE but I don't understand how I would use it.
Relevant discussions on github:
https://github.com/emberjs/ember.js/issues/683
https://github.com/emberjs/ember.js/pull/1925
Answer: If you declare nested resource, it's name must contain parent resource's name ended with dot: 'members.member', 'members.member.bottles', etc. Names of tail routes are simpler ('signin' insted of 'member.signin').
You can observe all application's routes with help of chrome extension (Ember Inspector).
Update from 14 Sep:
I used to an old versions of Ember and somehow missed an update in routing system. There is no need for this.resource call in the last Ember version (1.7.0). This means you can use only this.route method, which is easer and more clear: http://emberjs.jsbin.com/bukogojuzeni/1/edit?html,js,output

Emberjs Complex Routing


Alguém sabe o que eu tenho que fazer nessa table para que ela não fica torta assim?

Question:
    foreach ($registro as $reg){

    echo "

Nome
Telefone
Email

";
    echo "
";
    echo "
";
    echo "".$reg['nome']."	";
    echo "".$reg['telefone']."	";
    echo "".$reg['email']."	";
    echo "
";
    echo "
";
    echo "
";
}

Answer: O problema é que você está criando várias tabelas independentes (olhe o source da sua página para entender melhor).
Tire a estrutura da tabela do loop, assim terá várias linhas na mesma tabela:

         Nome
         Telefone
         Email

<?php
    foreach ($registro as $reg){
    echo "\t\t
";
    echo "\t\t\t".$reg['nome']."	";
    echo "\t\t\t".$reg['telefone']."	";
    echo "\t\t\t".$reg['email']."	";
    echo "\t\t
";
}
?>

Note que eu acrescentei alguns \t nos echo para indentar, você não precisa fazer isto, e se fizer, ponha o quanto for necessário para alinhar certinho com o resto do HTML (para ajudar você a entender). O \t é o caractere de tabulação, pode trocar por espaços em branco.
Comment: deu tudo certinho aqui, muito obrigada pela sua valiosa ajuda


Absolute Value Rational Inequalities Help Please

Question: I have been read plenty of questions on questions like this, but I still dont quite get it. For example, this question:
$$ \left| \frac{2x+1}{x-3} \right| \ge 2 $$
How would I go about solving this?
-The method in the example went on to just square both sides of the equation and from there it formed a quadratic equation to solve it. But, I dont get why you can just square it, dont we need to take into consideration since the expression is absolute value, it could also have a negative value as well? Like, 2x+1 can also be -(2x+1). Is it because, regardless if it were positive or negative, once you square it, the result would be the same? I am confused because other times, typically we would have 2 cases, one for 2x+1 > 0 and 2x+1 <0.
Another questions is, when can I just square both sides and continue from there? Is there a more complete way of doing it rather than just squaring both sides and be done? In what scenarios, cant I square both sides? And what is the alternative method to solving those types of problems? Thanks.
Comment: Is this $$\left|\frac{2x+1}{x-3}\right|\geq 2$$  ?
Comment: It completely depends on the problem and squaring may be useful to get rid of lots of square roots, et cetera
Comment: Can you see why $|x|\ge2$ is the same thing as $x^2\ge4$?
Comment: @Dr.SonnhardGraubner.  No, he wrote 2x + 1/x -  3 >= 2.
Comment: @WilliamElliot ... Was `/` in the original ASCII inequality meant to have high or low precedence? I interpreted it as low precedence. If that was a mistake, someone please roll back my edit.
Comment: @Dr.SonnhardGraubner Yes correct sir
Answer: You can square both sides of an inequality provided they are both nonnegative; this is because the squaring operation preserves order for nonnegative quantities. That is, given $x,y\geq 0,$ and $x\leq y,$ then it follows that $x^2\leq y^2.$ The reason is simple: Since both $x$ and $y$ are nonnegative, you can multiply $x\leq y$ by each in turn to get, respectively, $x^2\leq xy$ and $xy\leq y^2,$ from which the result follows immediately. Thus, you can square both sides of $x\leq y$ without qualms, provided that $x,y$ are nonnegative.
Recall that when the symbol $|{\cdot}|$ encloses an expression $E(x,y),$ say, we mean that the result -- here $\left |E(x,y)\right |$ -- is nonnegative, by definition -- that is, it is either positive, or else it vanishes. It follows from the explanation above that you can square both sides of your inequality, since $\text{LHS}\ge 0,$ and obviously  $2>0.$

The explanation above holds, strictly speaking, only for inequalities involving $\leq$ (and of course, $\geq$), called weak inequalities; not for those involving the strong $\lt.$ The point of difference may appear minimal, but is essential from a strict perspective. If we have two nonnegative $x,y$ satisfying $x\lt y,$ then we can square them without disturbing the order $\lt$ only provided $x,y$ do not vanish simultaneously, for it is easy to see in that case that we have a false statement right from the beginning, namely $0\lt 0.$ This is the caveat to the above statements.

PS. This is for completeness, as OP seems to need more clarification in this direction.
I shall now solve your inequality in order to ascertain that everything I said is indeed clear. So we have $$ \left| \frac{2x+1}{x-3} \right| \ge 2,$$ upon squaring both sides of which (this is legit as explained above) gives $$ \left| \frac{2x+1}{x-3} \right|^2 \ge 4.$$

We pause here and note that the equality $|A||B|=|AB|,$ for any quantities $A,B,$ holds. This is a property satisfied by the modulus operation, whose proof would take us away from the main goal (try to convince yourself of its truth; by cases, perhaps). In that equality, if $A=B,$ then we have the true statement $$|A||A|=|A|^2=|AA|=|A^2|=A^2,$$ the last equality following provided that $A$ is real.

Now returning to the main thread and applying the above result, we then have $$ \left| \frac{2x+1}{x-3} \right|^2= \left| \left (\frac{2x+1}{x-3} \right)^2\right|=\left (\frac{2x+1}{x-3} \right)^2\ge 4.$$
I believe you can now proceed from here.
Comment: Thanks, but for here, how do we know if the expression inside the modulus is non-negative? For example, if the x were -2, and you were to cross multiply ending up with |2x+1| >= 2|x-3|. How can you square, both sides not knowing what the value of x is in the modulus? If x were -2 the inequality sign has to change right?
Comment: @AndrewLee We don't care what's inside the moldulus symbol. We're squaring the modulus of whatever it is; since moduli are never nonnegative, it is legit if the other side of the inequality is also nonnegative. Thus, what you're squaring is $\left |E(x,y,\ldots)\right |,$ and not $E(x,y,\ldots).$
Comment: @AndrewLee I have added some more explanation in my answer. See if it now makes some sense to you.
Answer: Hint: For $$x\neq 3$$ we can write
$$|2x+1|\geq 2|x-3|$$ so we have to distinguish the following cases:
a) $x>3$ then we have $$2x+1\geq 2(x-3)$$
b)$-\frac{1}{2}\le x<3$ and we get $$2x+1>-2(x+3)$$
c) $x<-\frac{1}{2}$ and we have $$-(2x+1)>-2(x-3)$$
Comment: Thanks, I have used this method. But I have a problem with the squaring both sides of the inequality method. I don't get how you can square |2x+1| >= 2|x-3| on both sides, knowing that if x were negative, the inequality sign would have to change? I watched a video proving why you can square both sides of an inequality. For example |a|^2 > |b|^2, then what he wrote next was (a)^2 > (b)^2. You can square the modulus because absolute value is always positive but if you get rid of the modulus, a could be negative and upon squaring a, the inequality symbol wouldould have to swap, right?
Answer: For $x \ne 3$ we have
$\left|\frac{2x+1}{x-3}\right|\geq 2 \iff 4x^2+4x+1 \ge 4(x^2-6x+9)$
Can you proceed ?
Comment: It is $x^2-6x+9$.
Comment: Ooops ! Thanks.
Comment: Thanks! I get that you can square it, but how do you know that the expressions inside the modulus are non-negative? If they were, then the inequality symbol would have to change right? That's my problem here


confidence interval for linear regression coefficient with error following a t distribution

Question: I have this linear model $$=_0+_1+$$
where the error terms  are iid from a student t-distribution with constant degrees of freedom k. I want to construct a 95% confidence interval for $\hat{\beta_1}$. The general formula is ($\hat{\beta_1}-\alpha * se(\hat{\beta_1}), \hat{\beta_1}+\alpha * se(\hat{\beta_1})$). Normally if the error  follows a N(0,1) distribution, then $\alpha$ would be the 0.975 quantile for the N(0 1) distribution, which is 1.96. But since  follows at distribution here, I am not sure which distribution should I extract the quantile from.
Should I use quantile from the same t distribution where the error is from, or should I use a t distribution with degree of freedom =n-2 (n is the sample size, and minus 2 because I have $\beta_0, \beta_1$ in the model). Can anyone share some thoughts?
Thank you.
Comment: The quantile depends only on the confidence interval's "width" that you prefer. If it's a 95% confidence interval, why should your quantile be different. It will be the same: `0.975`.
Comment: sorry I meant to say which distribution should I extract the quantile from
Comment: you said you wanted to extract it from the t-distribution, don't you.
Comment: yeah, but I am not sure which degree of freedom should I use? Should I use k or n-2?
Comment: I don't know. The t-distribution was your assumption. What degree of freedom do you assume for your error term?
Comment: it's just a question that we asked to solve, so the k is given as a fixed number say 5, but n-2=38 in the example.
Comment: Obviously this degree of freedom of the error term can have nothing to do with the degrees of freedom of this linear model. The former is a frivolous assumption. Maybe your professor wants you to fit the real data to a certain degree of freedom and then compute a confidence interval. But I guess you may be not equipped to do this - too complicated.
Answer: I'm going to assume that you still use OLS estimation to fit your model (though you could easily justify departure from that and use the MLE instead in this case).  I'm also going to assume that you use a scaled T-distribution for the error, still using the multiplier $\sigma$ to allow the error term variance to have a variable scale.  If you don't include this parameter, just take $\sigma = 1$ in my answer and the result is the same.
Assuming you used OLS estimation, your estimated coefficient vector can be written as:
$$\hat{\boldsymbol{\beta}}
= \boldsymbol{\beta} + (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \boldsymbol{\epsilon}.$$
We then have:
$$\begin{align}
\mathbb{V}(\hat{\boldsymbol{\beta}})
&= [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}] \mathbb{V}(\boldsymbol{\epsilon}) [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[10pt]
&= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{I} [(\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T}]^\text{T} \\[6pt]
&= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1} \mathbf{x}^\text{T} \mathbf{x} (\mathbf{x}^\text{T} \mathbf{x})^{-1} \\[6pt]
&= \frac{k \sigma^2}{k-2} \cdot (\mathbf{x}^\text{T} \mathbf{x})^{-1}. \\[6pt]
\end{align}$$
Therefore, taking $\mathbf{M} \equiv (\mathbf{x}^\text{T} \mathbf{x})^{-1}$ to be the inverse-Gramian matrix in the regression, the true standard error here is:
$$\begin{align}
\text{se}_i \equiv \mathbb{S}(\hat{\beta}_i)
&= \sqrt{\frac{k}{k-1}} \cdot \sigma \cdot M_{i,i},
\end{align}$$
and the estimated standard error is:
$$\begin{align}
\hat{\text{se}}_i \equiv \hat{\mathbb{S}}(\hat{\beta}_i)
&= \sqrt{\frac{k}{k-1}} \cdot \hat{\sigma} \cdot M_{i,i}.
\end{align}$$
As you can see, the only difference here from the case of a normal distribution is that you have the additional factor $\sqrt{\tfrac{k}{k-2}}$ in the standard error term.  If you add this additional factor into the standard error for your confidence interval, it will account for the effect of the degrees-of-freedom parameter on the error variance.
Comment: +1 But then what happens if $k\le2?$
Comment: In that case the error variance is infinite, so the estimator standard error is also infinite.
Comment: The underlying question concerns whether this formula attains the desired coverage.  For smallish datasets and smaller degrees of freedom, I (and, I believe, the OP) suspect it might not, due to the heavy tails of the error variables.


Qt creator C++, adding many labels to statusbar

Question: I would like to add 3 items to statusbar. Shouldn't each be in its own label?
Is that possible? When I tried to add a second label, it gave me an error.
File mainwindow.h,
QLabel *m_statusLabel;
QLabel *m_pointLabel;

File mainwindow.cpp,
statusBar()->addWidget(m_statusLabel);
statusBar()->addWidget(m_pointLabel);

It works perfectly with one label, but when I add a second, then I get the following.
Starting C:\Users\Jansu\Desktop\cpp-praktikum05-alus\Joonistamine-build-      desktop\src\bin\Joonistamine.exe...
ASSERT: "d" in file ..\..\include/QtCore/../../src/corelib/tools/qscopedpointer.h, line   112
Invalid parameter passed to C runtime function.
Invalid parameter passed to C runtime function.
C:\Users\Jansu\Desktop\cpp-praktikum05-alus\Joonistamine-build-    desktop\src\bin\Joonistamine.exe exited with code -1073741819

Comment: That's probably not caused directly by adding the widgets to the status bar. How are you creating the `QLabel`s?
Comment: Why to add 3 different widgets and not just one which contains everything? Perhaps [QStatusBar](http://doc.qt.nokia.com/latest/qstatusbar.html) supports only one widget, but you can also try using `addPermanentWidget`.
Answer: I found the problem. I forgot to create QLabels, so I added this:
m_statusLabel = new QLabel(this);
m_pointLabel = new QLabel(this);


on the equivalence of notions of adjunctions

Question: let $C$ and $D$ be categories and $F : C \leftrightarrows D : G$ be a pair of functors. as is well known, there is a natural correspondence of

pairs of inverse isomorphisms $α : \hom_D (FX, Y) \leftrightarrows \hom_C (X,GY) : β$ natural in $X$ and $Y$, and
transformations $η \colon 1_C → GF$ and $ε \colon FG → 1_D$ such that $(εF)(Fη) = 1_F$ and $(Gε)(ηG) = 1_G$.

this correspondence essentially stems from the yoneda lemma, naturally in $X$ or $Y$ relating

transformations $\hom_D(FX,–) → \hom_C(X,G–)$ to elements in $\hom_C(X,GFX)$, so to transformations $1_C → GF$,
transformations $\hom_C(–,GY) → \hom_D(F–,Y)$ to elements in $\hom_D(FGY,Y)$, so to transformations $FG → 1_D$.

now, apparently, again by yoneda, the triangle identities for $1_F$ and $1_G$ relate to $βα = \mathrm {id}$ and $αβ = \mathrm {id}$ respectively.
– i'm having trouble seeing that!
let's look at $βα = \mathrm {id}$: so $α$ and $β$ are morphisms of functors $C^\mathrm {op} × D → \mathrm {Set}$ with a composition
$$\hom_D (F–,–) → \hom_C(–,G–) → \hom_C(F–,–),$$
which by locally invoking the yoneda embedding for all $X$ in $C$ says that some morphism of functors $F → F$ is the identity. apparently this morphism is precisely $(εF)(Fη)$ in the correspondence? why is that?
Answer: If you have a natural (in both variables) transformation $\Phi:\mathbf D(F-,-)\to \mathbf C(-,G-)$, and hence an adjunction $F\dashv G:\mathbf D\to \mathbf C$, observe that any component of the unit $\eta_C:C\to GF(C)$ can be recovered as $\Phi(\operatorname{id}_{F(C)})$, and similarly any component of the counit $\varepsilon_D=\Phi^{-1}(\operatorname{id}_{G(D)})$.
Hence from one hand $\Phi^{-1}\circ\Phi (\operatorname{id}_{F(C)})=\operatorname{id}_{F(C)}$, as they are inverses. If however you start with $\operatorname{id}_{F(C)}$ and apply $\Phi$ you get $\eta _C$; applying $\Phi^{-1}$, you get exactly $\epsilon_{F(C)}\circ F(\eta_C)$, since the universal element of the natural transformation $\Phi^{-1}_{-,GF(C)}: \mathbf C(-,GF(C))\to \mathbf D(F-,F(C))$ is $\varepsilon_{F(C)}$; in fact, this last assertion is contained in the proof of Yoneda's lemma. Thus this shows that $\epsilon_{F(C)}\circ F(\eta_C)=\operatorname{id}_{F(C)}$; to obtain the  other triangular identity, you can reason dually.


Prove that 3 points are collinear - Geometry problem

Question: If P, Q , R are interior points of the sides BC, CA
and AB of a triangle ABC and the corresponding Cevians AP, BQ and
CR  are concurrent at a point X .
We have :
$\frac{AX}{XP}=\frac{AR}{RB}+\frac{AQ}{QC}$ ( Van Aubel's theorem )
We suppose that :
(RQ) and (BC) are concurrent at a point L
(PR) and (CA) are concurrent at a point M
(PQ) and (BA) are concurrent at a point N
How can I prove that L,M,N are collinear ?
Thanks in advance
Comment: Do you have a drawing to submit?
Answer: This is a special case of the theorem of Desargues.
Desargue's theorem, wiki page
In our case, $X$ is the center of the perspective of the two triangles (with corresponding vertices on the same places in the writing)

$\Delta ABC$,
$\Delta PQR$.

(If this is not enough, i may try to give a proof adapted to the given situation, where we also have harmonic proportions on the side lines $BC$, $CA$, $AB$ by the given those points on each line.)
Comment: dan_fulea, good answer!
Answer: Comment:
As can be seen in figure point X and Line A'C' is common for triangles ABC and PQR, this may help.
Comment: Hey Sirous , thanks for your hint but how it helps for L,M,N ?


On git pull, "unable to resolve reference" and "Undefined error: 0"

Question: My morning started with git pull, just like any other morning:
$ git pull
error: unable to resolve reference refs/remotes/origin/card-declined: Undefined error: 0
From github.com:instaedu/instaedu
 ! [new branch]      card-declined -> origin/card-declined  (unable to update local ref)

(FYI, "card-declined" is a branch I pushed to GitHub just last night, and hasn't had any activity since then.)
I've never seen that error before, so I looked it up and found this question: git pull fails "unalble to resolve reference" "unable to update local ref". The top-voted answer there says to try running git gc --prune=now, so I did that:
$ git gc --prune=now
error: bad ref for refs/remotes/origin/card-declined
error: bad ref for refs/remotes/origin/card-declined
fatal: loose object b28488de1ef3f6887eef99c0316a654d1803e6af (stored in .git/objects/b2/8488de1ef3f6887eef99c0316a654d1803e6af) is corrupt
error: failed to run repack

It didn't seem to fix anything, though now I see that something seems to have been corrupted since yesterday.
Any idea how to resolve this?
Comment: perhaps a fresh `git clone`? it looks like your repo is corrupt...
Comment: @nneonneo That may be what I end up doing, but I'm still curious to see if it can be fixed first.
Comment: possible duplicate of [git pull fails "unable to resolve reference" "unable to update local ref"](http://stackoverflow.com/questions/2998832/git-pull-fails-unable-to-resolve-reference-unable-to-update-local-ref)
Answer: Although I re-cloned the repository so I could continue working, I did keep the old corrupted one around, in case I happened to find a way to fix it. Unfortunately, I found no such fix.
If you run into this problem, your best bet is just to git clone the repository again. Hopefully you've already pushed your latest changes to a remote before the repo got corrupted.
Answer: I solved the issue by deleting the bad refs, e.g.:
del .git\refs\remotes\origin\card-declined
del .git\refs\remotes\origin\card-declined

It was mostly a hunch, but it seems to have worked because I did not care about the troublesome refs.


Function is printing multiple times

Question: I've written a function that takes as input a positive integer n and recursively returns a list of factorial values 1!, 2!, . . . , n!.
However, I can't get it to print the list just a single time. I've tried placing the print statement in various loops/locations and it always prints at least 3 times. I've also tried printing the function but it returns 'None', it only seems to print if I have  a print statement in the function. Why is this?
Here is my code:
def calc_Factorial(my_list, n, x, e):

    if not my_list:
        my_list.append(n-(n-1))
        calc_Factorial(my_list, n, x, e)
    else:
        if len(my_list) < n:
             my_list.append(my_list[e]*x)
             calc_Factorial(my_list, n, x+1, e+1)

    if len(my_list) == n:
        print(my_list)

e = 0
x = 2
n = int(input('Enter number: '))
my_list = []
calc_Factorial(my_list, n, x, e)

Comment: `my_list.append(n-(n-1))` : isn't this just 1? Also, why not just use a for loop instead? I don't know why you need to repeat the function over and over.
Comment: Yes it will always be 1, not sure exactly why I wrote it like that. And the idea was to practice recursion rather than just doing it with a for loop, which is why it looks messy.
Answer: The value is being printed multiple times because you are using print inside the recursive function. You should return instead of printing inside the recursive function.
def calc_Factorial(my_list, n, x, e):

  if not my_list:
    my_list.append(n-(n-1))
    return calc_Factorial(my_list, n, x, e)
  else:
    if len(my_list) < n:
         my_list.append(my_list[e]*x)
         return calc_Factorial(my_list, n, x+1, e+1)

    if len(my_list) == n:
        return (my_list)

e = 0
x = 2
n = int(input("Enter number: "))
my_list = []
print (calc_Factorial(my_list, n, x, e))

Answer: Change the print statement to return my_list:
def calc_Factorial(my_list, n, x, e):
    if not my_list:
        my_list.append(n-(n-1))
        calc_Factorial(my_list, n, x, e)
    else:
        if len(my_list) < n:
             my_list.append(my_list[e]*x)
             calc_Factorial(my_list, n, x+1, e+1)
    if len(my_list) == n:
        return my_list

>>> calc_Factorial(my_list, n, x, e)
[1, 2, 6, 24, 120, 720, 5040]
>>>

Answer: def calc_Factorial(my_list, n, x, e):

    if not my_list:
        my_list.append(n-(n-1))
        return calc_Factorial(my_list, n, x, e)
    else:
        if len(my_list) < n:
             my_list.append(my_list[e]*x)
             return calc_Factorial(my_list, n, x+1, e+1)

    if len(my_list) == n:
        print(my_list)
        return my_list

Your key problem here is that you should return calc_Factorial(my_list, n, x+1, e+1) every time when your call your recursive. If you don't return to the recursion, the rest of the code will be executed eventually. That's why you saw multiple print.
As long as you call recursive with return in this case. You should always only see one time print (even through you don't return my_list:
if len(my_list) == n:
    print(my_list)

But usually in the recursive function, people just return the value you want eventually. However, you can use print serve the purpose of debugging.
Answer: What you're seeing is because of this part:
if len(my_list) < n:
     my_list.append(my_list[e]*x)
     calc_Factorial(my_list, n, x+1, e+1)

if len(my_list) == n:
    print(my_list)

After the first if statement, the function will continue to execute, and now
n == len(my_list) so the list will be printed. (In effect this means that every call to the function will end up printing something)

Function is printing multiple times


How to run a background work in Oreo when a user unlocks the screen?

Question: Is there any way to run background work when user unlocks the screen in Android O (app is in background state)? As far as I know broadcast actions like android.intent.action.USER_PRESENT won't work anymore and JobScheduler has no conditions for such event. Thanks for any suggestions!
Answer:
Is there any way to run background work when user unlocks the screen
  in Android O (app is in background state)?

Unfortunately a normal app cannot monitor device unlock action. If your app is enabled as DeviceAdmin, then there is a chance to detect when user has unlocked the phone. But this will only be received the first time user succeed after having previously failed to unlock the device.


Mostrando dados ao digitar

Question: Preciso mostrar duas informações quando o usuário digitar o valor da comissão, no caso as informações que deveria mostrar seria IR e Valor líquido.
Por exemplo, se o usuário digitar 1.200,00, no label IR apareceria 18,00% (1.200 x 1,5%) e no label Valor líquido o total líquido (1.200 - 18) que no caso daria 1.182,00.
Acredito que teria que montar alguma função em javascript, mas sou leigo com essa linguagem, ou se existir alguma outra forma, toda ajuda será bem vinda.
Obs. a intenção é somente mostras os dois valores IR e Valor Líquido sem precisar salvar nada, pois o que preciso salvar mesmo seria o valor da comissão.
Meu código:
<form method="post"action="processa.php">
    Valor comissão:
    <input type="text" name="valor_comissao">

    IR:
    <input type="text">

    Valor Líquido:
    <input type="text">

    <button type="submit">Enviar</button>
</form>

Answer: Podes fazer isso usando o keyup no input assim:

const [valor, ir, liquido] = [...document.querySelectorAll('form input')];

/* ou em browsers antigos:
var inputs = document.querySelectorAll('form input');
var valor = inputs[0];
var ir = inputs[1];
var liquido = inputs[2];
*/

function formatar$(nr) {
  return nr.toLocaleString('pt-BR');
}

valor.addEventListener('keyup', function() {
  var nr = Number(this.value);
  var _ir = nr * 1.5 / 100;
  ir.value = _ir + '%';
  liquido.value = formatar$(nr - _ir);
});
<form method="post" action="processa.php">
  Valor comissão:
  <input type="text" name="valor_comissao">

  IR:
  <input type="text">

  Valor Líquido:
  <input type="text">

  <button type="submit">Enviar</button>
</form>
Comment: Desculpa a ignorância, mas precisa importar alguma biblioteca para usar essa função? não consegui executar a função aqui..kk
Comment: @SmokeRohden não precisas de nenhuma biblioteca, é tudo JavaScript nativo. Se não consegues pôr a funcionar cria um jsFiddle com um exemplo que eu ajudo. Mas como teste podes colocar o JavaScript depois do HTML na página.
Comment: Até dava para fazer a criação das variáveis dos `inputs` com *Destructuring*
Comment: Entendi. coloquei no final deu certo!
Comment: Obrigado @Sergio valeu mesmo
Comment: @Isac sim, ficava mais limpo `const [valor, ir, liquido] = [...document.querySelectorAll('form input')];`

Mostrando dados ao digitar


Are probability distributions for sample or population?

Question: When we take a sample, does the relative frequencies of sample  follow sample probability distributions or the population probability distribution ?
Also, why does events have to follow a certain distribution ? What natural laws force them to follow those underlying distribution ?
Answer: In principle, the sample set of data (having size, say, $M$) follows a corresponding sample distribution. For example, for a set of normally distributed data points with a priori unknown ensemble variance, this variance will need to be estimated from the data itself, which then gives rise to a theoretical t-distribution with $N$ degrees of freedom for comparison.
A major issue and difficulty is to be sure what is the precise degrees of freedom $N$ ($\leq M$), which is a parameter of the sampling distribution. Not all sample points are guaranteed to be independent (and identically distributed). If $N$ is large (and some other weak conditions are satisfied), then the central limit theorem may help to get an asymptotic distribution to approximate for the sampling distribution. If other parameter(s) of the distribution are large or small, these asymptotic distributions may in turn lead to further other asymptotic or limit distributions.
Usually, events do not "have to" follow certain distributions, particularly when these are based on models that were built for them, because models are usually approximations of reality themselves. However, if they do follow the derived or assumed distribution "closely", this increases the confidence in and accurate of that model, until a better (and possibly more complex) new model is used.


Boundness Theorem of continuous functions by finding a finite open cover

Question: I have a homework question that asks to prove that continuous functions are bounded on a close interval by utilizing the facts that (1) if $f$ is a continuous function then at each point of the function there exists an open interval on which it bounded (2) that the collection of all these open intervals is an open cover of some arbitrary closed interval $X$ and (3) that a finite subset of this open cover is also an open cover of $X$.
I've successfully proved (1) and (3), [(2) is given], and I suppose that I can prove the boundness theorem by performing a union on all the elements of (3); I just don't understand is why we couldn't perform such an operation on (2).
Answer: Suppose you have a cover $(U_i)_{i \in I}$ of your interval and you know that $f|_{U_i}$ is bounded by a constant, say $M_i$. Then $f$ is bounded by $\sup\limits_{i \in I} M_i$. If your cover $(U_i)_{i \in I}$ is not finite, you cannot guarantee that this supremum is finite. Therefore you choose a finite subcover and then, since the supremum of finitely many numbers is clearly finite, you get that $f$ is bounded.


Setting focus to an embedded Flash movie/HTML embed element using Javascript/jQuery

Question: Is there a way to set focus to the embed HTML element using JavaScript? Test case: embedded YouTube videos on a page.
I have no control over the embedded Flash element. So, is there a way to set focus on it by using only JavaScript?
I read somewhere that calling the element.focus() method works only in IE. I need a browser-independent way that works in Chrome/Firefox.
Thanks!
Answer: This only works in Internet Explorer.
http://kb2.adobe.com/cps/155/tn_15586.html
Answer: I've tried to do this too, and ended up to a nice solution using jquery:
var gotoflash=jQuery("#flash_file").offset().top;jQuery("html:not(:animated),body:not(:animated)").animate({ scrollTop: gotoflash}, 1000);
where:
 flash object code here

Comment: I tried this. But as far as I can comprehend, this is meant to scroll the page to the flash object. I actually want to set focus to the element.

So that, for eg., in YouTube, after setting focus to the element, I can use the keyboard shortcuts like "space" for playing/pausing the video
Comment: hm..i understood... try this,it works:  ,where OBJECTID is: ...
Answer: It can be done by adding flash content dynamically, for example with swfobject.
Answer: I haven't confirmed this, but you could try:
function thisMovie(movieName) {
    if (navigator.appName.indexOf("Microsoft") != -1) {
         return window[movieName];
    } else {
         return document[movieName];
    }
}

    // Call from within another function:
    thisMovie("FlashObjectID").focus();
    thisMovie("FlashObjectID").showFlash();
    // showFlash() is an AS3 ExternalInterface call from JS to .swf which establishes the TextInput.setFocus(); method

source: http://www.htmlforums.com/archive/index.php/t-64150.html


optmize a select query (if necessary creating index)

Question: I have 2 tables:
chats (id, ..., chat_status_id) // (about 28k records)
chat_messages(id, chat_id, send_date, ...) // (about 1 million records)

I need to get chats of certain status with latest message.
This is the select I am using, but it's pretty slow (it works in the end):
SELECT c.*,
       p1.*
FROM   chats c
       JOIN chat_messages p1
         ON ( c.id = p1.chat_id )
       LEFT OUTER JOIN chat_messages p2
                    ON ( c.id = p2.chat_id
                         AND ( p1.send_date < p2.send_date
                                OR ( p1.send_date = p2.send_date
                                     AND p1.id < p2.id ) ) )
WHERE  p2.id IS NULL
AND c.chat_status_id = 1
ORDER  BY p1.send_date DESC

I do not know howto optimize it.
Comment: Which table is `chat_status_id` in?  (Please qualify it.  And provide `SHOW CREATE TABLE`)
Comment: An additional _historical_ archive table. A search engine with more filtering. A cache of more popular messages (when there is a usage count). Also check the **explain** command. Also you do not seem to use JPA (like eclipseLink) with java entities. Which would cache java entities. (Though that is not necessarily an advise to use JPA.)
Comment: @JoopEggen I will chek all of your advices, except java (this is a php laravel project). thanks
Comment: Does this answer your question? [How to Optimize Queries in a Database - The Basics](https://stackoverflow.com/questions/3191623/how-to-optimize-queries-in-a-database-the-basics)
Comment: [What are your most common sql optimizations?](https://stackoverflow.com/q/1332778/3404097)
[When and why are database joins expensive?](https://stackoverflow.com/q/173726/3404097) [Tips for asking a good SQL question](https://meta.stackoverflow.com/q/271055/3404097) [Asking query performance questions](https://dba.meta.stackexchange.com/q/3034/43932)
[Re SQL "performance".](https://stackoverflow.com/a/24196511/3404097)
Answer: I would start with a few index updates.  First, your WHERE clause is based on the CHATS table status, that should be first in your index order, but can have the ID as well AFTER to support the joins.  Next, your messages table.  Similarly, those are JOINed based on the CHAT ID, not its own unique table ID as the primary consideration, but does use the ID as the carry-along for conditional testing of p1.id < p2.id.  Suggestions as follows.
Table          Index
Chats          ( Chat_Status_Id, id )
Chat_Messages  ( Chat_id, id, send_date )

Comment: I suggest a change to `Chat_Messages:  (chat_id, send_date,  id)`
Comment: @RickJames, I might agree, but if their data is such that the IDs are always in sequence relative to the send_date (via auto-increment, and not across multiple databases where that might not be the case), their query could be simplified with p1.ID < p2.id without concern of the date.  But agree send_date/id could be switched.
Answer: Give this a try:
SELECT  c.*, p1.*
    FROM  chats c
    JOIN  chat_messages p1  ON ( c.id = p1.chat_id )
    WHERE  NOT EXISTS
    (
        SELECT  1
            FROM  chat_messages p2
            WHERE  c.id = p2.chat_id
              AND  ( p1.send_date < p2.send_date
                      OR  ( p1.send_date = p2.send_date
                              AND  p1.id < p2.id ) )
    )
    WHERE  c.chat_status_id = 1
    ORDER BY  p1.send_date DESC

With
chats:  INDEX(chat_status_id, id)
chat_messages:  INDEX(chat_id, send_date,  id)


PHP : replace double spaces only between non-whitespace characters

Question: I have a date in this format (extracted from a specific script), on which I'd like to remove ALL spaces :
$date="Date:         Tue Aug  2 10:43"

Quite easy, but the trick is: prior to this, I'd like to add a "0" before the "2" (or any other 9th first days of the month), BUT the "0" has to replace the second space between "Aug" and "2".
What is the best way to achieve this ?
Keep in mind that the date will (obviously) change each day, so we can't simply do something like this:
$date=str_replace("Aug  2","Aug 02",$date);

Instead, I think the best way would be to do something like:
$date=str_replace("[x]  [x]","[x] 0[x]",$date);

[x] meaning: "Any non-whitespace character" (please excuse me for this approximation !)
Answer: use date() and strtotime() to do this mission
$date = strtotime('Tue    Aug     2 10:43'); //white spaces won't effect
echo date('D M 0\2 h:i',$date);
// output the date just replace the 2 with your 9th of month letter

Comment: Must be the "nicest" solution of all, but, in this specific case, Akremon solution suited me just fine, as it was the easiest for me to implement. Many thanks for you help anyway !
Answer: I saw two possibilities :

Use a regex function.preg-replace.php
Use a DateTime::createFromFormat and then use Datetime::format() datetime.createfromformat.php
Answer: Hm, maybe thet solution?
$date=preg_replace("/(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)  (\d)(\s+)/","$1 0$2$3",$date);

Comment: Maybe not the "nicest" solution, but actually the simplest to implement into my code. Worked like intended. Many Thanks !
Answer: Looking at your input data, it seems that your specific script produces a formatted output - meaning that it uses padding with spaces. This means that probably the length of the string is always the same - regardless of the actual date and time in it. If this is true, then you can use a very simple code:
if($date{22} == ' ') $date{22} = '0'; // replace space with zero
$date = preg_replace('/  +/', ' ', $date); // convert multiple spaces into single space

However, if your specific script does not produce a formatted otput then you will have to use something little different, but again very simple:
$date = preg_replace('/  +/', ' ', $date); // convert multiple spaces into single space
$arr = explode(' ', $date); // split text into words by spaces
if($arr[3] < 10) $arr[3] = '0'.$arr[3];
$date = implode(' ', $arr); // combine words

Comment: I used Lakremon solution, but thanks anyway for your comment and your time, hope it shall help somedy in the future.


JS objects attached to HTML elements

Question: I've been getting more into oop with js (I'm classically trained in Java) and I have four boxes. I want to do something like this:

<\div>

<\div>

<\div>

<\div>

   Function Box(){
       //PRIVATE VAR
    var count=0;
   }
   Box.prototype.move = function(){
         this.css({left:50});
   }

   Box.prototype.click= function(){
         this.count++;
    }

    //create new instances of box and assign or connect to the four HTML elements
   For (var i = 0; i < $('.box').length;  i++){
          $('.box')[i] = new Box();
   }

Basically I want each box to have its own private click count property that gets incremented each time it gets clicked.
Is this a desirable pattern or is it a poor practice to connect an HTML element like this? Should I be passing HTML elements to the functions instead? And if so, how would I keep an objects private click var in-sync with a specific element.
I obviously know there are simpler jQuery ways to do this but I want to focus on oop.
Comment: count is a local variable and not a property and your Function has a capital F.
Comment: So I'm doing this on my
Comment: iPhone and it autocorrected that. That's supposed to be almost like pseudo code...I don't need syntactical help I need the overall idea of what I'm trying to accomplish...and why it's not currently working
Comment: And count has to be a private var
Comment: You will need to pass in a reference to the element and you need to learn that nothing is private in that pattern. `this.count` in the click method is going to be `undefined`.
Answer: Here is a fiddle explaining what u want achieve.
https://jsfiddle.net/2e24a3kx/
First, you need to find all boxes.
Then for every box, create new box, pass a reference to DOM object, and connect handler.
   var boxes = $.find('.box');
   for(var i = 0; i < boxes.length;  i++) {
       var box = new Box();
       box.$view = $(boxes[i]);
       box.connectClick();
   }

Count is "private variable". It is created as local scope variable at calling function Box. Box function "remembers" all local scope variables. With creation of object with new, you need functions created locally (this.increateCounter) so they have access to count variable. With getCounter / increaseCounter functions in Box, you have access to counter.
In function connectClick im passing to jQuery variable a function with binded (this). Beacuse this is our object - box. While jquery at connecting handlers change this variable to DOM. Function Bind changes function this variable to our Box, bind returns new function.
 https://www.google.pl/search?q=mdn+Bind&oq=mdn+Bind&aqs=chrome..69i57j69i60j69i59j69i61j69i59l2.1082j0j7&sourceid=chrome&es_sm=93&ie=UTF-8
  function Box(){
       //PRIVATE VAR
       this.$view;
       this.increaseCounter = function () {
           count++;
       };

       this.getCounter = function () {
           return count;
       }
        var count=0;
   }

Jquery is diffrent, not simpler. Its just more structural way. While you model it in OOP u have to do a little more effort to maintain some structure.
I think the best pattern is to use pure JavaScript function addEventListener(event, object),
where you can directly pass as object Box.
Box have then implement function onEventReceived (event)
http://www.thecssninja.com/javascript/handleevent

JS objects attached to HTML elements


Error while calculating dot product in Theano

Question: I have the following simple code written in Theano and I am getting error while compiling function f:
<code>import theano.tensor as T
x = T.dmatrix('x')
w = T.dmatrix('w')
y = T.dot(x,w)

f = function([x,w],y)

ImportError: ('The following error happened while compiling the node', Dot22(x, w), '\n', 'dlopen(../.theano/compiledir_Darwin-13.3.0-x86_64-i386-64bit-i386-2.7.8-64/tmpTqQ0iS/b6135ddd19d268ad3182850548638ec3.so, 2): Library not loaded: libmkl_intel_lp64.dylib\n  Referenced from: ../.theano/compiledir_Darwin-13.3.0-x86_64-i386-64bit-i386-2.7.8-64/tmpTqQ0iS/b6135ddd19d268ad3182850548638ec3.so\n  Reason: image not found', '[Dot22(x, w)]')
</code>
What is going wrong on my side?
Answer: From the error message you get it seems like theano cannot find the mkl library file libmkl_intel_lp64.dylib.
How do you configure your theano? especially, the blas library?
Please check this section of theano installation manual.
Comment: @Lanc read the installation manual I linked to. it's all there.
Comment: Thanks!! How do I check if Blas is configured properly?


Prove that $\sum_{1\le i<j\le n}\frac{x_ix_j}{(1-x_i)(1-x_j)} \le \frac{n(n-1)}{2(2n-1)^2}$

Question: Let $n \ge 2$ be a an integer and $x_1,...,x_n$ are positive reals such that $$\sum_{i=1}^nx_i=\frac{1}{2}$$
Prove that
$$\sum_{1\le i<j\le n}\frac{x_ix_j}{(1-x_i)(1-x_j)}\le \frac{n(n-1)}{2(2n-1)^2}$$
Here is the source of the problem (in french) here
Edit:
I'll present my best bound yet on $$\sum_{1\le i<j\le 1}\frac{x_ix_j}{(1-x_i)(1-x_j)}=\frac{1}{2}\left(\sum_{k=1}^n\frac{x_k}{1-x_k}\right)^2-\frac{1}{2}\sum_{k=1}^n\frac{x_k^2}{(1-x_k)^2}$$
This formula was derived in @GCab's Answer.
First let $a_k=x_k/(1-x_k)$
so we want to prove
$$\left(\sum_{k=1}^na_k\right)^2-\sum_{k=1}^na_k^2\le \frac{n(n-1)}{(2n-1)^2}$$
But since $$\frac{x_k}{1-x_k}<2x_k\implies \sum_{k=1}^na_k<1 \quad (1)$$
Hence $$\left(\sum_{k=1}^na_k\right)^2\le \sum_{k=1}^na_k$$
Meaning $$\left(\sum_{k=1}^na_k\right)^2-\sum_{k=1}^na_k^2\le\sum_{k=1}^na_k(1-a_k)$$
Now consider the following function $$f(x)=\frac{x}{1-x}\left(1-\frac{x}{1-x}\right)$$
$f$ is concave on $(0,1)$ and by the tangent line trick we have $$f(x)\le f'(a)(x-a)+f(a)$$
set $a=1/2n$ to get $$a_k(1-a_k)\le\frac{4n^2\left(2n-3\right)}{\left(2n-1\right)^3}\left(x_k-\frac{1}{2n}\right)+ \frac{2(n-1)}{(2n-1)^2}$$
Now we sum to finish $$\sum_{k=1}^na_k(1-a_k)\le \frac{2n(n-1)}{(2n-1)^2}$$
Maybe by tweaking $(1)$ a little bit we can get rid of this factor of $2$
Comment: @PNT Could you give some more information on the contest? I.e. what knowledge is assumed / What is the level of the contest? Who is it aimed at? This could greatly restrict the possible techniques
Comment: Are you not going to accept Dr. Mathva's answer, especially considering his is the first correct answer, having introduced a general inequality proving method and not being awarded any bounty?
Comment: This is a highschool contest. So I'm fairly sure the official solution doesn't use his method. I want an elementary solution that I can possibly use in upcoming contests @Hans
Comment: From a math olympiad, I'll post the original problem just wait a minute.
Comment: Is it from an active (on-going) contest? In that case see https://math.meta.stackexchange.com/q/16774/42969
Comment: No, you can read the date in the paper. It says 6 May so it's yesterday
Comment: It's aimed for high school students, for knowledge I think it requires just basic inequalities like CS or AM-GM. @Dr.Mathva
Comment: @PNT: Will you get the solutions once the points have been awarded?
Comment: @PNT Does it works with $\frac{x}{1-x}-\left(x+2x^{2}\right)\leq 0$ for $0\leq x \leq 1/2$ ?
Comment: @PNT I think you have a typo when you set $a=1/(2n )$ is it not $a_k$ instead of $x_k$ ? And with strong convexity we have a better result as $$f'\left(\frac{1}{2n}\right)\left(x-\frac{1}{2n}\right)-\frac{\left(x-\frac{1}{2n}\right)^{2}f''\left(\frac{1}{2n}\right)}{2}+f\left(\frac{1}{2n}\right)$$
Comment: Would it get rid of the factor of 2? @ErikSatie
Comment: @PNT I haven't tried but for sure it's better .
Answer: Edit. Eventhough I had in mind the version of the ACM I needed, it all messed up when I tried to find a reference. As for the valid version, I will refer to Vasile Cirtoaje's Algebraic Inequalities. Old and new Methods, p. $267$.

We will employ a powerful technique developed by Vasile Cirtoaje back in $2006$ called the Arithmetic Compensation Method (see, for instance, this document).
Let to this end $$F(x_1, \ldots, x_n):=\sum_{1\leqslant i<j\leqslant n}\frac{x_ix_j}{(1-x_i)(1-x_j)}$$ which is clearly symmetric and continuous on $S:=\left\{(x_1, \ldots,x_n)\mid \sum_{i=1}^n x_i=\frac12, \forall i: x_i\geqslant 0\right\}$. We will now refer to the Remark 1.1 from the document linked above, which basically states that

If $$F(x_1,x_2,x_3,\ldots,x_n)>F\left(\frac{x_1+x_2}2, \frac{x_1+x_2}{2}, x_3,\ldots,x_n\right)\label{(i)}\tag{i}$$
implies $F(x_1,x_2,x_3,\ldots,x_n)\leqslant F(0, x_1+x_2, x_3, \ldots, x_n)$, then $$F(x_1,x_2,x_3,\ldots,x_n)\leqslant \underset{1\leqslant k\leqslant n}\max F\left(\frac1{2k}, \ldots, \frac1{2k},0,\ldots ,0\right)$$

Notice that (\ref{(i)}) is equivalent to (where we will denote, for brevity, $y:=\frac{1}2(x_1+x_2)$)
\begin{align*}
\frac{x_1x_2}{(1-x_1)(1-x_2)}-\frac{y^2}{(1-y)^2}+\left(\frac{x_1}{1-x_1}+\frac{x_2}{1-x_2}-\frac{2y}{1-y}\right)\sum_{i=3}^n \frac{x_i}{1-x_i}>0\\
\iff \frac{y^2-x_1x_2}{(1-x_1)(1-x_2)(1-y)^2}\left[2y-1+2(1-y)\sum_{i=3}^n \frac{x_i}{1-x_i}\right]>0\\
\iff 2y-1+2(1-y)\sum_{i=3}^n \frac{x_i}{1-x_i}>0
\end{align*}
Where the last equivalence follows from $y^2-x_1x_2\geqslant 0$. We will now turn to the second inequality:
\begin{align*}
F(x_1,x_2,x_3,\ldots,x_n)- F(0, x_1+x_2, x_3, \ldots, x_n)\leqslant0\\
\iff \frac{x_1x_2}{(1-x_1)(1-x_2)}+\left(\frac{x_1}{1-x_1}+\frac{x_2}{1-x_2}-\frac{2y}{1-y}\right)\sum_{i=3}^n \frac{x_i}{1-x_i}\leqslant0\\
\iff \frac{x_1x_2}{(1-x_1)(1-x_2)(1-2y)}\left[1-2y+2(y-1)\sum_{i=3}^n \frac{x_i}{1-x_i}\right]\leqslant 0\\
\end{align*}
Which is easily seen to follow from (\ref{(i)}). Thus, we conclude that \begin{align*}
F(x_1,x_2,x_3,\ldots,x_n)&\leqslant \underset{1\leqslant k\leqslant n}\max F\left(\frac1{2k}, \ldots, \frac1{2k},0,\ldots ,0\right)\\
&= \underset{1\leqslant k\leqslant n}\max  \binom{k}{2}\frac{1}{(2k-1)^2}\\
&= \underset{1\leqslant k\leqslant n}\max  \frac{k(k-1)}{2(2k-1)^2}
\end{align*}
But $f: x\mapsto \frac{x(x-1)}{2(2x-1)^2}$ is increasing on $[1,\infty)$, and, hence, the result follows.
Remark. Based on the difficulty of the other contest problems, this solution seems a bit overkill to me, but I have failed in the attempt to find a simpler method, since most well-known inequalities work in the other direction...
Comment: I'm confused... Don't we want to prove $\leq \frac{k(k-1)}{2(2k-1)^2}$?
Comment: +1, wonderful proof and method. Would you mind I ask you a question regarding the paper https://www.ijpam.eu/contents/2012-80-3/6/6.pdf you quoted before? Did the author make a small mistake in Equation (3.6) in that the summand thereof should be $\displaystyle\frac1{(mx_i-x_1x_2y)(mx_i-t^2y)}$? It seems the author misses the factor $mx_i-t^2y$ on the denominator.
Comment: Just wondering: For the proof of the ACM, the iteration process can in pinciple be infinitely long, or? Starting with $x_1>...>x_n$, we have $y_1=y_2=\frac{x_1+x_2}{2}$ and $x_1>y_1=y_2>y_3>...y_n=x_n$. Repeating the step successively with $y_2$ and $y_3$ will give a nested sequence $z_{1,k},...,z_{n,k}$, where $z_{m,0}=x_m$ and $z_{m,1}=y_m$, with the properties $$x_1>z_{1,k}>...>z_{n,k}>x_n \, .$$ $z_{1,k}$ decreases monotonically, and $z_{n,k}$ increases monotonically in $k$, while both being bounded below/above. Thus we have convergence, but why does it converge to the same number?
Comment: @Diger: Did you see Lemma 2.1-2.3 in https://www.ijpam.eu/contents/2012-80-3/6/6.pdf?
Comment: Nitpicking: As I understand it, the used "AC method" requires the domain $S$ to be *compact.*
Comment: @MartinR You're absolutely right: we need compactness in order to guarantee that the minimum (or maximum) is attained. However, we should not worry too much about this issue here, as proving the inequality for $x_i\,{\color{red}{\geqslant}}\, 0$ trivially implies that the inequality is true for strictly positive $x_i$.
Comment: I like the method. But could you please answer @Diger's question that the direction of your final inequality appears reversed?
Comment: @Diger, sorry for that. I had the right version of the ACM in mind, but quoted the wrong one. I've edited it
Comment: @Hans, sorry for that. See the edit :)
Comment: No, but thanks ;)
Answer: You should be able to look for the maximum, by the method of Lagrange. We set
$$F(x_1,...,x_n) \equiv \sum_{i\neq j} \frac{x_i x_j}{(1-x_i)(1-x_j)} \stackrel{\text{wts.}}{\leq} \frac{n(n-1)}{(2n-1)^2} \, ,$$
and define the Lagrange-function as $$L(x_1,...,x_n) = F(x_1,...,x_n) - \lambda \left(x_1+...+x_n-1/2\right) \,.$$
At an extremum we have $$\nabla L = 0 \qquad , \qquad \nabla=\begin{pmatrix} \partial_{x_1} \\ \vdots \\ \partial_{x_n} \\ \partial_\lambda\end{pmatrix} \, ,$$
in coordinates
\begin{align}\frac{2}{(1-x_k)^2} \sum_{\substack{i=1 \\ i\neq k}}^n \frac{x_i}{1-x_i} &= \lambda \quad\text{for}\quad k=1,...,n  \tag{1} \\
\sum_{i=1}^n x_i &= 1/2 \tag{2} \, .\end{align}
Multiplying Equation (1) by $\frac{(1-x_k)^2}{2}$ and adding $\frac{x_k}{1-x_k}$ to both sides gives
$$c=\sum_{i=1}^n \frac{x_i}{1-x_i} = \frac{\lambda}{2} (1-x_k)^2 + \frac{x_k}{1-x_k} \tag{3}$$
where the sum on the LHS is now independent on $k$ and we can consider it as a constant $c$. We now sum (3) from $k=1$ to $n$
\begin{align}
nc&=\frac{\lambda}{2} \left( n - 1 + \sum_{k=1}^n x_k^2\right) + c \\
\Rightarrow \quad c&= \frac{\lambda}{2} + \frac{\lambda}{2(n-1)} \sum_{k=1}^n x_k^2 > \frac{\lambda}{2} \tag{4}
\end{align}
which we will need to use in a second. Rearranging Equation (3) gives a monic polynomial in $x_k$
$$x_k^3 - 3x_k^2 + \left( 3 - \frac{2+2c}{\lambda} \right)x_k + \frac{2c}{\lambda} - 1 = 0 \, . \tag{5}$$
A cubic can have either only 1 real solution, in which case there is nothing to show, since then $x_1=...=x_n$ follows, or it can have 3 real solutions. In this case there appears to be a great variety of combinations of the three roots for the various $x_k$. However, since the product of the three roots is equal to $1-\frac{2c}{\lambda}<0$ by Equation (4) and (5), either all three roots are negative (discard), or only one root is negative and we are left with two choices for the various $x_k$. But then, since the sum of all three roots is equal to $3$ by Equation (5) and one root is negative, the sum of the two positive roots must be strictly greater than $3$. This implies, that at least one of them is strictly greater than $1/2$, which stands in contradiction to Equation (2). Thus, we are left with only one valid real solution satisfying Equation (2) and $$x_1=...=x_n=x$$
follows. It is then easy to see that $x=1/2n$.
It remains to show, that this extremal point is actually a maximum. The hessian of $L$ is given in coordinates by
$$\frac{1}{2} \, {\rm Hess}_{k,l} (L) = \frac{2\delta_{k,l}}{(1-x_k)^3} \sum_{i\neq k} \frac{x_i}{1-x_i} + \frac{1-\delta_{k,l}}{(1-x_k)^2(1-x_l)^2} \qquad \text{for} \qquad k,l=1,...,n \\
\frac{1}{2} \, {\rm Hess}_{n+1,k} (L) = \frac{1}{2} \, {\rm Hess}_{k,n+1} (L) = -1 \qquad \text{for} \qquad k=1,...,n \\
\frac{1}{2} \, {\rm Hess}_{n+1,n+1} (L) = 0 \, .$$
With $x_k=x=1/2n$ the first line becomes
$$\frac{1}{2} \, {\rm Hess}_{k,l} (L) = \frac{1-\frac{\delta_{k,l}}{n}}{\left(1-\frac{1}{2n}\right)^4} \qquad \text{for} \qquad k,l=1,...,n$$
and in matrix-form this looks like
$$\frac{1}{2} \, {\rm Hess} (L) = \frac{1}{\left(1-\frac{1}{2n}\right)^4} \begin{pmatrix}
1-1/n & 1 & \dots & 1 & -1 \\
1 & 1-1/n & \dots & 1 & -1 \\
\vdots & \vdots & \ddots & \vdots & \vdots \\
1 & 1 & \dots & 1-1/n & -1 \\
-1 & -1 & \dots & -1 & 0  \end{pmatrix} \, .$$
Then, defining the variational vector $\epsilon=(\epsilon_1,...,\epsilon_n,\epsilon_\lambda)^t$, we find for the second variation
$$\frac{1}{2} \sum_{k,l=1}^{n+1} \epsilon_k \, {\rm Hess}_{k,l} (L) \, \epsilon_l = - \frac{\sum_{k=1}^n \epsilon_k^2/n}{\left(1-\frac{1}{2n}\right)^4}<0$$ $\forall \epsilon \in {\mathbb R}^{n+1}$ satisfying
$$\sum_{k=1}^n \epsilon_k = 0 \, ,$$
which is a consequence of Equation (2).
Comment: I'm not familiar with lagrange multipliers. Are you sure there is no other method using Jensen or tangent line?
Comment: Probably there must be some trivial solution, if I interpret @G Cab's "hint" correctly, but unfortunately he doesn't want to share ;). While his hint is pretty obvious, it did not help me in any way to find such simple solution. Lagrange multipliers is the first thing I think of, when trying to maximize/minimize something, given a constraint.
Comment: +1. Nice Lagrange multiplier proof.
Answer: I found the AoPS link https://artofproblemsolving.com/community/c6h2503722p21150538.
It is a problem from "Problems From the Book", 2008, Ch. 2, which was proposed by Vasile Cartoaje. The proof there is very nice. I put the proof here.
Let us write the inequality as
$$\left(\sum_{i=1}^n \frac{x_i}{1 - x_i}\right)^2 \le \sum_{i=1}^n \frac{x_i^2}{(1 - x_i)^2} + \frac{n(n-1)}{(2n-1)^2}.$$
Using Cauchy-Bunyakovsky-Schwarz inequality, we have
$$\left(\sum_{i=1}^n \frac{x_i}{1 - x_i}\right)^2
\le \left(\sum_{i=1}^n x_i\right)
\left(\sum_{i=1}^n \frac{x_i}{(1-x_i)^2}\right)
= \sum_{i=1}^n \frac{x_i/2}{(1-x_i)^2}.$$
Thus, it suffices to prove that
$$\sum_{i=1}^n \frac{x_i/2}{(1-x_i)^2} \le \sum_{i=1}^n \frac{x_i^2}{(1 - x_i)^2} + \frac{n(n-1)}{(2n-1)^2}$$
or
$$\sum_{i=1}^n \frac{x_i(1 - 2x_i)}{(1-x_i)^2} \le \frac{2n(n-1)}{(2n-1)^2}.$$
Note that $x \mapsto \frac{x(1-2x)}{(1-x)^2}$ is concave on $[0, 1/2]$.
Using Jensen's inequality, the desired result follows.
Comment: I didn't expect seeing a problem from this Book in a math olympiad contest.
Comment: @PNT Yes. Also, it seems an old problem according to the book.
Comment: I expected such a proof, but thanks very much for sharing.
Comment: @Diger Thanks. The proof uses calculus. My second proof uses no calculus.
Answer: Hint:
Indicating as $S_n , \, T_n$
$$
\begin{array}{l}
 T_n  = \sum\limits_{1 \le i < j \le n} {a_i a_j }  \\
 S_n  = \sum\limits_{1 \le i,j \le n} {a_i a_j }
  = \sum\limits_{1 \le i \le n} {\sum\limits_{1 \le j \le n} {a_i a_j } }
  = \sum\limits_{1 \le i \le n} {a_i } \sum\limits_{1 \le j \le n} {a_j }
  = \left( {\sum\limits_{1 \le i \le n} {a_i } } \right)^2  \\
 \end{array}
$$
then you also have that
$$
\begin{array}{l}
 S_n  = \sum\limits_{1 \le i,j \le n} {a_i a_j }
  = \sum\limits_{1 \le i < j \le n} {a_i a_j }  + \sum\limits_{1 \le i = j \le n} {a_i a_j }
  + \sum\limits_{1 \le j < i \le n} {a_i a_j }  =  \\
  = 2T_n  + \sum\limits_{1 \le i \le n} {a_i ^2 }  \\
 \end{array}
$$
Answer: My second proof: without using calculus
We use the following bound:
$$\frac{x_ix_j}{(1-x_i)(1-x_j)}
\le \frac{1}{(2n-1)^4} + \frac{16n^2(n-1)^2}{(2n-1)^4}\cdot \frac{x_ix_j}{1-x_i-x_j}. \tag{1}$$
(See Remarks at the end for details.)
Taking $\sum_{1\le i < j \le n}$ on (1), it suffices to prove that
$$\frac{1}{(2n-1)^4}\cdot \frac{n(n-1)}{2} + \frac{16n^2(n-1)^2}{(2n-1)^4}\sum_{1\le i < j \le n} \frac{x_ix_j}{1-x_i-x_j} \le \frac{n(n-1)}{2(2n-1)^2} $$
or
$$\sum_{1\le i < j \le n} \frac{x_ix_j}{1-x_i-x_j} \le \frac18. \tag{2}$$
@Sangchul Lee gave a very nice proof for (2) as follows (I rewrote it compactly).
See: Prove or disprove $\sum\limits_{1\le i < j \le n} \frac{x_ix_j}{1-x_i-x_j} \le \frac18$ for $\sum\limits_{i=1}^n x_i = \frac12$($x_i\ge 0, \forall i$)
WLOG, assume that $x_i < 1/2$ for all $i$.
Using Cauchy-Bunyakovsky-Schwarz inequality, we have
$$\left(\frac{1}{1/2 - x_i} + \frac{1}{1/2 - x_j}\right)(1/2 - x_i + 1/2 - x_j) \ge 4$$
which results in $$\frac{1}{1 - x_i - x_j} \le \frac14\left(\frac{1}{1/2 - x_i} + \frac{1}{1/2 - x_j}\right) = \frac{1}{2 - 4x_i} + \frac{1}{2 - 4x_j}.$$
Thus, we have
\begin{align*}
 \sum_{1\le i < j \le n} \frac{x_ix_j}{1-x_i-x_j}
 &\le \sum_{1\le i < j \le n} \left(\frac{1}{2 - 4x_i} + \frac{1}{2 - 4x_j}\right)x_ix_j \\
 &= \frac12 \sum_{i, j : \, i\ne j} \left(\frac{1}{2 - 4x_i} + \frac{1}{2 - 4x_j}\right)x_ix_j \qquad\qquad (\mathrm{symmetry})\\
 &= \frac12\left[\sum_{i,j} \left(\frac{1}{2 - 4x_i} + \frac{1}{2 - 4x_j}\right)x_ix_j - \sum_{i=1}^n  \frac{2x_i^2}{2 - 4x_i} \right]\\
    &= \frac12\left[\sum_{i=1}^n \frac{x_i}{2 - 4x_i}\sum_{j=1}^n x_j + \sum_{i=1}^n x_i \sum_{j=1}^n \frac{x_j}{2-4x_j}- \sum_{i=1}^n  \frac{2x_i^2}{2 - 4x_i} \right]\\
 &= \frac12\left[\frac12\sum_{i=1}^n \frac{x_i}{2 - 4x_i} + \frac12 \sum_{j=1}^n \frac{x_j}{2-4x_j}- \sum_{i=1}^n  \frac{2x_i^2}{2 - 4x_i} \right]\\
&= \frac12\sum_{i=1}^n \left(\frac12 \frac{x_i}{2 - 4x_i} + \frac12  \frac{x_i}{2-4x_i}-   \frac{2x_i^2}{2 - 4x_i} \right)\\
 &= \frac12 \sum_{i=1}^n \frac{x_i}{2}\\
 &= \frac18. \tag{3}
\end{align*}
We are done.

Remarks:
The idea behind (1) is given as follows.
First, when $x_1 = x_2 = \cdots = x_n = \frac{1}{2n}$, the original inequality occurs with equality.
When $x_i = x_j = \frac{1}{2n}$,
we have $x_ix_j/(1-x_i-x_j) = \frac{1}{4n(n-1)}$.
Using $\frac{1}{1+u} \ge \frac{1}{1+v} - \frac{1}{(1+v)^2}(u-v)$ for $u = x_ix_j/(1-x_i-x_j)$ and $v=\frac{1}{4n(n-1)}$, we have
\begin{align*}
 \frac{x_ix_j}{(1-x_i)(1-x_j)} &= 1 - \frac{1}{1 + x_ix_j/(1-x_i-x_j)}\\
 &\le \frac{1}{(2n-1)^4} + \frac{16n^2(n-1)^2}{(2n-1)^4}\cdot \frac{x_ix_j}{1-x_i-x_j}.
\end{align*}
Comment: @Diger It is just $\left(\frac{1}{1/2 - u} + \frac{1}{1/2 - v}\right)(1/2 - u + 1/2 - v) \ge 4$.
Comment: How do you prove $$\frac{1}{1 - u - v} \le \frac14(\frac{1}{1/2 - u} + \frac{1}{1/2 - v})$$ using CS? I see that this is the AM-HM inequality, but not sure about CS.


Interchange limit and integral

Question: suppose we have a sequence of continuous functions $(f_n)_{n\geq 1}$ with $f_n:[0,\infty)\rightarrow \mathbb{R}$ such that $\int_{0}^{\infty}\vert f_n(x) \vert<\infty$ for all $n\in\mathbb{R}$. Further, let $\lim\limits_{n\rightarrow \infty}f_n(x) = 0$ for all $x\geq 0$.
Is it true that $$\lim\limits_{n\rightarrow \infty}\int_{0}^{\infty}f_n(x)\cdot g(x) dx = 0,$$
if $g:[0,\infty)\rightarrow \mathbb{R}$ is a continuous and absolutely integrable function (i.e. $\int_{0}^{\infty}\vert g(x) \vert<\infty$)? Can I interchange the order of integration and the limit? (If needed, we may also assume that $\int_{0}^{\infty}\vert f_n(x)\cdot g(x) \vert<\infty$ for all $n$).
Best regards
Answer: If $f_n\to 0$ uniformly then the answer is affirmative (see the comments below). In general the answer is no, however. Here is a counter example. Consider the sequence
$$f_n(x)=e^{-(x-n)^2+x}\;,$$
and let $g(x)=e^{-x}$. Then $f_n\to 0$ pointwise but not uniformly on $[0,\infty)$. By translation invariance of $dx$, we have
$$\lim_{n\to \infty}\int_0^{\infty}e^{-(x-n)^2+x}e^{-x}dx=\lim_{n\to \infty}\int_0^{\infty}e^{-(x-n)^2}dx =\lim_{n\to\infty}\int_{-n}^{\infty}e^{-y^2}dy=\sqrt{\pi}$$
Comment: You do not really need uniform convergence. It suffices to have $|f_n (x)| \leq C $ for all $n $ and (almost) all $x $.
Comment: But if $|f_n (x)| \leq C$ for all $x, n$, then $|f_n (x) \cdot g(x)| \leq C \cdot |g(x)|$, where the right-hand side is in $L^1$. Also, $f_n (x) \cdot g(x) \to 0$ pointwise, so that the dominated convergence theorem yields $\int f_n (x) \cdot g(x) \, dx \to 0$.
Comment: Yes, I did not want to say anything against your counterexample. I only objected to the sentence "unless $f_n \to 0$ uniformly, the answer is no". This sounds as if the claim was **always** false, unless we have uniform convergence. But in fact, it suffices to have $|f_n(x)| \leq C$ (which is implied by uniform convergence). (BTW: +1)
Comment: @PhoemueX Yes you're right. I made a mistake. My $f_n$'s are not uniformly bounded...thanks for pointing that out.

Interchange limit and integral


Can support vector machine be used in large data?

Question: With the limited knowledge I have on SVM, it is good for a short and fat data matrix $X$, (lots of features, and not too many instances), but not for big data.
I understand one reason is the Kernel Matrix $K$ is a $n \times n$ matrix where, $n$ is number of instances in the data. If we have say, 100K data, the kernel matrix $K$ will have $10^{10}$ elements, and may take ~80G of memory.
Is there any modification of SVM that can be used in large data? (Say on the scale of 100K to 1M data points?)
Comment: It would help potential respondents if you were to discuss the objective of the SVM beyond merely "large data." That said and knowing nothing else about your query, is there any reason an SVM can't be leveraged into a divide and conquer algorithm?
Comment: What are you using the SVM for? Could you use an alternative method?
Answer: As you mention, storing the kernel matrix requires memory that scales quadratically with the number of data points. Training time for traditional SVM algorithms also scales superlinearly with the number of data points. So, these algorithms aren't feasible for large data sets.
One possible trick is to reformulate a kernelized SVM as a linear SVM. Each element $K_{ij}$ of the kernel matrix represents the dot product between data points $x_i$ and $x_j$ after mapping them (possibly nonlinearly) into a feature space: $K_{ij} = \Phi(x_i) \cdot \Phi(x_j)$. The feature space mapping $\Phi$ is defined implicitly by the kernel function, and kernelized SVMs don't explicitly compute feature space representations. This is computationally efficient for small to medium size datasets, as the feature space can be very high dimensional, or even infinite dimensional. But, as above, this becomes infeasible for large datasets. Instead, we can explicitly map the data nonlinearly into feature space, then efficiently train a linear SVM on the feature space representations. The feature space mapping can be constructed to approximate a given kernel function, but use fewer dimensions than the 'full' feature space mapping. For large datasets, this can still give us rich feature space representations, but with many fewer dimensions than data points.
One approach to kernel approximation uses the Nyström approximation (Williams and Seeger 2001). This is a way to approximate the eigenvalues/eigenvectors of a large matrix using a smaller submatrix. Another approach uses randomized features, and is somtimes called 'random kitchen sinks' (Rahimi and Recht 2007).
Another trick for training SVMs on large datasets is to approximate the optimization problem with a set of smaller subproblems. For example, using stochastic gradient descent on the primal problem is one approach (among many others). Much work has been done on the optimization front. Menon (2009) gives a good survey.
References
Williams and Seeger (2001). Using the Nystroem method to speed up kernel machines.
Rahimi and Recht (2007). Random features for large-scale kernel machines.
Menon (2009). Large-scale support vector machines: Algorithms and theory.


IPTables: restrict outgoing protocols for specific ip range

Question: Regarding the OUTPUT chain: I want to have a default policy of ACCEPT, but for a specific destination CIDR, I want to allow outgoing connections only via SSH. Incoming traffic from that CIDR must remain unrestricted. Currently the OUTPUT chain has no restrictions at all.
I'm thinking this:
iptables -F OUTPUT
iptables -P OUTPUT ACCEPT
iptables -A OUTPUT -o eth0 -p tcp -d myCIDR --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -o eth0 -p tcp -d myCIDR -m state --state NEW -j REJECT

Am I close?
Thanks!
Answer: Your question is unclear. Do you want to allow only SSH connections?
If so, you can use these rules:
iptables -A INPUT -p tcp -s cidr_here --dport 22 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp --sport 22 -m conntrack --ctstate ESTABLISHED -j ACCEPT

You have written the 22nd port to the OUTPUT chain with --dport (destination port). It is not a destination port in this case. It should be a source port.
Comment: Actually I think my question is quite clear, I only want to allow ssh connections from that server to that CIDR. Connections to that CIDR over other tcp ports must be blocked. Regarding the port, when one initiates an ssh connection it's the destination port that is 22, isn't it? Look at example #8 in this link: https://crm.vpscheap.net/index.php?rp=/knowledgebase/29/25-Most-Frequently-Used-Linux-IPTables-Rules-Examples.html
Comment: Sorry, I understood what you meant now. There is a website. It is portquiz.net. I used its IP address for testing purpose.
I used these 2 IPtables rules:

`iptables -A OUTPUT -o eth0 -p tcp -d 184.108.40.206/32 --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT` and
`iptables -A OUTPUT -o eth0 -p tcp -d 184.108.40.206/32 -m state --state NEW -j REJECT`

This command is working fine: `nc -v 18.104.22.168 22`
This command is not working (as expected): `nc -v 22.214.171.124 80`

Do you have another rule in the NAT chain? You can check it with: iptables -vnL -t nat
Comment: Yes, there is a problem with the rejecting part, and I can't spot it.
Comment: Actually I take that back, I tried it on a test server of my own and it seems to work. I can ssh to the specific CIDR, but I can't connect to port 80 or any other non-22 port.


How to start a new Redmine installation with new issue numbers from a preset number?

Question: I lost my Redmine database but retained the repositories which have issue numbers in the git commit messages.
What I want to do is to create a new repo but set the starting number for the issues from an arbitrary number, eg issue 2000.
But I also want to create issues whose number I can set below the starting point, eg 1001 etc, so that I can link them to the commits which are already in the repository logs.
One option I can think of is to prepopulate the database with dummy subjects and descriptions and change the subjects and descriptions afterwards. The other option parse the commit logs for the issue numbers and their notes and enter them as the issues subjects and descriptions.
It is something like this possible and are there some tools are libraries which can help accomplish this?
Comment: Too bad that you lost your original DB.

I would recommend writing a script that creates issues based upon your git commit messages, that way you will still be able to see relation between issue and committed revisions.
Answer: Redmine uses a normal database like mysql so you could set a initial value for the autoincrement within the issue id.
How to change the autoincrement:
https://stackoverflow.com/a/1485685/4607988
Now I have more time :)
Open the database - hopefully mysql-  and type in this query:
<code>ALTER TABLE issues AUTO_INCREMENT=2001;
</code>
Then your problem should be solved.


Linear (mixed-effects) model for skewed and negative data / residuals

Question: I am currently fitting a linear mixed-effects model to my data where the outcome variable can have both positive and negative values (integers).
The issue now is, that my outcome data is skewed with a  heavier tail in the positive direction. As a consequence, my residuals (as plotted with a qqnorm plot) deviate from a qq line in both directions.
Usually, I would go ahead and do something like a log-transformation, but in this case this is not possible as I have both positive and negative data. Using a generalized linear mixed-effects model does not solve my problem as well due to similar reasons (negative data).
A further thing to note is that I have a lot of data (multiple millions of samples).
Comment: 1. If your data is ordinal, "negative" makes no sense.  You can just add a constant because the numbers are meaningless.  2. Data cannot be skewed in both directions. 3. Log transformation of ordinal data makes no sense.
Comment: 1.) You are right, it is not ordinal per-se; it is a simple integer variable with positive and negative values. 2.) Skewed positively
Answer:
The most likely effect of heavy-tailed residuals will be to give you unbiased point estimates with overly optimistic (too-narrow) confidence intervals and (too-small) p-values.  Depending on your goals, it might not matter that much if you have millions of data points ...
If you actually have ordinal data, you should consider fitting an ordinal model! e.g ordinal::clmm() in R.
The robustlmm package in R fits robust LMMs (see the vignette, i.e. vignette("rlmer",package="robustlmm") once you have downloaded/installed/loaded the packages).
Comment: I have updated my question regarding that I am not actually have ordinal data. Regarding your first point: I am mostly interested in the coefficients and p-values.


Finding The Central Tendency From A List Of Averages

Question: Say you have data on the average food consumed per type of animal for a 30 day period. Not the total amount of food itself, but the averages per day per that type of animal.
Also assume the number of dogs, cats, and snakes would vary and are not consistent between groups or across days within groups, but what that variation and samples are exactly are not known from the data.
For instance the data could look like this:
<code>                     dogs, cats, snakes
Avg Food lbs. Day 1    2    1.5    .3
Avg Food lbs. Day 2    3.1  1.2    .43
Avg Food lbs. Day 3    1.9  1.3    .2
Avg Food lbs. Day 4    2.2  1.6    .25

etc etc
</code>
How can I calculate the central tendency for each group, and how can I compare groups' respective central tendencies? Would taking the median of the averages accomplish the first part of the question?
If these questions can't be answered given the above constraints, what more information would be necessary to find each respective list of averages' central tendencies?
Comment: Because the number of animals can vary by day and species, you obviously can't use an ordinary ANOVA which requires equal variances. But you may get some useful tentative results from `oneway.test` in R, which does not require equal variances for the three species. // Fictitious data: `set.seed(804);  y1 = rnorm(30, 2.3, .5);  y2 = rnorm(30, 1.2, .2); y3 = rnorm(30, .3, .1)`; format: `y=c(y1,y2,y2);  gp = rep(1:3, each=30)` then analysis of means uses; `oneway.test(y~gp)`, which returns significant P-value $\approx 0.$ // If your data significant, then follow up with properly designed expt.
Answer: Since the group sizes are NOT assumed to all be equal here, you need to know the sample sizes in each individual group (or the proportion of data in each group) to compute the averages you want.
If you had that info, then you could compute the averages using the law of total expectation.
Comment: Applying the "the proportion of data in each group" is in essence a weighted average, yes?
Comment: @JLuu Yes, absolutely. The proportions serve as the weights in a weighted average. That is what the law of total expectation says.


Filtering a List of SelectListItem to create list SelectList with unique values

Question: I want to filter out items from the select list if they are already in the list of items belonging to a particular entity.
So if an entity already has an item, the item wont be available to choose from in the list.
Error in code below:

Argument 1: cannot convert from 'int' to
  System.Collections.Generic.KeyValuePair<int, string>

 //list of subject areas belonging to a specific course
 public List<KeyValuePair<int, string>> SubjectAreas { get; set; } = new
 List<KeyValuePair<int, string>>();

 //list of all subject areas available
 public List<SelectListItem> SubjectAreasForDropdown { get; set; }

 //the below throw the error mentioned above
 SubjectAreasForDropdown.ForEach(
     a => a.Selected = SubjectAreas
     .Contains(Convert.ToInt32(a.Value)));

Answer: SubjectAreas is a List<KeyValuePair<int, string>>
When you do contains, It will check for a KeyValuePair Object, where you are supplying int.
I feel you need to use
a => a.Selected = SubjectAreas[Convert.ToInt32(a.Value)]
Also could you brief what are you trying to acheive
Comment: Cannot implicitly convert type 'System.Collections.Generic.KeyValuePair' to 'bool'
Answer: Try this:
a => a.Selected = SubjectAreas.Any(x=>x.Key == Convert.ToInt32(a.Value))

Comment: Thanks but whilst this has no errors it doesn't give the desired result. The goal is to only choose elements for the select list that aren't contained in subjectAreas
Comment: After ForEach, you can use Selected == false to find all not exists item. Have you try?


Using Autoencoders to generate equal feature length for variable length data

Question: Autoencoders can be used to generate a fixed-length feature representation of the input data. Now consider the case that your input data (e. g. videos) has unequal length. So you would train autoencoders for your individual data points in order to engineer the features of your input data.
Is that a "valid" approach in terms of stochastic properties? Because I am wondering whether this reduction from e. g. 2 minutes material to a fixed number of features compared to a video of 30 seconds is valid to generate a reasonable model. What would be alternatives to this approach?
Answer: It's valid in the sense that you can do it. In fact the most basic sequence to sequence translation / autoencoder models do the same thing -- encoding a variable length sequence in a fixed size latent space. It's not too different from setting a video encoder to produce a fixed size output. You are correct that too much variation might produce weird results. This is usually mitigated by attention mechanisms.
Alternatively, with a sequence to (latent) sequence to sequence model, you could produce a variable length latent space.


Issue with the combination of ssh, bash --login and exec

Question: Here are few situations I'm in.
foo is the ssh server to which password less login is set up.
$ ssh foo
$ bash --login -c 'exec $0 "$@"' ls -lrt

It's working as expected.
ssh foo bash --login -c ls -lrt

This is not working as expected. Only ls is working ignoring the -lrt parameter.
And my requirement is to run the 1st example with ssh like
ssh foo bash --login -c 'exec $0 "$@"' ls -lrt

I know I've problems from the 2nd example itself. But I was just trying something to make it work and to find where the problem is. The 3rd example is not showing any error/output at all.
How do I proceed to achieve the 3rd example?
Comment: Why the complication of `ssh foo bash --login -c ls -lrt` over a simpler `ssh foo 'ls -lrt'` ?
Answer: You can do ssh foo bash --login -c '"exec \$0 \$@"' ls -lrt.
To find out what actually happens, strace is your friend.
ssh foo strace -vff -e execve bash --login -c 'exec $0 "$@"' ls -lrt gives me something like:
execve("/bin/bash", ["bash", "--login", "-c", "exec", "zsh", "ls", "-lrt"], [LANG=…]
My shell is zsh, which should give you a hint to what's happening here: That $0 is interpreted by the shell spawned to parse the command line sent over the SSH command.  If we just escape the $0 and "$@" parts like: ssh foo  strace -vff -e execve bash --login -c 'exec \$0 "\$@"' ls -lrt we get:
execve("/bin/bash", ["bash", "--login", "-c", "exec", "$0", "$@", "ls", "-lrt"], ["LANG=... As you can see here, bash is called with the arguments --login, -c, exec, $0 and so on. This doesn't get us all the way to where we want to be, since it's now effectively just calling exec by itself.  To work around this, we need to quote what comes after the -c argument so it's not unpacked too early, like ssh foo strace -vff -e execve bash --login -c '"exec \$0 \$@"' ls -lrt:
execve("/bin/bash", ["bash", "--login", "-c", "exec $0 $@", "ls", "-lrt"], ["LANG=…
This is accepted by bash, since it allows you to use $@, but if you're writing proper POSIX sh, you should use "$@", in which case the right answer is ssh foo strace -vff -e execve bash --login -c '"exec \$0 \"\$@\""' ls -lrt:
execve("/bin/bash", ["bash", "--login", "-c", "exec $0 \"$@\"", "ls", "-lrt"], ["LANG=…
I assume you're using ls -lrt as a proxy for some other more complex command that actually needs bash --login and such, else just do what  suggested in the first comment and use ssh foo 'ls -lrt' instead.
Comment: Thanks. You are right. That's just a proxy. The command I'm using is to use a different library and the command itself is 2-3 lines long. Thanks for the `strace` information. I was searching for something like that but couldn't word a proper search string in google.
Answer: The ssh accepts exactly one argument as a command, therefore it should be in the quotes or apostrophes. The same applies for the bash -c argument -- it is just a single argument that is accepted. The others are passed to the bash instead of ls. The following example works just fine:
ssh foo 'bash --login -c "ls -lrt"'


Java 8 stream - melhora de performance

Question: Estou implementado um método que recebe um valor inteiro k (que representa a quantidade de "vagas") e duas listas (p e q) de Integer e efetua algumas operações.
Utilizando stream, verifico quais elementos da lista p possuem valor >= que os valores da lista q. Caso exista, preciso armazenar a posição (índice + 1) de cada item numa outra lista. Contudo esta terceira lista só pode ter uma quantidade de elementos <= o valor k passado por parâmetro.
O código implementado encontra-se abaixo, contudo preciso melhorar o desempenho dele (acredito que os 2 forEach existentes estão piorando o desempenho).
public static List<Integer> kthPerson(int k, List<Integer> p, List<Integer> q) {

        List<Integer> busList = new ArrayList<>();

        q.stream().forEach(time -> {
            List<Integer> lista = new ArrayList<>();

            IntStream.range(0, p.size()).filter(i -> (p.get(i) >= time && lista.size() < k)).forEach(i -> lista.add(i+1) );
            int size = lista.size();
            if(size < k){
               busList.add(0);
            }else{
                if (lista != null && !lista.isEmpty()) {
                    busList.add(lista.get(size - 1));
                }
            }
        });
        return busList;

    }

}

Comment: @MarcosAntônio eu vi sua questão e as respostas. Obrigado por recomendar o link. Como você fez para remover os casos repetidos? Você implementou utilizando for aninhados ou usou stream?
Answer: Dada a sua descrição, acho que seu código não está certo:

verifico quais elementos da lista p possuem valor >= que os valores da lista q. Caso exista, preciso armazenar a posição (índice + 1) de cada item numa outra lista. Contudo esta terceira lista só pode ter uma quantidade de elementos <= o valor k

Testei o seu código com as listas abaixo:
List<Integer> p = Arrays.asList(10, 20, 30, 40, 50, 60, 70, 80, 90);
List<Integer> q = Arrays.asList(10, 20, 30, 40, 50);
System.out.println(kthPerson(3, p, q));

E o resultado foi:
[3, 4, 5, 6, 7]

Só que a posição 3 (que corresponde ao índice 2) de p é o número 30, que não é maior que os valores de q (já que q possui os valores 40 e 50, que são maiores que 30). Além disso, o resultado tem 5 elementos, que é maior que k.
Enfim, não ficou muito claro o que é para fazer. Eu entendi de duas maneiras:

verificar os elementos de p que são maiores que todos os elementos de q
verificar os elementos de p que são maiores que o elemento de q que está na mesma posição

Veremos soluções para cada caso.

Opção 1
Se a ideia é verificar os elementos de p que são maiores que todos os elementos de q, então não precisa percorrer q várias vezes. Basta achar o maior elemento de q e verificar quais elementos de p são maiores ou iguais a ele. Depois basta pegar os primeiros k elementos e retornar a lista:
public static List<Integer> kthPersonOpcao1Stream(int k, List<Integer> p, List<Integer> q) {
    // encontrar o maior elemento de q
    int maxQ = Collections.max(q);
    return IntStream
        // iterar pelos índices de p
        .range(0, p.size())
        // pegar os elementos maiores ou iguais a maxQ
        .filter(i -> p.get(i) >= maxQ)
        // pegar somente os k primeiros
        .limit(k)
        // somar 1 ao índice
        .map(i -> i + 1)
        // converter os valores int para Integer
        .boxed()
        // coletar os valores em uma List
        .collect(Collectors.toList());
}

Mas se a preocupação é desempenho, talvez você não devesse usar streams, já que elas são mais lentas que um loop tradicional. Claro que para poucas listas pequenas não fará tanta diferença assim, mas de qualquer forma, a solução sem stream seria bem simples:
public static List<Integer> kthPersonOpcao1Loop(int k, List<Integer> p, List<Integer> q) {
    int maxQ = Collections.max(q);
    List<Integer> result = new ArrayList<>();
    for (int i = 0; i < p.size() && result.size() < k; i++) {
        if (p.get(i) >= maxQ) {
            result.add(i + 1);
        }
    }
    return result;
}

Repare na condição do for (i < p.size() && result.size() < k), que verifica se eu já comparei todos os elementos de p e se a lista de resultados tem k elementos. Assim ele já cobre também os casos em que são encontrados menos que k elementos (pois k é o tamanho máximo que a lista de resultados pode ter, mas nada garante que sempre encontrará k elementos).

Opção 2
Se a ideia é comparar cada elemento de p com o elemento de q que está na mesma posição, primeiro precisamos saber qual das listas é menor (por exemplo, se p tiver 10 elementos e q tiver 4, eu não preciso verificar os 10 elementos de p, basta verificar os 4 primeiros).
O código é bem similar à opção 1, a diferença é que em vez de comparar os elementos de p com o maior elemento de q, eu comparo apenas com os que estão na mesma posição:
// com stream
public static List<Integer> kthPersonOpcao2Stream(int k, List<Integer> p, List<Integer> q) {
    // só preciso iterar até o tamanho da menor das listas
    int size = Math.min(p.size(), q.size());
    return IntStream
        // iterar pelos índices até "size"
        .range(0, size)
        // pegar os elementos de p maiores ou iguais ao elemento de q na mesma posição
        .filter(i -> p.get(i) >= q.get(i))
        // pegar somente os k primeiros
        .limit(k)
        // somar 1 ao índice
        .map(i -> i + 1)
        // converter os valores int para Integer
        .boxed()
        // coletar os valores em uma List
        .collect(Collectors.toList());
}

// sem stream
public static List<Integer> kthPersonOpcao2Loop(int k, List<Integer> p, List<Integer> q) {
    int size = Math.min(p.size(), q.size());
    List<Integer> result = new ArrayList<>();
    for (int i = 0; i < size && result.size() < k; i++) {
        if (p.get(i) >= q.get(i)) {
            result.add(i + 1);
        }
    }
    return result;
}

Comment: Muito obrigado pela sua resposta.
Comment: @Yan Se a resposta resolveu o seu problema, vc pode aceitá-la, [**veja aqui**](https://pt.meta.stackoverflow.com/q/1078/112052) como e porque fazê-lo. Não é obrigatório, mas é uma boa prática do site, para indicar a futuros visitantes que aquilo resolveu o problema. Não esqueça que [**vc também pode votar**](/help/privileges/vote-up) na resposta, caso tenha achado útil.
Answer: Encontrei uma boa solução para esse problema nessa pargunta How to find all K-th elements for each expiration time, o melhor tempo obtido foi utilizando Arrays ao invés de List/Streams. Caso o problema te forneça os dados em List é só fazer um parse para Array e no final realiza outro parse de volta para List, só dessa forma consegui passar em todos os caso de teste.
public static int[] kth(int k, int[] t, int[] e) {
        int[] kth = new int[e.length];

        if (k > t.length)
            return kth;

        Arrays.sort(e);
        int c = 0;

        for (int i = 0; i < e.length; i++) {
            for (int j = 0; j < t.length; j++) {
                if (t[j] >= e[i])
                    c++;

                if (c >= k) {
                    kth[i] = j + 1;
                    break;
                }
            }

            if (c < k)
                break;

            c = 0;
        }

        return kth;
}

Comment: @MarcosAntônio eu vi sua questão e as respostas. Obrigado por recomendar o link. Como você fez para remover os casos repetidos? Você implementou utilizando for aninhados ou usou stream?
Comment: @Yan eu removi os elementos repetidos usando Stream e no final criei o array List com todos os elementos

Java 8 stream - melhora de performance


Cache a block by a custom entity rendered within

Question: I struggle with getting a custom block cached correctly:

we have a custom entity "Politician"
For that entity we have several routes where we want to render a "politician header" with some basic information about the politician
the politician header is a custom template which is rendered in a custom block
the problem is: when caching is activated the block always shows the politician data from the first politician which was rendered.

Here is the code for the block plugin - to keep it simple I do not copy the method getPoliticianFromUrl() here:
/**
 * Provides a block with the profile header for a politician. We do not render
 * it on the public edit form path
 *
 * @Block(
 *   id = "aw_frontend_politician_header",
 *   admin_label = @Translation("Politician profile header"),
 *   category = "Abgeordnetenwatch"
 * )
 */
class PoliticianProfileHeader extends BlockBase {
  /**
   * {@inheritdoc}
   */
  public function build() {
    $politician = $this->getPoliticianFromUrl();
    if ($politician == NULL) {
      return [];
    }

    return array(
      '#theme' => 'politician_header',
      '#politician' => $politician,
      '#cache' => [
        'keys' => ['politician', $politician->id()]
      ]
    );
  }

  public function getCacheContexts() {
    $politician = $this->getPoliticianFromUrl();
    if ($politician != NULL) {
      return Cache::mergeContexts(parent::getCacheContexts(), $politician->getCacheContexts());
    }
    return parent::getCacheContexts();
  }

  public function getCacheTags() {
    $politician = $this->getPoliticianFromUrl();
    if ($politician != NULL) {
      return Cache::mergeTags(parent::getCacheTags(), $politician->getCacheTags());
    }
    return parent::getCacheTags();
  }
}

Then I render the politician header - again I just put in interesting parts:
function template_preprocess_politician_header(&$variables) {
  /** @var \Drupal\pw_basic\Entity\Politician $politician */
  $politician = $variables['politician'];
  /** @var \Drupal\Core\Render\Renderer $renderer */
  $renderer = \Drupal::service('renderer');
  $renderer->addCacheableDependency($variables, $politician);
  $variables["#cache"]["contexts"][] = 'user.roles';
  $variables['#cache']['keys'] = ['politician_header', $politician->id()];
}

I thought when I define cache keys this ensures that different versions of the render array are cached. This does not happen - not for the custom politician header template (for which I do not find any cached data in cache_render tanle - is this just cached with Twig caching?) and not for the block.
My idea was to create a custom cache context based on my Politician entity but as far as I understand cache contexts they depend on more global circumstances such as the active theme, the current user and things like that. I hoped the Entity API would generate such a context in a magical way so I tried $politician->getCacheContexts() but nothing.
What do I miss?
Comment: The key is the function you didn't show, getPoliticianFromUrl(), but judging based on the function name you have a cache dependency on the current URL, which means you need a cache context `url.path`.
Comment: Using "key" in the previous comment is a bit misleading, this was meant as key point, not cache key, which by the way you don't need at all, because the block has already cache keys and each time you add cache keys you get another cache entry, so this results in three cache entries which contain basically the same content.
Comment: No, getPoliticianFromUrl() is not the key point: the block is used on 5 or 6 different urls but the content of the block keeps the same, it depends on the poltician, not the url. So the cache context should be the politician otherwise we have 5 or 6 versions of the block with the same content.

With my cache keys "politician" und "$politician->id()" I thought I would get one version of the block for each politician. I will try "politician_123" (123 = id) as a key as this is what I think I need?! But as said: I defined the keys above and they are NOT used for the cid of the block
Comment: Yes, this is what I was trying to point out, the cache keys you've set are not contexts but store extra cache entries additional to the block. As context you can use a core context like url.path or route (if you retrieve the entity from a route parameter) or, to make caching more efficient, create a custom context, but for this to decide you need to know what getPoliticianFromUrl() is doing.
Comment: ... and what amount of data we are talking about. Caching is not an exact science, it is always a trade-off between cache efficiency and code complexity. Don't over-optimize.
Comment: But the cache keys DID NOT save additional cache entries/ versions of the block - this is what I tried to explain. They had no influence on the cache items stored in cache_render for the block
Comment: Cache keys don't bubble up. If you put a set of cache keys in a render element it will be cached on its own. You have tried to build three levels of caching in the original code of the question, although I'm not sure you can set cache keys in a preprocess hook.
Comment: Ah thank you so much... I thought cache keys bubble up... but it is strange that the Block building class sets the cache key hard - I cannot change the cache key in my own plugin's build() method. This method gets called by another class where the cache keys are set without respecting the cache keys I defined in the render array :-(
Answer: By any reason I do not really get the block rendering to respect my cache keys set wherever. It only respects set cache contexts so I create a custom cache context:
In pw_basic.services.yml (pw_basic is the name ofthe module)
  cache_context.politician:
    class: Drupal\pw_basic\Cache\PoliticianCacheContext
    arguments: ['@current_route_match']
    tags:
      - { name: cache.context }

The politician cache context class:
class PoliticianCacheContext implements CacheContextInterface {

  use GetPoliticianFromUrlTrait;

  /**
   * @var \Drupal\pw_basic\Entity\Politician
   */
  protected $politician;

  public function __construct(CurrentRouteMatch $current_route_match) {
    $this->politician = $this->getPoliticianFromUrl();
  }

  /**
   * {@inheritdoc}
   */
  public static function getLabel() {
    return t('Politician');
  }

  /**
   * {@inheritdoc}
   */
  public function getContext() {
    $cache_context = '';
    if ($this->politician !== NULL) {
      $cache_context = $this->politician->id();
    }
    return $cache_context;
  }

  /**
   * {@inheritdoc}
   */
  public function getCacheableMetadata() {
    return new CacheableMetadata();
  }
}

Then I cleared the block plugin class from any cache logic and just added the following in template_preprocess_politician_header(), the preprocess function of the template rendered within the block:
$variables["#cache"]["contexts"][] = 'politician';

What I do not really like is the service character of the cache context classes. Why is it only possible to set the context in such a global way? Do I miss somethin? I need to use this global workaround and check for the politician from URL which makes this context not as flexible as I want it to be. Is there a way to give a variable to a cache context directly when defining it in a render array?