Fitting a power law to the relationship between two variables

Question: I have a small set of data that I need to fit a curve to  (see image and data below).

algaeConc    adultDens
2.934961    6665.145469
2.921015    6665.144453
2.910769    6665.142377
2.908331    6665.138925
2.907658    6665.134515
2.907278    6665.128736
2.907130    6665.121034
2.907160    6665.117992
2.907326    6665.124547
2.907594    6665.131266

The data came out of a simulation model for which I was stressing a parameter (filtration rate) then recording what happened to my variables (algae concentration and adult density). Now, I need to fit a curve to this, and I have to admit I never saw this kind of decline before. So, I would like some suggestions about how to fit a curve to this. It seems to me that a power law might be needed, but I have never worked with those before and am not sure how to go about it.
I've spent all day reading about power laws and trying to fit it on R but only managed to confuse myself, so any help will be highly appreciated.
Edit: The X axis is inverted to match the stress on the filtration parameter (increasing stress, caused a decrease in algae concentration).
Comment: Have you tried drawing what power law relationships look like?
Comment: I have. It seems to me that the formula below should be the correct one, but I'm not sure:
adult density = a * algae concentration^(-b)

Right now I'm trying to fit this formula in R using the poweRlaw package while I wait to see some cyber opinions, but no success. It think the last value in the table is bugging up the whole thing, because it rises instead of dropping further.
Comment: I've only just noticed that you have reversed your x-axis in the plot. *Why would you do that without saying you had done so*? Indeed why do it at all? Just showing the plot without being *very* clear you had flipped the axis is very likely to mislead people. If you want to lead answerers into giving bad advice, that's a good way to do it. Please unflip the plot or give a compelling reason in your text for flipping it around.
Comment: It could also be quite useful to people who might answer if you'd paste the data you plotted (in ASCII form) into your post. After you paste it in, you'll need to select it with the code tool at the top of the edit window to indent it 4 spaces so it maintains its line-by-line format
Comment: Thank you Glen. I have implemented the changes you asked for. Also, I'm sorry about the X axis, I plotted this way right away because of the stress test, and completly forgot to point it out.
Answer: Here's the unflipped version of your plot:

No choices of $a$ and $b$ in $y=ax^{-b}$ (where $y$ is adult density and $x$ is algae concentration)  will fit that shape of relationship; they don't increase and flatten off up to a positive asymptote:

(with negative $a$ you can get it to increase to an asymptote, but the fit will be all negative)
Indeed, the range of your $x$-variable is so narrow (relative to its mean say) that a typical power curve will look effectively like a straight line.

A quick power-law diagnostic plot:
If you don't have intuition for power laws, to see whether a power law is likely to be a reasonable description, plot log y against log x (it doesn't matter which base of logs). It should look close to linear on that scale.
In your case we can see immediately that won't happen (indeed the coefficients of variation are so small that the log-scale plot looks a lot like the first plot above aside from the axis scales).
There are a number of nonlinear functions used for this kind of "saturation" relationship we see in the plot of the data (dozens of them, really). It's best to proceed from theoretical notions of what is happening (which might lead to a differential equation, say, and then to a functional form) than to guess at functional forms; indeed that sort of theoretical understanding is where most of the common increasing-to-an-asymptote type models arise.

Here's a few examples of nonlinear models that increase to an asymptote (at least for some parameter values); most of them will fit your data poorly:
\begin{eqnarray}
E(y) &=& ax/(b+x)     \\
E(y) &=& a(1-\exp(-bx))   \\
E(y) &=& a (1 - \exp(-b(x - c))) \\
E(y) &=& a+(c-a)\exp(-bx) \\
E(y) &=& a - c(x-d)^{-b} \\
E(y) &=& a \exp(-b/x)
\end{eqnarray}
There are many others besides. You should be looking at subject-area knowledge to develop a model, though -- trawling through such a list of models to find one that fits effectively ruins any statistical inference on the same data, so unless you want to go collect another set of data (or indeed possibly several more), don't do that. Strictly speaking you should have a model in place before you even collect data.

Indeed there further consequences to using data-generated models / data-generated hypotheses that I haven't touched on there; there are a number of posts on site that discuss the effect of this kind of approach to choosing models.
In the situation where there's simply no subject-area knowledge to work form, and where the experimenter can collect any amount of data easily, such curve fitting strategies (trawling through a list) may have some value in helping to come up with plausible models -- as long as we take great care to make sure that the choice isn't just an artifact of our one set of data and we take similarly great care not to let the model-selection process affect our ultimate estimation and inference.
For example if we take enough data that we can do out of sample testing (i.e. we can see if the model performs well on data we didn't use to pick the model) and other people repeat the experiment (so we can see whether the same curve tends to be picked up across multiple independently-run experiments - that it's robust to the small changes in the setup that naturally occur when one goes to a different lab with their own pieces of equipment and different people running the experiment).
Once that's all done and the model actually can be seen to "work" in some sense, we can then sample an entirely new data set on which to perform estimation and inference (again, where possible, using out-of-sample assessment to avoid over-reliance on asymptotic approximations and on assuming the model is actually the data-generating process when computing thinks like intervals and standard errors). If we can't do something like that, such list trawling can be quite dangerous, leading to models that tell us something about the particulars of our data and our modelling process but little of value about the thing we want to model.
Comment: A good idea, I think. A literature search may help identify one or two potential models (and perhaps likely flaws), so you may be able to go into such a discussion with more context than you currently have.
Comment: Thanks for the explanation Glen. Indeed I underestimated the complexity of fitting a curve to this set of data. It seemed to be a straightforward case when I saw the scatter plot, but now I see that's not the case.Since I dont fully understand these formulations, I think I need to discuss the results further with my supervisors instead of hunting for a fit that I won't be able to discuss due to my inexperience. Once again, thanks.
Answer: I fit the ASCII text data you posted to over 600 known, named equations with four or less parameters using a genetic algorithm for initial parameter estimation, and the best candidates were mostly hyperbolic type equations.  Here is an example that I consider to be a good candidate equation:
y = x / (a + bx) + Offset

a =  6.8885679248813901E+05
b = -2.3699198001685570E+05
Offset =  6.6651456685672019E+03

Degrees of freedom (error): 7
Degrees of freedom (regression): 2
Chi-squared: 3.09776595618e-05
R-squared: 0.964515690092
R-squared adjusted: 0.954377315833
Model F-statistic: 95.1351435065
Model F-statistic p-value: 8.41641753468e-06
Model log-likelihood: 49.2347614958
AIC: -9.24695229916
BIC: -9.15617677126
Root Mean Squared Error (RMSE): 0.00176004714601

a = 6.8885679248813901E+05
       std err: 2.86714E+10
       t-stat: 4.06822E+00
       p-stat: 4.75990E-03
       95% confidence intervals: [2.88464E+05, 1.08925E+06]

b = -2.3699198001685570E+05
       std err: 3.39266E+09
       t-stat: -4.06877E+00
       p-stat: 4.75659E-03
       95% confidence intervals: [-3.74723E+05, -9.92607E+04]

Offset = 6.6651456685672019E+03
       std err: 2.13899E-06
       t-stat: 4.55728E+06
       p-stat: 0.00000E+00
       95% confidence intervals: [6.66514E+03, 6.66515E+03]

Coefficient Covariance Matrix
[  6.47885451e+15  -2.22866161e+15  -4.28731703e+07]
[ -2.22866161e+15   7.66637461e+14   1.47483491e+07]
[ -4.28731703e+07   1.47483491e+07   4.83345536e-01]

Comment: What the output's account of degrees of freedom misses is the "researcher degrees of freedom" used up in the search for a model across *600* options. The p-values here don't carry their usual properties or meaning (the same goes for standard errors and confidence intervals and a number of other quantities) -- as a simulation study would readily show. Indeed by ignoring the effect of the search on their properties, almost all of the information in the output is biased or misleading in one way or another (at least if interpreted in the usual way). ... the downvote wasn't mine, by the way


Find the function $g∈U$that minimises $\|f−g\|$

Question: Let $V=C\big([−1,1] ,\mathbb R\big)$ be the vector space of continuous real-valued functions defined on $[−1,1]$ with the usual addition and scalar multiplication of functions.  Equip with the inner product
$$\langle f,g\rangle =\int_{-1}^1\ f(x)\ g(x)\ dx\,.$$
Let $f\in C\big([−1,1] ,\mathbb R\big)$ be given by $$f(x) =\begin{cases}0 &\text{if }−1\leq x\leq 0\,,\\  x &\text{if }0\leq x\leq 1\,.\end{cases}$$
Let $B = \left\{\frac1{\sqrt2}, \cos(\pi x), \sin(\pi x)\right\} \subseteq V$ and $U = \text{span}(B)$.
Find the function $g∈U$that minimises $\|f−g\|$, i.e.,
$$\|f−g\|≤\|f−h\| \text{ for all }h\in U\,.$$
Not sure of this one im a bit confused and no where near the answer
Answer: $B$ is an orthonormal set. The answer is $\frac a {\sqrt 2} +b\cos (\pi x)+c \sin (\pi x)$ where $a=\langle f, \frac 1 {\sqrt 2} \rangle, b=\langle f, \cos (\pi x) \rangle, c=\langle f, \sin (\pi x) \rangle$. For example, $a=\int_0^{1} x \frac 1 {\sqrt 2} dx=\frac 1 {2\sqrt 2}$.


Drupal 7: hierarchy disabled setting on taxonomy vocabulary

Question: I would like to disable taxonomy hierarchy for a vocabulary.
In older versions of Drupal there used to be a setting on the vocabulary page:

Hierarchy: disabled

Can do I achieve this with Drupal 7?
Answer: I found how to do it. I ended up using hook_form_FORM_ID_alter on the taxonomy form in the admin area, like so:
<code>function hook_form_taxonomy_overview_terms_alter(&$form, &$form_state) {
        $form['#parent_fields'] = FALSE;
}
</code>


How is it possible to deliver email from qmail to a LMTP server?

Question: I have a qmail server, which cannot be changed into anything else, because it is part of a much larger system. It should route all emails to a Zarafa server, which only uses the LMTP protocol and to which I don't have physical access.
How is it possible to make qmail deliver the emails to the LMTP server?
Comment: Also, could someone please create the tag _lmtp_? Thanks!
Comment: And maybe the question should be migrated to ServerFault? (if that's possible without losing the bounty I just assigned to it)
Comment: This question seem better suited at serverfault.com
Answer: I don't see any indication that qmail supports LMTP.  Could you redirect to an exim process which runs the LMTP delivery.  This could be configured as a pipe delivery target.  Alternatively, exim could run on another port or server.
Comment: No, I couldn't. I'm trying to avoid installing Postfix/exim/whatever other email server besides qmail.
Comment: As far as I can see from the documentation I have seen you will need a shim of some sort which supports LMTP.  Exim does not need to be run as a mail server.  It can be configured to act only as an MDA.


Find the following limit $\lim_{x\to \infty} \operatorname{arccosh}(x) - \log_e x$

Question: Find the following limit using the fact that $ \operatorname{arccosh} (x) = \log_e \left(x + \sqrt {x^2-1}\right) $
$$ \lim_{x\to \infty} \operatorname{arccosh}(x) - \log_e x$$
Answer: Following the hint, since $\log A-\log B= \log \frac A B$, we have
$$\operatorname{arccosh}(x) - \log x=\log \left(1 + \frac{\sqrt {x^2-1}}x\right)$$
then it suffices to determine
$$\lim_{x\to \infty}\frac{\sqrt {x^2-1}}x$$
Answer: The required limit is $\ln{2}$.
\begin{align}
\lim_{x\to\infty} \text{arccosh(x)}-\ln{x} &=\lim_{x\to\infty} \ln{\left(x+\sqrt{x^2-1}\right)-\ln x}\\\\&= \lim_{x\to\infty} \ln{\left(1+\frac{\sqrt{x^2-1}}{x}\right)}\\\\
&= \ln{\left(1+\lim_{x\to\infty}\frac{\sqrt{x^2-1}}{x}\right)}\\\\
&= \ln{\left(1+\lim_{x\to\infty}\frac{|x|\sqrt{1-\frac{1}{x^2}}}{x}\right)}\\\\
&=\ln{(2)}
\end{align}


SQL Report Builder: get value from group within tablix

Question: Please forgive the vague title of this question. Perhaps the below will ask my question better.
Consider the below aggregated table:
Fruit   Units   FruitSales%
-----   -----   -----------
Apples     10             ?
Oranges    20             ?
Bananas    10             ?
NonFruit   10             ?

TOTAL      50             ?

I need the FruitSales% column to be: Fruit / (Total - NonFruit)
If NonFruit is a product name of its own, how do I get its value for use in other calculations in the tablix?
I imagine my formula for the FruitSales% is something like:
Sum(Fields!Units.Value) / (ReportItems!txtTotalUnits.Value - SumIf(Fields!Fruit = "NonFruit", Fields!Units.Value)

However, SumIf does not exist and even if it did, it would be specific to the current row.
And while I'm here, ReportItems!txtTotalUnits.Value, I have obviously named that text box, but is there a cleaner way to reference it?
Answer: Say the underlying DataSet (which I've called FruitDataSet) looks like this:

I've created a simple report based on this data:

The Fruit Sales % expression is:
=Sum(IIf(Fields!fruit.Value <> "NonFruit", Fields!units.Value, 0))
  / Sum(IIf(Fields!fruit.Value <> "NonFruit", Fields!units.Value, 0), "FruitDataSet")

This gives what I think is the correct results:

There are two things to note about the expression:

By running the Sum against an IIf expression, you can control what gets included in the totals - here I'm setting NonFruit explicitly to 0.
By setting the Scope of the aggregate expression, you can get overall totals to use to work out total percentages - in the report I'm getting a total using FruitDataSet and comparing this to the group-level total to get a % value.

The way you're referencing the textbox total is fine; the only other option would be to use an expression each time you want the total - if this is outside a Tablix you would need to explicitly set the Scope, e.g. the DataSet.
Comment: Thank you very much Ian, that is perfect and so well explained. I really appreciate the time you spent on this!


Is this method of finding the expected value of the square a random variable correct?

Question: Suppose x is a discrete random variable with values 2,3,1 and probabilities 0.2,0.3, and 0.4 respectively. NOw say we have the function y=x2+3 and we want to find the expected value of this equation. So, in my opinion, we first have to square all the values of x so they become 4,9 and 1 and then multiply them with their respective probabilities and then add them all up and in the end, just add 3 to the answer.
This works out to give the answer 6.9, but my teacher at the university solvedthis question another way. He said that since x2 is just x times x, the expected value of x squared should be the E(x) times E(x) so we can just multiply the value of x with its probability and then add em all up and then square the answer. This gives the answer 5.89. So which way is correct?
I'm pretty sure that the way I think it should be done is correct so if it is, then I'm gonna have to confront my teacher about this matter. And I just know that he'll state the rule that the expected value of the product of two or more random variables is equal to the product of their individual expected values.  How do I explain to him why this rule doesn't apply here?
I think it is because of the rule applies to two different random variables, but here x is the same random variable.
Comment: Just to clarify, you're calculating $\bar{y}$ rather than $\langle y \rangle$. You're calculating the average value of y over a sample, rather than the expected value of y
Comment: https://en.wikipedia.org/wiki/Law_of_the_unconscious_statistician
Comment: @Ferdi hmm.. can you elucidate?
Answer: Given
$E[X]=\sum_i x_i f(x_i)$
and
$E[X^2]=Var[X]+E[X]^2$
We can see that in general $E[X^2] \neq E[X]^2$, unless $Var[X]=0$
Comment: Great, never thought to use this relationship. this verifies my answer. thanks


Qual é a forma mais simples para criar uma máscara para números em PHP?

Question: Eu tenho duas maneiras de criar máscaras para números em PHP, porém não sei se é a mais elegante e eficaz, por isso gostaria de opniões para melhorar o código.
Preciso que a função se comporte da seguinte forma:

Formate os números com menos de 8 números preenchendo-os com zero.
Crie um separador (pode ser um hífen) no meio desses 8 números. Ou seja, 11112222 transformará em 1111-2222.

Tenho já esses dois exemplos:
Exemplo 1:
implode('-', str_split(sprintf('%08s', $numero), 4));

Exemplo 2
$formatado = sprintf('%08s', $numero);

substr($formatado, 0, 4) . '-' . substr($formatado, 4, 8);

Alguém sabe uma maneira melhor, em termos de performance e elegância, sem ter que chamar várias funções para criar uma máscara em PHP?
Answer: Uma forma de resolver isso, é combinar str_pad() para preencher os zeros a esquerda* caso a string tenha menos que 8 caracteres, chunk_split() para inserir o hifen a cada 4 caracteres e o trim() para remover o hifen adicionado no final.
<?php

function mascaraTelefone($numero){
    $numero = str_pad($numero, 8, '0', STR_PAD_LEFT);
    return trim(chunk_split($numero, 4, '-'), '-');
}

echo mascaraTelefone('9999') .PHP_EOL;
echo mascaraTelefone('99998888') .PHP_EOL;
echo mascaraTelefone('77') .PHP_EOL;
echo mascaraTelefone('666') .PHP_EOL;

Exemplo - ideone
Baseado em: Inserção de ponto a cada 5 caracteres
Uma variante desse código é trocar chunck_split() por substr_replace() que substitui uma cópia do caracter que está na posição X(terceiro argumento) pelo hifen no caso.
function mascaraTelefone($numero){
    $numero = str_pad($numero, 8, '0', STR_PAD_LEFT);
    return substr_replace($numero, '-', 4, 0);
}

* O comportamento padrão de str_pad() é adicionar os caracteres a direita, isso pode ser modificado ao informar o quarto argumento da função
que possuiu os valores:
STR_PAD_RIGHT|Adiciona a direita(padrão caso o quarto argumento seja omitido
STR_PAD_LEFT |Adiciona a esquerda
STR_PAD_BOTH |Adiciona na esquerda e na direita

Comment: quem é esse tal de `chunk_split`!? não me lembro dele
Comment: Ooooh! Que legal! Muito bacana, esse `chunk_split`!
Comment: Acho que vou adotar o `str_pad` também. Toda vez eu tenho que ficar utilizando o `sprintf` com aquela concatenação feia `"0{$size}s%"`.
Comment: No caso não deveria ter indicado no str_pad o pad_type como STR_PAD_LEFT? Por padrão será preenchido à direita do valor.
Comment: @MarcelodeAndrade bem lembrado, eu troquei o código e a explicação da resposta xD
Answer: Formatação genérica com máscara dinâmica:
function format_string($mask, $str, $ch = '#') {
    $c = 0;
    $rs = '';

    /*
    Aqui usamos strlen() pois não há preocupação com o charset da máscara.
    */
    for ($i = 0; $i < strlen($mask); $i++) {
        if ($mask[$i] == $ch) {
            $rs .= $str[$c];
            $c++;
        } else {
            $rs .= $mask[$i];
        }
    }

    return $rs;
}

$str = '05055344410'; // Exemplo para telefone
echo format_string('###-####-####', $str);
$str = '20150827'; // Exemplo para datas
echo '
' . format_string('####-##-##', $str);

Comment: O ponto que julgo mais interessante é a flexibilidade. Permite-se formatar strings para múltiplos propósitos, não somente números de telefones localizados. A inspiração não vem de JQuery, é um script que uso desde 2000 ou 1999. Não lembro. Nessa época não tinha stack, google, jquery. Era na raça mesmo.
Em muitos casos, o tamanho do código não significa melhor desempenho. Na primeira resposta há o uso de 3 funções. Dentro dessas 3 funções há todo um processamento, laços de repetição, talvez também uso de ER, etc. Teoricamente usa muito mais códigos. Isso custa mais processamento e memória.
Comment: A princípio eu pensei que seria muito código (em relação ao código do @rray). Mas reparei que a sua funcionalidade está parecida com o `jQuery.mask`, e isso é muito bom
Answer: Já está muito bem respondido, mas eu gostaria de deixar aqui uma pequena técnica que usa vsprintf e str_split juntos.
A função vsprintf tem como finalidade formatar uma string, baseando-se nos argumentos passados. Porém diferente de sprintf, ao invés de passar n parâmetros, você passa um array. Sendo assim, poderíamos, utilizar a função str_split para separar as strings e, com vsprintf, aplicar a formatação.
Nesse exemplo, passei um segundo argumento para str_split, que fará que a string seja dividida de 4 em 4, como é o caso dos telefones.
function formatar_telefone($telefone) {
     return vsprintf('%s-%s', str_split($telefone, 4));
}

É claro que haverá problemas caso o número do telefone não contenha 8 dígitos. Então nesse caso, pode ser adicionar uma verificação dentro da função, com um lançamento de um erro ou exceção. Ou basta apenas preencher os valores com 0, como o @rray sugeriu.
Nesse caso, poderíamos preencher usando sprintf para preencher com zeros, através da expressão %08d:
function formatar_telefone($telefone) {
     vsprintf('%s-%s%', str_split(sprintf('%08d', $telefone)));
}

A expressão %08d significa que você quer formatar um número, forçando a ser reconhecido como um dígito, e, caso tenha menos que 8 caracteres, será preenchido com 0.


How to find an orthonormal basis for $ P_{2}(\mathbb R)$ equipped with the inner product $\langle f,g\rangle =f(-1)g(-1)+f(0)g(0)+f(1)g(1)$?

Question: Given $ P_{2}(\mathbb R)$ the vector space of real polynomials of degree less than or equal to 2 equipped with the inner product $$\langle f,g\rangle=f(-1)g(-1)+f(0)g(0)+f(1)g(1).$$ How would I use the Gram-Schmidt process to find an orthonormal basis ?
Comment: Projection which is used in Gram Schmidt algorithm is defined with respect to the inner product, so start with an arbitrary basis for $P_2(\mathbb R)$, express the projection in terms of inner products and then carry out the calculations.
Comment: @Bib-lost Look again. The polynomial space in question is $P_2(\mathbb R)$.
Comment: The vector space of real polynomials has infinite dimension; Gram-Schmidt will not give you a result in a finite time.
Comment: I would guess $P_2$ is the subspace of order maximum 2.
Comment: What exactly don't you understand about how to apply G-S to this space?
Comment: @mathreadler yes thats right.
Comment: I would really appreciate an answer
Comment: Before you can get an answer, you must clarify @ amd's question:  *what* don't you understand about how to use Gram–Schmidt?  It's an algorithm, the application of which in this case @mathreadler has described.
Comment: I only know how to use gram Schmidt with vectors and I know for polynomials it included integration but because this space is equipped with this inner product I am not sure what to integrate
Comment: So your problem is dat you do not know how to calculate the inner product of two polynomials?
Comment: As @mathreadler says, Gram–Schmidt inherently involves neither Euclidean vectors nor integration; it involves only taking inner products.  We have a familiar inner product on Euclidean vectors, and a familiar inner product on $C([0, 1])$ (or on the restriction of $P_2(\mathbb R)$ there) coming from integration, but there's nothing special about them.  Just substitute this new inner product where you would have used the old ones.
Comment: @Bib-lost, it sounds like the problem is not realising that there is an inner product to be computed at all; @ aprilanalysis refers to Gram–Schmidt for polynomials as involving integration, which suggests that he or she has internalised a particular inner product as a black-box operation, without realising its significance.
Comment: Integration is just one form of inner product that can be defined for functions. In this case the inner product is instead a pairwise product of pointwise values.
Comment: In the solution they take v1= 1 and then say that w1= 1/sqrt(3). I am very unsure about where the sqrt(3) comes from.
Comment: To get started, you need a spanning set (preferably a basis) for the inner product space to apply the Gram-Schmidt process (and get an orthonormal basis).  So choose a convenient basis for the real polynomials of degree at most two.
Answer: Take any basis, for instance the canonical basis $(1, X, X^2)$
First normalise $1$ :
We have $\langle 1,1\rangle = 1+1+1 = 3$ so the normalised vector is $e_1 = {1\over \sqrt{3}}$
Now find the part of $X$ orthogonal to $1\over \sqrt{3}$. That's easy since $\langle 1,X \rangle = 0$
And normalise $X$ : $$\langle X,X\rangle = 1+0+1 = 2$$ so the normalised vector is $$e_2={X\over \sqrt 2}$$
Finally find the part of $X^2$ orthogonal to $span(1,X)$, ie :
$$\begin{align}X^2 - \langle X^2 ,{X\over \sqrt2}\rangle{X\over \sqrt 2} - \langle X^2, {1\over \sqrt 3}\rangle {1\over \sqrt 3} &= X^2 - {1\over 2}(-1+0+1){X} + {1\over 3}(1+0+1)\\ &= X^2 - {2\over 3}\end{align}$$
Now normalise this vector : $$\langle X^2 - {2\over 3}, X^2 - {2\over 3}\rangle = \langle X^2, X^2\rangle -4\langle X^2, {1\over 3}\rangle + 4\langle {1\over 3}, {1\over 3}\rangle = 2-{8\over 3} + {4\over 3} = {2\over 3}$$
Thus $e_3 = \sqrt {3\over 2}\left(X^2-{2\over 3}\right)$
In conclusion Gram-Schmidt process applied to the canonical basis gives you the orthonormal basis : $$\left({1\over \sqrt 3}, {X\over \sqrt 2}, \sqrt {3\over 2}\left(X^2-{2\over 3}\right)\right)$$
Note : I highly recommend spending some time on it to make sure you understand the idea behind it and are confortable enough about it to do it yourself. One good exercise you can now do is apply the process on the same basis but treating the vectors in a different order (say $X^2$, then $X$ and finally $1$). This will give you a different basis than the one we obtained here.
Comment: @aprilanalysis: see my hint for how to apply the scalar product.
Comment: How did you get <1,1> = 1+1+1 =3 ? Other than that very nice answer thank you.
Comment: @mathreadler got it thanks.
Answer: Hint I can help you with first part of the problem
( First we may want to convince ourselves that we are actually dealing with an inner product here, but let's assume the question creator is not devious enough to lie about it being one. )
Let us first define projection with the scalar product between two polynomials, $f$, $g$, having coefficients $f_2,f_1,f_0$ and $g_2,g_1,g_0$ respectively:
$$f(x) = f_2 x^2 + f_1 x + f_0 \\g(x) = g_2 x^2 + g_1 x + g_0 $$
$$\langle f,g \rangle = f(-1)g(-1) + f(0)g(0)+f(1)g(1)$$
$$\begin{align*}f(-1)g(-1) &= /\text{convince yourself}/ = (f_2-f_1+f_0)(g_2-g_1+g_0)\\
f(0)g(0) &= /\text{convince yourself}/ = (f_0)(g_0)\\
f(1)g(1) &= /\text{convince yourself}/ = (f_2+f_1+f_0)(g_2+g_1+g_0)\end{align*}$$
Now you can gather all terms up and simplify and you have the first piece of the puzzle done.
Comment: still not sure actually. because this gives sqrt(11) when computed ?
Comment: What? You don't even mention to me what you are trying to calculate then give me a random number. How would you expect me to be able to help you then?
Comment: sorry about that. For <1,1> = 1+1+9 = sqrt(11) with this method ?
Comment: how do you insert the <1,1> in place of the
Comment: what does <1,1> assign to the $g_2,g_1,g_0,f_2,f_1,f_0$ ?
Comment: okay got it sorry.
Comment: i think there's a typo also for the last bracket is it not supposed to be g1 instead of f1?
Comment: Yep, you are right.


Como fazer a seguinte consulta sem usar INNER JOIN?

Question: Tenho a tabela A com os seguintes campos:
ID | ID_PAIS_ENVIO | ID_PAIS_RECIBO
1  | 23            | 47
//...

Tenho a tabela B com os seguintes campos:
ID | NOME_PAIS
23 | Brasil
47 | Portugal
//...

Preciso de uma query que retorne o seguinte resultado:
ID | ID_PAIS_ENVIO | NOME_PAIS_ENVIO | ID_PAIS_RECIBO | NOME_PAIS_RECIBO
1  | 23            | Brasil          | 47             | Portugal

É possível obter o resultado acima sem fazer INNER JOIN 2 vezes na mesma query?
Comment: Qual é o problema em fazer dois JOINs?
Answer: Sim, basta usar aliases diferentes para a tabela em cada JOIN:
SELECT
    Envios.*,
    PaisEnvio.nome_pais AS nome_pais_envio,
    PaisRecibo.nome_pais AS nome_pais_recibo
FROM Envios
    INNER JOIN Paises AS PaisEnvio
    ON PaisEnvio.id = Envio.id_pais_envio
    INNER JOIN Paises AS PaisRecibo
    ON PaisRecibo.id = Envio.id_pais_recibo

Comment: Obrigado, mas veja, era isso que eu queria evitar, no fim do post pergunto se é possível sem fazer INNER JOIN 2 vezes, uma única consulta que retorna 2 linhas em apenas 1.
Comment: Ops, não percebi essa parte. Até é possível fazer um único JOIN, mas nesse caso você não conseguirá distinguir os países de envio e recebimento pelo próprio SQL. E terá duas linhas para cada envio no resultado.
Comment: A nível de performance, haveria algum impacto considerável?
Comment: Acredito que não, Filipe. Mas só testando pra dizer com certeza.
Answer: Use a seguinte query:
SELECT PAIS_ENVIO.NOME_PAIS, PAIS_RECIBO.NOME_PAIS
FROM A
INNER JOIN B PAIS_ENVIO on A.ID_PAIS_ENVIO = B.ID
INNER JOIN B PAIS_RECIBO on A.ID_PAIS_RECIBO = B.ID

Comment: Entendi. Bati o olho no título e entendi que vc precisava do INNER JOIN. Você pode tentar com subqueries.
Comment: Obrigado, mas veja, era isso que eu queria evitar, no fim do post pergunto se é possível sem fazer INNER JOIN 2 vezes.
Answer: set @entrada = 23;
set @saida = 47;

SELECT
    e.id_pais as entrada, s.id_pais as saida
FROM
    tbl_pais s,
    tbl_pais e
WHERE (s.id_pais = @saida AND e.id_pais = @entrada);

Não saberia outra forma de fazer sem algum tipo de join... Valeuu! :D
Answer: Com Sub Query ou Sub Select:
SELECT
   TabelaA.ID,
   TabelaA.ID_PAIS_ENVIO,
   (SELECT TabelaB.NOME_PAIS FROM TabelaB WHERE TabelaB.ID=TabelaA.ID_PAIS_ENVIO) AS NOME_PAIS_ENVIO,
   TabelaA.ID_PAIS_RECIBO,
   (SELECT TabelaB.NOME_PAIS FROM TabelaB WHERE TabelaB.ID=TabelaA.ID_PAIS_RECIBO) AS NOME_PAIS_RECIBO
FROM TabelaA

Obs: A pergunta é sem JOIN, mas, é recomendável o uso do JOIN por ser mais rápido, já o Sub-Select pode degradar consideravelmente a sua pesquisa.
Referencia:

Trabalhando com Subquery's
Answer: Respondendo a pergunta: Não. Você tem duas chaves diferentes e precisa de um JOIN para cada chave pra buscar a descrição da segunda tabela.
Qualquer solução diferente disso irá alterar o resultado da sua consulta, ou então solucionar fora do banco de dados (pela aplicação, gerando uma coleção de países e atribuindo a descrição).
Answer: Tem como fazer sem join ligando as tabelas no WHERE .
   SELECT a.id
     , a.id_pais_envio
     , b_e.nome_pais as nome_pais_envio
     , a.id_pais_recibo
     , b_r.nome_pais as nome_pais_recibo

  FROM tabelaA as a
     , tabelaB as b_e
     , tabelaB as b_r

 WHERE a.id_pais_envio = b_e.id
   AND a.id_pais_recibo = b_r.id


In what cases is it a better choice to return a response than to edit the HttpServletResponse?

Question: I've currently found out three ways to write responses for endpoints in a @RestController annotated class:
@RestController
public class StringController {

    @GetMapping("/1")
    public ResponseEntity<List<String>> getString1(){
        return new ResponseEntity<>( new ArrayList<>(
                Arrays.asList("A", "B", "C", "D")
        ), HttpStatus.BAD_REQUEST );
    }

    @GetMapping("/2")
    @ResponseStatus(HttpStatus.ACCEPTED)
    public List<String> getString2(){
        return new ArrayList<>(
                Arrays.asList( "E", "F", "G", "H" ) );
    }

    @GetMapping("/3")
    public void getString3(HttpServletResponse response ) throws IOException {
        ObjectMapper mapper = new ObjectMapper(  );
        response.setContentType( "application/json" );
        response.setStatus( 321 );
        List<String> list = new ArrayList<>( Arrays.asList( "E", "F", "G", "H" ) );
        mapper.writeValue( response.getOutputStream(), list );
    }

}

For methods 1 and 2, they're simply returning a response. I wouldn't really ask about between the two since I think @ResponseStatus is simply just a newer addition.
But as for method 3, it does not return a response, rather it edits the very response itself via HttpServletResponse. In what cases is it a better choice to create and return a new response than to edit the HttpServletResponse?
P.S. I know my status codes are all wrong, I was simply experimenting with them.
Comment: Something like method 3 would make sense when your response is too large to fit in memory and you want to stream it rather than store and forward.
Answer: Those are Spring specific functionality. If you're not using Spring, you have to use HttpServletResponse directly (assuming you're not using another framework).
As you see, the code looks more readable when you can see the status code and response type directly from the method instead of having to look inside.
So the rule of thumb is, don't use HttpServletResponse directly unless you need to, and you shouldn't need to do it very often. There's nothing bad about it, but it's cleaner to code with Spring's annotations.


Para uma busca sem resultados, a resposta HTTP deve ser 404, 204 ou 200 com o corpo vazio?

Question: Quando buscamos algum recurso no servidor com algum tipo de filtro, por exemplo, uma busca de usuários, é retornado uma lista com os dados (em JSON, XML ou até já formatado em HTML)
Porém algumas pesquisas podem resultar em nenhum dado, então uma requisição para /users/name/guilherme, sendo que existe essa rota, porém não existe nenhum usuário com o nome guilherme, o mais semânticamente correto é retornar o código 404 ou um array vazio ([])?
O mesmo se aplica quando buscamos algo específico, como buscar o usuário (singular) de id N (/users/7)? No caso em vez de retornar um [] retornaria null
Pergunta para recompensa:
O status 204 No Content, segundo a rfc 7231:

O código de status 204 (Sem conteúdo) indica que o servidor resolveu com sucesso o pedido e que não há conteúdo para enviar no corpo da resposta

Isso pode ser interpretado como uma consulta que não retorna nenhum registro. Mas também como uma requisição que retorna um 204 sempre deverá retornar esse mesmo status, independentemente se houve alterações nos dados, se houver uma mudança no retorno, deve-se entender como uma mudança na API (a não ser que retorne algum status de erro)
Então qual cenário seria correto o uso? Ambos?
Comment: Eu sempre utilizo o 404 NotFound, e muitos projetos no github também utilizam o 404, gosto de seguir esse padrão porque é auto explicativo e amplamente utilizado. Segue um artigo bacana sobre o assunto:
https://www.infoq.com/articles/webber-rest-workflow
Comment: Retorne o *status 200* com um array vazio. O *status 404* deve ser utilizado para quando um recurso não for encontrado (por exemplo, a URL não existir). No teu exemplo, a URL existe, o que não existe são os dados.
Comment: Segundo a *W3*, no protocolo de definições de *status code*: `404 = O servidor encontrou nada que corresponda ao URI de solicitação.`
Comment: Isso é totalmente relativo e baseado na interpretação e definição de quem constrói a API, por exemplo, eu acredito que 404 não deva ser utilizado, visto que serve para dizer se você conseguiu realizar a chamada ao serviço, e que ele não está fora do ar. Eu utilizaria o 204 (No-Content) visto que o servidor está no ar, realizou a pesquisa, e não encontrou nenhum conteúdo.
Comment: Utilizando o exemplo dos links do  @Samuel, 404 pra mim seria "ei barista me veja um drink" e o barista não está no bar no momento, e 204 seria "ei barista me veja o drink X" e ele responde "infelizmente não tenho o X"
Comment: @ngueno Discordo do uso do *204*. Esse código significa que o servidor fez tudo certo, mas *não precisa* retornar um dado. Um bom exemplo disso é em requisições do tipo *DELETE*, que você não precisa retornar os dados deletados.
Comment: @ngueno considerando as diferentes respostas nos comentários eu diria que é uma pergunta pertinemte, e não acho "relativo e baseado na interpretação e definição de quem constrói a API", porque, apesar da pessoa poder criar uma API que deleta dados com o verbo GET, é semânticamente incorreto, assim como existe um status uma forma melhor de responder a requisição nesses cenários
Comment: @Guilherme, levo em consideração o [Modelo de Maturidade de Richardson](https://martinfowler.com/articles/richardsonMaturityModel.html), a representação GET pode ser correta para o DELETE? Sim, ou não também! Se você modela sua API considerando o último nivel de maturidade está errado, se modela em outro nível a representação GET atende, é diferente fazer uma requisição DELETE para `users/1` e fazer um GET para `deleteUser/1`. Ambos estão corretos? Sim! Ambos atendem? Sim! Como disse é relativo
Comment: O retorno de um DELETE pode ser também 200 porque a entidade foi deletada, pode ser 204 por não haver retorno, cabe a interpretação de quem criou a API de documentar qual código está usando
Answer: Primeiro, preciso dizer que uma das URLs parece estar mal construída.
Pelo texto entende-se que a URL /users/name/guilherme retornaria todos os usuários que possui name igual a guilherme. Se é uma busca, não é um recurso do servidor, então não pode ser mapeado para uma "uniform resource locator". Buscas são feitas por query strings, então a URL deveria ser algo como:
/users/?name=guilherme

Neste caso, o caminho que será acessado no servidor¹, definido pelo path da URL, será /users, portanto o recurso será devidamente localizado, será efetuada a busca e, neste caso, não serão encontrados resultados. Esta situação configura uma requisição tratada com sucesso pelo servidor, portanto uma resposta "200 OK"; como não foram encontrados resultados, provavelmente o corpo da resposta será uma lista vazia ou estrutura análoga:
HTTP/1.1 200 OK
Content-Type: application/json

{
    "total": 0,
    "users": [],
    "query": {
        "name": "guilherme"
    }
}

Perceba que retornar uma lista vazia não significa responder com o corpo vazio. Isso dificilmente será recomendado. Explícito é sempre melhor que implícito. Por exemplo, na resposta acima não só foi retornado "users": [] indicando a ausência de resultados como foi retornado "total": 0 e a busca efetuada. Quem for tratar a resposta não terá dúvidas que a busca foi feita com sucesso e não obteve resultado. Não ter dúvidas é sinal de ausência de ambiguidade e isso sempre é interessante.
Já para a URL /users/7 a situação muda. Você não está efetuando uma busca, mas sim acessando um recurso específico do servidor. Neste caso, não existindo um usuário com identificador igual a 7, o retorno deverá ser "404 Not Found".
Assim como comentado em Pular a rota devido a parametro mal formatado é um erro de sintaxe? a resposta 404 indicará que no momento da requisição o recurso não existe, mas no futuro, por ventura, poderá existir. Isso é importante para o cliente que está fazendo a requisição pois ele saberá que poderá futuramente fazer a mesma requisição e obter uma resposta diferente.
Mas perceba, isso não é uma regra, apenas algo que se aproxima ao idiomático do HTTP, pois busca eliminar ao máximo a redundância. Dependendo dos requisitos a redundância pode ser benéfica quando é de interesse praticar a segurança por obscurantismo². Em alguns casos gerar a redundância nas respostas HTTP é proposital para ocultar as regras de implementação da API, porém este é um contexto difícil de se trabalhar, pois aumentando a redundância você intrinsecamente aumenta o grau de dificuldade das integrações.
É comum, no entanto, vermos APIs com o retorno para /users/?name=guilherme sendo a resposta 404 da mesma forma:
HTTP/1.1 404 Not Found
Content-Type: application/json

{
    "total": 0,
    "users": [],
    "query": {
        "name": "guilherme"
    }
}

E é comum também vermos a resposta com o corpo indicando os detalhes da resposta, o que diminuiria a redundância nesse exemplo. Se você reduz a redundância pelo corpo, o código da resposta passa a ser supérfluo, então dependerá diretamente das regras de implementação do seu projeto, só busque ser coerente. Ao definir a resposta em um desses recursos, utilize-a para todos os recursos semelhantes. Uma resposta 404 para /users/?name=guilherme e uma resposta 200 para /posts/?category=sports deixaria sua API incoerente e prejudicaria ainda mais qualquer integração.
204 No Content
A resposta 204 não deve ser usada para indicar que a busca não teve resultados. Semanticamente falando - e é isso que importa quando analisamos a fundo qual resposta utilizar - quando fazemos uma busca esperamos como resultado uma lista; quando a busca não possui resultados, será esperado que recebemos uma lista vazia. O servidor recebeu a requisição, a entendeu e conseguiu interpretá-la e executá-la. Retornar a resposta 204 neste caso implicará que o cliente assuma por inferência que não houveram resultados, mas nunca lhe dará a segurança disso.
Seria o mesmo que você chegar a um feirante, pedir se ele possui tomates para vender e ele não te responder. Você pode entender que por não ter te respondido ele não possua os tomates, mas também pode ser que ele não entendeu que foi uma pergunta e, portanto, julgou que não havia necessidade de uma resposta. Gerar dúvidas no cliente não é semântico. Diferentes clientes podem entender de forma distinta e a confusão estará armada. Você pode até documentar muito bem sua API e deixar explícito que haverá o retorno 204 quando não encontrar resultado, mas nesse caso você acaba gerando uma dependência muito grande com sua própria documentação sem qualquer tipo de benefício.
Quando a resposta 204 No Content deve ser usada?
Quando a resposta para a requisição não demandar um corpo. A situação mais comum disso é em atualizações de informações. Imagine a situação: o usuário acessou seu perfil no site, entrou em editar perfil e, assim, está vendo todas as informações atuais no formulário. Ele decide então alterar alguns campos e salvar. Nesse momento o usuário já possui todas as informações atualizadas na tela, por que você iria transmitir essas mesmas informações do servidor para o cliente sendo que você já as possui no cliente? Isso seria adicionar uma carga desnecessária na comunicação. Para o cliente em si isso talvez nem faça tanta diferença; o tempo gasto a mais para transmitir o corpo possivelmente seria de alguns milissegundos (ou segundos em casos piores)³, mas nada que chegasse a afetar a usabilidade da API. Porém, para um servidor que precisa responder milhares de requisições por hora, essa informação a menos que ele não precisará enviar pode fazer muita diferença.
Imagine que você precisa solicitar ao estagiário que ele mude a posição da cafeteira no escritório. Hoje ela fica na cozinha, mas você quer que ela seja instalada na sua mesa. Você pode chegar pro estagiário e pedir "Saudações terráqueo, por favor, mude a posição da cafeteira. Traga ela da cozinha para a minha mesa". O estagiário, eficiente como sempre, prontamente faz a mudança e a fim de te informar que a tarefa foi executada ele chega até você e responde uma das opções abaixo:

Pronto!
Pronto, mudei a cafeteira de lugar. Agora ela está instalada na sua mesa.

Concorda que ambas as respostas são completas e não deixam dúvidas do que foi feito? Mesmo que a resposta seja apenas "Pronto!", você já sabia que a cafeteira estava na cozinha e que seria instalada na sua mesa, então somente uma confirmação já seria suficiente. De forma análoga, a segunda resposta também te dá a confirmação que a tarefa foi realizada, com a diferença que ela também afirma aquilo que você já sabia.
Na resposta HTTP é a mesma coisa. Se você já possui conhecimento atual e você o modifica, você tem todas as ferramentas necessárias para inferir qual será o novo estado do recurso, assim apenas uma confirmação de sucesso bastará.
Inclusive, a resposta 204 No Content costuma possuir o cabeçalho ETag, que é um valor único responsável por identificar o estado do recurso. Assim, junto com as informações do recurso você possuirá a ETag atual do mesmo, que você deverá enviar junto com a requisição de atualização através do cabeçalho If-Match. Se o recurso no servidor estiver exatamente no mesmo estado que o cliente possuía, a atualização é autorizada e o novo ETag é devolvido na resposta. Caso o recurso esteja em um estado diferente do que o cliente possui, a atualização das informações deverá ser recusada informando ao cliente que a versão do recurso que ele possui está desatualizada. Isso evita o problema de "lost update".

Como evitar conflito de dados entre duas requisições PUT no HTTP?

Vamos supor a situação: tenho as informações de um usuário (nome e e-mail) e desejo atualizá-las. Eu acesso o recurso:
>> GET /user/1 HTTP/1.1
>> Host: localhost
>> ...

Obtendo a resposta:
<< HTTP/1.1 200 OK
<< Content-Type: application/json
<< ETag: 1000001
<< ...
<<
<< {"name": "Anderson Carlos Woss", "email": "anderson@localhost"}

Eu altero o e-mail e mando salvar, fazendo, assim, a requisição de atualização:
>> PUT /user/1 HTTP/1.1
>> Host: localhost
>> Content-Type: application/json
>> If-Match: 1000001
>> ...
>>
>> {"email": "anderson@woss"}

Neste caso, eu tenho o nome e e-mail originais e sei o que foi mudado, então sei que o novo estado será {"nome": "Anderson Carlos Woss", "email": "anderson@woss"}, a resposta à requisição não precisa me dizer isso, seria redundante. Então uma possível resposta utilizando a HTTP 204 seria:
<< HTTP/1.1 204 No Content
<< ETag: 1000002
<< ...

Veja que a resposta informará o novo valor de ETag indicando o novo identificador do recurso. Se for feita outra requisição de atualização informando o valor antigo, a atualização não ocorre e a resposta 412 Precondition Failed será retornada:
>> PUT /user/1 HTTP/1.1
>> Host: localhost
>> Content-Type: application/json
>> If-Match: 1000001
>> ...
>>
>> {"email": "john@due"}

A resposta seria:
<< HTTP/1.1 412 Precondition Failed
<< Content-Type: application/json
<< ETag: 1000002
<< ...
<<
<< {"error": "As informações estão desatualizadas. Atualize-as antes de fazer uma edição"}

Isso evitaria que a primeira alteração fosse sobrescrita pela segunda (problema de "lost update").

(1): Lembrando que a URL é um valor opaco e não necessariamente reflete a implementação do servidor, então dizer que o caminho acessado no servidor a partir do path da URL nem sempre é válido.
(2): Segurança por obscurantismo: Como se chama a técnica de esconder componentes para assegurar informações?
(3): Desconsiderando casos especiais em que o corpo da resposta poderia ser gigantesco.
Comment: Uma verdadeira aula, muito bom.
Answer: Acredito que o Status 404 (Not Found) seja mais adequado a recursos que de fato, não forem encontrados, mas esperava-se que fossem. Caso seja uma busca, do tipo pode ou não ser encontrado, eu usaria o 204, pois sinaliza que a busca não retornou resultados. Um array vazio: [] seria interessante, por ser autoexplicativo.
Answer: Eu utilizaria 404 também, pois de acordo com a rfc rfc7231 que define os códigos HTTP este código diz respeito ao um recurso que foi procurado no servidor e não foi encontrado, então não se limita somente a url inválidas.
rfc:
 The 404 (Not Found) status code indicates that the origin server did
   not find a current representation for the target resource or is not
   willing to disclose that one exists.

Já o código 200 informa que a solicitação foi atendida com sucesso, ou seja, deve seu usado quando encontrou o recurso que estava sendo procurado.
rfc
The 200 (OK) status code indicates that the request has succeeded.

Fontes:
Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content

https://www.rfc-editor.org/rfc/rfc7231
Comment: Como bem disse, o *status 200* informa que ocorreu tudo OK. E, por mais que não tenha trazido resultados, a requisição foi *OK*, ela funcionou como o esperado. Na [rfc7231](https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html), temos: *404 Not Found = The server has not found anything matching the Request-URI.*
Comment: Oi @ValdeirPsr, o link que você consultou acabou te induzindo ao erro, creio que o google provavelmente, este link que você colocou na verdade se refere a RFC2616, ela foi suspensa e substituída pelas RFCs (7230-7237), consulta o link https://www.w3.org/Protocols/rfc2616/rfc2616.html ai tem essa informação. Obrigado
Comment: Eu só utilizaria 404 se `name` fosse um identificador único, se for uma pesquisa deveria ser 200, pois tanto o endpoint quanto a pesquisa estão corretos, apenas não tem resultado.
Não acho semântico um retorno de erro para uma requisição que foi feita corretamente.
Answer: Confesso que a ideia do 200 OK me deixa com um pouco de duvidas, eu imagino a seguinte situação:
Você chega em uma loja pra pedir alguma coisa para o balconista, ai ele volta de mãos vazias e responde "Aqui está". esse seria o caso do 200 com body contendo 0 ocorrencias
HTTP/1.1 200 OK
Content-Type: application/json

{
    "total": 0,
    "users": [],
    "query": {
        "name": "guilherme"
    }
}

Eu acredito que ele deveria voltar ao balcão e te dizer "Desculpe, não encontrei oque você pediu", esse seria então o caso do 404 Not Found.
HTTP/1.1 404 Not Found

No caso do 404, colocar um body com 0 ocorrências me parece redundante, uma vez que algo que não foi encontrado não poderia ter um numero de resultados maior que 0
Comment: No seu exemplo o 404 está mais para "cheguei em uma loja e não encontrei um vendedor, então não consegui fazer meu pedido"


LibGDX textures do not appear after reloading

Question: I'm working with an app which is using OpenGL ES through libgdx. There are undo/redo functions in my app. When the user clicks one of the buttons, the image in the app (which was created via OpenGL ES) is disposed. After that, in the OpenGL thread is loading a new image. Redo is doing the same function. Sometimes when I click undo/redo, the image in ImageView becomes black, sometimes text in TextView becomes black. I think the problem is in dispose() method, but I don't understand why the image and text becomes black. Maybe someone can help me.
Code of dispose:
if(obj.getTexture() != null){
    obj.getTexture().dispose();
}

Code of load texture:
texture = new Texture(Gdx.files.absolute(fileName));

Undo code:
int indexUndo = undoObjects.size() - 1;
SpriteState ss = undoObjects.get(indexUndo);
if(object instanceof PhotoSprite){
     PhotoSpriteState pState = (PhotoSpriteState)ss;
     if(pState.getTypeChanging() == TypeSpriteChanging.ChangeTexture){
        object.saveRedoState(TypeSpriteChanging.ChangeTexture);
        PhotoSprite photoObj = (PhotoSprite)object;
        photoObj.setFileName( pState.getFilename() );
        photoObj.setGeometry( pState.getGeometry() );
        photoObj.setTexture(null);
     }
}

Redo code:
int indexRedo = redoObjects.size() - 1;
SpriteState ss = redoObjects.get(indexRedo);
if(object instanceof PhotoSprite){
     PhotoSpriteState pState = (PhotoSpriteState)ss;
     if(pState.getTypeChanging() == TypeSpriteChanging.ChangeTexture){
        object.saveRedoState(TypeSpriteChanging.ChangeTexture);
        PhotoSprite photoObj = (PhotoSprite)object;
        photoObj.setFileName( pState.getFilename() );
        photoObj.setGeometry( pState.getGeometry() );
        photoObj.setTexture(null);
     }
}

Code laod texture:
public void render(PhotoSprite psObject, int zIndex) {
    ShaderProgram shader = psObject.getShader();
    if(!shader.isCompiled()){
        Log.i("ERROR", "SHERROR " + shader.getLog());
    }
    if(psObject.getTexture() == null){
        psObject.setTexture(loadTexture(FILE_PATH + psObject.getFileName()));
    }
    psObject.getTexture().bind(0);

    shader.begin();
    shader.setUniformMatrix("u_worldView", psObject.getMatrix(zIndex));
    shader.setUniformi("u_texture", 0);
    // psObject.getMesh()
    MeshUtil.MESH.render(shader, GL20.GL_TRIANGLES);
    shader.end();
}

Initial state:

After using undo/redo
Comment: Show us how you handle the different lifecycle callbacks... show, hide, pause, resume...
Comment: We need to see *way* more code than this. The bug is somewhere in your undo/redo logic.
Comment: Sorry, I added code undo/redo. About show, hide, pause, resume, show and hide wasn't realized. In pause and resume, I don't have work with OpenGL, and this methods don't call when user click undo/redo
Comment: You will need to show the part of your code that disposes and the part that reloads the texture. There is not enough context here to see what is happening to your texture. Also, are you using actual Android ImageViews, or do you mean LibGDX sprites?
Comment: I added code which texture is loading. In ImageView, texture is hiding. With opengl texture is all ok.
Answer: I find answer. Problem was in destroy texture out of GLThread. Hope it will help someone.


Cortar string por um caractere específico e com índice no BigQuery

Question: Tenho alguns dados que preciso fazer um corte apartir da segunda ocorrência do pipe "|", uso Substring_Index no MySQL, porém no BigQuery não consegui achar uma solução que contempla o índice da ocorrência.
O que eu tenho:

2463|134|87
457|4|0
3|113544|9887132

O retorno que preciso:

2463|134
457|4
3|113544

O que Tentei:

SELECT STRPOS ('3|113544|9887132','|') AS Exemplo1
SELECT SPLIT('3|113544|9887132','|') [offset(2)] AS Exemplo2
Answer: Consegui essa solução que me ajudou a chegar no resultado.
select concat(split('444|555|5','|') [offset(0)],"|",split('444|555|5','|') [offset(1)])

Resultado:
444|555


Limiting distribution of first order statistic x

Question:
Good evening everyone, I am attempting a problem on limiting distributions. The problem is as seen in the picture above. Could you please help me with part c) of this problem. The answer as per the solutions at the back is
$$ 0 \,\text{for}\, x\le 1 \\
1-\frac{1}{y}\, \text{for} \,x\gt1$$
I dont quite understand why, I thought it would be $1 \, \text{for} \, x\gt 1$.
Thank you!
Comment: would you like to include your working so that others don't have to repeat your work?
Comment: If this is a homework problem, it would need a `self-study` tag, or it risks being deleted due to [site policy](https://stats.stackexchange.com/tags/self-study/info).
Comment: Please have a look here: https://stats.meta.stackexchange.com/a/3176/. And type out the exam question in the picture.
Comment: This is not homework, I am just practicing!
Answer: From the specified distribution, for all $x \geqslant 1$ you have:
$$\begin{equation} \begin{aligned}
F_{X_{1:n}}(x) \equiv \mathbb{P}(X_{1:n} \leqslant x)
&= 1 - \mathbb{P}(X_{1:n} > x) \\[6pt]
&= 1 - \mathbb{P}(\min \{ X_1,...,X_n \} > x) \\[6pt]
&= 1 - \prod_{i=1}^n \mathbb{P}(X_i > x) \\[6pt]
&= 1 - \prod_{i=1}^n x^{-1} \\[6pt]
&= 1 - x^{-n}.
\end{aligned} \end{equation}$$
Hence, for all $x \geqslant 1$ you have:
$$\begin{equation} \begin{aligned}
F_{X_{1:n}^n}(x) \equiv \mathbb{P}(X_{1:n}^n \leqslant x)
&= \mathbb{P}(X_{1:n} \leqslant x^{1/n}) \\[6pt]
&= 1 - (x^{1/n})^{-n} \\[6pt]
&= 1 - x^{-1}.
\end{aligned} \end{equation}$$
This gives us back the original distribution of $X_i$.  This distribution does not depend on $n$ so its limiting distribution is just the original distribution.


Prove that $a_j=b_j$ for the $2$ sequences $a$ and $b$

Question:
Let $n$ is a natural number and $(n,6)=1$. Given $2$ sequences $a$ and
  $b$ such that $a_1>a_2>\ldots a_n$ and $b_1>b_2>\ldots b_n$. And for
  all $1 \leq j < k <l \leq n$, it is given that
  $a_j+a_k+a_l=b_j+b_k+b_l$. Prove that $a_j=b_j$ for all $j$.

I don't know if this question is easy or not, but I did prove it without using much given information. Please tell the mistake in it.

First substitute $k=2$ and $j=1$ and consider $2$ $l's$ $l_1$ and
  $l_2$ and then subtract the $2$ equations to get
  $a_{l_{2}}-b_{l_{2}}=a_{l_{1}}-b_{l_{1}}$. Now substitute $1,2,3$ and
  $2,3,4$ and then subtract to get a similar equation. Now substitute
  $1,3,4$ to get $a_1+a_3+a_4=b_1+b_3+b_4$. From this equation and the
  previous one, it implies $a_j-b_j=0$ and the result follows.

I did not use half of the given information and I know I am going wrong somewhere, but where.
Please help. Thanks.
Comment: I think $(n,6)=1$ is just so that $n$ is not $2,3$ or $4$.
Comment: The sequences are formed by **real** numbers, right?
Answer: I don't see anything wrong with your approach.  If I may paraphrase it (and replace your condition on n with $n≥5$):
For each $l\in (3,4, 5, ..., n)$ we have:
$$a_1+a_2+a_l = b_1+b_2+b_l$$
Whence $(a_l- b_l)$ is independent of l, at least for $l≥3$. Let $\Delta$ denote the common difference $(a_l- b_l)$ for $l≥3$.  Now, so long as n is at least 5 we can look at the sum $a_3+a_4+a_5$ but on the one hand that equals $b_3+b_4+b_5$ by assumption and on the other it equals $b_3+b_4+b_5+3\Delta$.  Hence $\Delta=0$.  Thus all the terms coincide except (possibly) the first two.  But of course we have $a_1+a_3+a_4 = b_1+b_3+b_4 = b_1+a_3+a_4$ which tells us that $a_1=b_1$ and similarly for the second term.
Just to say:  You have specified a very large number of equations.  It is not surprising that they determine the system nor that you could ignore a bunch of them.


Get the distribution of a dataset

Question: this is a conceptual question. In data mining, the problem often arises that scientists/data engineers are using an expert guess for the underlying distribution of their data. Often their assumption is based on previous results of academic research and does not really reflect the real distribution of the dataset.
However, I would like to get the real distribution of my dataset? Any recommendations where to start? Does this even make sense to get the real distribution? Can thisv task be automated?
I appreciate your answer!
btw I am mostly using R as my data analysis tool
Comment: This is a frequent question here. Generally speaking, you *can't*. Let's say your data look exponential. That doesn't mean they *are* exponential. You can never prove it by looking at the data. As you look at larger and larger samples sizes, you may be able to exclude many alternatives, but there will always be an infinite number of alternatives that will fit the data about as well *even if the data are truly exponential*.
Comment: E.g. see the first few paragraphs [here](http://stats.stackexchange.com/questions/58220/what-distribution-does-my-data-follow/58241#58241), and some of the comments [here](http://stats.stackexchange.com/questions/102630/basic-question-importance-of-modelling-a-distribution-to-data/102634#102634)
Comment: You can sometimes find out when data is inconsistent with some distribution, but you can't say that it *is* from some distribution.
Answer: There is no "real distribution" of your data. There are models of the distribution of your data, and some are more useful than others. There are several ways to compare distributions.

Fit distributional models to data using maximum likelihood. Compare the models based on their information criterion scores (e.g., AIC, BIC, DIC, WAIC).
Fit distribution models to data using Bayesian methods. Once you have recovered the Bayesian posterior distribution, choose a series of test statistics, which are not parameters or sufficient statistics in the distribution model. Compute the posterior probability that the test statistic is at least as extreme as what is found in the data. The distribution model with the fewest "significant" Bayesian p-values is the best model.

Of course, there is also the possibility of including all of the distribution models within some super Bayesian estimation model, which includes parameters dictating the relative weights of each of these models. That would be the best thing to do, but is often completely unfeasible.
Answer: The topic you're looking for is 'density estimation'.
The first thing to note is that you will probably not find that a mathematically familiar distribution fits your data (unless you work on radioactive decay, when the Poisson really is what you want...)
The second thing to note is that one may, subject to various theorem-specific technical conditions, approximate any distribution arbitrarily well given enough data with a mixture of more familiar distributions e.g. from the exponential family.
For example, a simple approach to estimating the distribution of continuous data might be a finite mixture of Gaussians. Maximum Likelihood estimates for parameters are available using an EM algorithm. (EM algorithms are easiest to implement for exponential family).  Naturally other methods are possible. McLachlan et al. 2000 is a good place to start reading.  The R package flexmix will help you fit these models.
The basic issues that arise in this exercise involve not having enough samples in all parts of the data space - the curse of dimensionality - and consequently the question of how to trade-off bias and variance in one's density model.
Density estimation is in this sense inherently harder than building conditional models such as regression and classification.  For example, in the latter problem one needs only to characterise the area of the data distributon where the class decision boundaries lie rather than all of it.
Finally, note that for many problems the distribution of your data is not relevant.  A standard motivation for regression modeling is to observe that some P(X, Y) can be decomposed into P(X) the density of the input data and P(Y | X) the conditional density of the target data.  If it is further assumed that the parameters governing these two distributions are a priori independent (as they will be when distinct mechanisms assign X and generate Y from it) then the density P(X) can be ignored when trying to learn about P(Y | X), which is convenient because the latter is typically a much lower dimensional distribution to characterise.

Get the distribution of a dataset


creating a redux like stores using custom hooks

Question: Here I implemented redux like store using custom hooks. everything goes well and code executed correctly but problem is that in reducer under switch statement "TOGGLE" there I return a updated state which is finally stored in globalstate but if I returned empty object {} instead of {products: updated} still globalstate updating correctly with a change that has been done in reducer...since i am not passing globalstate reference then how it is updated correctly
and what listeners exactly do in dispatch method in code
import MarkFavMulti from "./MarkFavMulti";
import classes from "./MarkFav.module.css";
import useStore from "../HookStore/Store";
import {reducer2} from "../SampleReducer";

const MarkFav = props => {
    const [outfit, dispatch] = useStore(reducer2);

    const onClicked = (id) => {
        dispatch({type: "TOGGLE", id: id});
    }

    const element = outfit.products.map((item) => {
        return <MarkFavMulti cloth={item.name}
            favorite={item.favorite}
            price={item.price}
            key={item.id}
            clicked={onClicked.bind(this, item.id)} />
    });

    return (

            {element}

    );
};

export default MarkFav;

import {useState, useEffect} from "react";

let globalState = {};
let listeners = [];

const useStore = (reducer) => {
    const setState = useState(globalState)[1];

    const dispatch = (action) => {
        let curr = Object.assign({},globalState);

        const newState = reducer({...curr}, action)
        globalState = {...globalState,...newState};

        for(let listener of listeners) {
            listener(globalState);
        }
    };

    useEffect(()=>{
        listeners.push(setState);
        return () => {
            listeners.filter(item => item !==setState);
        }
    },[setState]);

    return [globalState, dispatch];
};

export const initStore = (initialState) => {
    if(initialState) {
        globalState = {...globalState, ...initialState};
    }
}

export default useStore;

let initialState = {
    products: [
        { id: 1, name: "shirt", price: "$12", favorite: false },
        { id: 2, name: "jeans", price: "$42", favorite: false },
        { id: 3, name: "coat", price: "$55", favorite: false },
        { id: 4, name: "shoes", price: "$8", favorite: false },

    ]
}

const configureStore = () => {
    initStore(initialState);
};

export default configureStore;

export const reducer2 = (state=initialState, action) => {
    switch (action.type) {
        case "TOGGLE":
            let update = {...state};
            let updated = [...update.products];

            updated = updated.map(item => {
                if(item.id === action.id) {
                    item.favorite = !item.favorite;
                    return item;
                }

                return item;
            });
            return {products: updated};
//if we return {} ...it will updated correctly in globalstate

        default:
            throw new Error("not reachable");
    }
}

Answer: The behavior that you are describing is due to this object assignment right here:
item.favorite = !item.favorite;

Here you are directly mutating the properties of the item object.  You probably thought that it would be fine since you are using a copy of the products array.
let update = {...state};
let updated = [...update.products];

What actually happens is that updated is a "shallow copy" of the original array.  The array itself is a new array, but the items in that array are the same items as in the state.  You can read more about that here.
You need to return a new item object instead of mutating it. Here's a concise way to write it using the ternary operator.
   case "TOGGLE":
      return {
        ...state, // not actually necessary since products is the only property
        products: state.products.map((item) =>
          item.id === action.id
            ? {
                ...item,
                favorite: !item.favorite
              }
            : item
        )
      };


Shooting at the wrong hoop beyond the three point line

Question: I recently saw a game where a player tried to save the ball from going out of bounds, and in the process scored in the wrong hoop. This player was saving the ball from beyond the three-point line.
Should this have been awarded with three points? The score in the game only increased by 2, but I'm not sure if there is a rule which accounts for this circumstance.
Answer: The FIBA rules have defined how points are awarded for an own goal.

Article 16.2.2
If a player accidentally scores a field goal in his team's basket, the goal counts 2 points and shall be recorded as having been scored by the captain of the opposing team on the playing court.

NCAA Men's Basketball rules are a bit more detailed in this regard (emphasis mine).

Rule 5 Article 4
When a player scores a field goal in the opponent's basket, it shall count
two points for the opponent regardless of the location on the playing
court from where it was released. Such a field goal shall not be credited
to a player in the scorebook but shall be indicated with a footnote.

So it looks like an own goal is worth two points only, irrespective of the location from where the ball was released.
PS: NCAA uses the term 'opponent's basket' while FIBA uses the term 'his team's basket'. This is down to the definition of own and opponent's basket which is contradictory for both. For FIBA, refer Rule 1 Article 1.2 and for NCAA, refer Rule 1 Section 1 Article 2.
Comment: I have used the PDF link in my answer currently, but if someone could point me towards the website version of the rules, I would be grateful.


Cost function for cross entropy

Question: I'm currently studying from Hinton's neural network course and he just introduced the cost function used with the softmax output function:
\begin{align}
C   &= -\sum_{j}t_j\log y_j  \\[5pt]
y_j &= \frac{e^{z_j}}{\sum_{k \in group}e^{z_k}}
\end{align}
In the slides, he says that "C has a very big gradient when the target value is 1 and the output is almost zero". This makes sense as we will want to minimize $C$ since the target and output are not the same. However, what if the target value is 0? Here, wouldn't $C$ always be zero so it doesn't matter what weights you have in $y_j$?
Answer: The equation you wrote for $C$ looks like the cross entropy loss for a single data point in a multiclass classification problem. In that case, $y_j$ would be the classifier's estimated probability that the class is $j$ given the input. $t_j$ would be an 'indicator variable' that takes the value $1$ if the true class of the data point is $j$ and $0$ otherwise. $t_j$ must be $1$ for exactly one value of $j$ (because the data point has a single, definite class). Since the sum is taken over all $j$ (i.e. all classes), there would be no problems with everything being zero.

Cost function for cross entropy


How to properly call the vulkan.hpp constructor for command buffers?

Question: I have been changing some vulkan code to use the vulkan.hpp structures and methods.
Since I like RAIIs, I am using the Unique wrappers, to not have to explicitely handle resource management.
So far I have been able to create 2 versions of each wrapping method I am making, one version creates a non unique object, and another method calls the first and then initializes a unique handle from the first. Example:
vector<vk::UniqueFramebuffer> CreateFramebuffersUnique(vk::SurfaceKHR surface,
    vk::PhysicalDevice phys_device, vk::Device device, vk::RenderPass render_pass,
    vector<vk::ImageView> image_views)
{
    auto framebuffers =
        CreateFramebuffers(surface, phys_device, device, render_pass, image_views);
    vector<vk::UniqueFramebuffer> u_framebuffers;
    for(auto &fb : framebuffers)
        u_framebuffers.push_back(vk::UniqueFramebuffer(fb, device));

    return u_framebuffers;
}

The above method creates an array of framebuffers and then re-initializes each framebuffer as a unique framebuffer prior of returning it.
I tried doing the same with command buffers:
vector<vk::UniqueCommandBuffer> CreateCommandBuffersUnique(vk::SurfaceKHR &surface,
    vk::PhysicalDevice &phys_device, vk::Device &device, vk::RenderPass &render_pass,
    vk::Pipeline &graphics_pipeline, vk::CommandPool &command_pool,
    vector<vk::Framebuffer> &framebuffers)
{
    auto command_buffers = CreateCommandBuffers(surface, phys_device, device, render_pass,
        graphics_pipeline, command_pool, framebuffers);

    vector<vk::UniqueCommandBuffer> u_command_buffers;
    for(auto &cb : command_buffers)
        u_command_buffers.push_back(vk::UniqueCommandBuffer(cb, device));

    return u_command_buffers;
}

The above technically works, but upon program termination, the validation layers complain about a mistake upon resorce de allocation:
validation layer: vkFreeCommandBuffers: required parameter commandPool specified as VK_NULL_HANDLE
UNASSIGNED-GeneralParameterError-RequiredParameter
4096
validation layer: Invalid CommandPool Object 0x0. The Vulkan spec states: commandPool must be a valid VkCommandPool handle (https://www.khronos.org/registry/vulkan/specs/1.1-extensions/html/vkspec.html#VUID-vkFreeCommandBuffers-commandPool-parameter)
VUID-vkFreeCommandBuffers-commandPool-parameter

This happens because the command pool field of the unique handle is not set properly:
(gdb) p t_command_buffers[0]
$6 = {<vk::PoolFree<vk::Device, vk::CommandPool, vk::DispatchLoaderStatic>> = {m_owner = {m_device = 0x555555ec14e0}, m_pool = {m_commandPool = 0x0},
    m_dispatch = 0x7fffffffe2f7}, m_value = {m_commandBuffer = 0x555555fe6390}}

I checked, and although there is a commandBuffer.getPool(), there is no setPool().
Any suggestions on properly setting the field?
Answer: You're not including the source or the signature of the CreateCommandBuffers function you're calling internally from CreateCommandBuffersUnique.  However, lets assume it returns a type of std::vector<vk::CommandBuffer>
It looks like you're then looping over those and wrapping them manually in vk::UniqueCommandBuffer instances.  However, you're passing the cb (assumed here to be a vk::CommandBuffer and a vk::Device to the UniqueCommandBuffer constructor here
u_command_buffers.push_back(vk::UniqueCommandBuffer(cb, device));

I'm actually not clear on how that even compiles, unless vk::Device has an operator ()().  Regardless, the second parameter to a vk::UniqueCommandBuffer should be a deleter object.
So it turns out that all of the UniqueXXX types derive from UniqueHandle<>, which in turn derives from UniqueHandleTraits<...> which is specialized for each type, in order to give it a default deleter type.  Most of the UniqueHandleTraits<...> specializations use an ObjectDestroy<...> template because most objects only need their creating Device to destroy them.  However, the UniqueHandleTraits<CommandBuffer> and UniqueHandleTraits<DescriptorSet> specializations use PoolFree<...> for their deleters.
Therefore, your use of
vk::UniqueCommandBuffer(cb, device));

implicitly turns into
vk::UniqueCommandBuffer(cb,
    vk::PoolFree<Device, CommandPool,Dispatch> { device, {}, {} }
)

That {} after device is where the pool is supposed to be specified.
See the corresponding code from vk::Device::allocateCommandBuffersUnique
PoolFree<Device,CommandPool,Dispatch> deleter( *this, allocateInfo.commandPool, d );
for ( size_t i=0 ; i<allocateInfo.commandBufferCount ; i++ )
{
  commandBuffers.push_back( UniqueCommandBuffer( buffer[i], deleter ) );
}

To fix your code you either need to use vk::Device::allocateCommandBuffersUnique or replicate it's behavior, specifically create a deleter object or lambda and pass it as the second parameter to the UniqueCommandBuffer ctor.
Based on my examination of the UniqueHandleTraits, it looks like you'd be able to fix your code just by changing this line:
    u_framebuffers.push_back(vk::UniqueFramebuffer(fb, device));

to
    u_framebuffers.push_back(vk::UniqueFramebuffer(fb, { device, command_pool }));

Comment: The above syntax not only compiles but works for almost all objects (I used it for swap chains, devices, framebuffers, commandpool... the only object it didn;t work on was CommandBuffers), I will append a temptative solution that fixes the problem, at least for now.

But you are correct about the types of the return values of each function
Comment: The other types you mention aren't allocated from a pool.  In fact if you look at vulkan.hpp and search for `UniqueHandleTraits` you'll see that most of vulkan types have specialized deleters based on `ObjectDestroy` but pool allocated resources like `CommandBuffer` and `DescriptorSet` use a deleter specializations based on `PoolFree`.  It looks like the reason your code even compiles is that `PoolFree` has default arguments for everyhing in it's ctor, so your code essentially calls `PoolFree instance(device, {}, {})`
Answer: Upon playing around I have found a solution that solves the issue, although I would prefer doing things more closely to the example I provided with framebuffers:
vector<vk::UniqueCommandBuffer> CreateCommandBuffersUnique(vk::SurfaceKHR &surface,
    vk::PhysicalDevice &phys_device, vk::Device &device, vk::RenderPass &render_pass,
    vk::Pipeline &graphics_pipeline, vk::CommandPool &command_pool,
    vector<vk::Framebuffer> &framebuffers)
{
    vk::CommandBufferAllocateInfo alloc_info(command_pool,
        vk::CommandBufferLevel::ePrimary, framebuffers.size());
    auto[result, u_command_buffers] = device.allocateCommandBuffersUnique(alloc_info);
    if (result != vk::Result::eSuccess)
        Log::RecordLogError("Failed to allocate command buffers!");

    auto command_buffers = CreateCommandBuffers(surface, phys_device, device, render_pass,
        graphics_pipeline, command_pool, framebuffers);

    for(auto &cb : command_buffers)
    {
        u_command_buffers[&cb - &command_buffers[0]].reset(cb);
    }

    return std::move(u_command_buffers);
}

Comment: Don't use `std::move` here.  see https://stackoverflow.com/questions/4986673/c11-rvalues-and-move-semantics-confusion-return-statement/4986802#4986802
Comment: How can I prevent the constructor from being called in that case? i.e how do I extend the life of the unique handle?


entity reference and exposed filter

Question: Hello I would like to ask about views and exposed reference filters. I have problem with displaying my custom view for exposed filter. Can you please help?

Its kind of complex so I will try to describe the structure. I made view which should be used as exposed filter and this view works fine, but cant push it to appear in other view as exposed filter.
All of my users have some cateogies as taxonomy terms. Also every content has this category so I want to show to user only the content which is in category assigned to specific user. There can be more than one category assigned to user. Content has also single value entity reference to manufacturer.
And now I need to have a page for every content type where and I want to display all content in users category with exposed filter of manufacturers. In this filter must not be all manufacturers but only which ,there is content created with this manufacturer and the content is assigned to same category as the user.

Little example for one content type:

user1 - category: dj, profi video
user2 - category: hifi
user3 - category: dj, hifi

content1 - manufacturer: m1; category: dj, hifi
content2 - manufacturer: m2; category:dj
content3 - manufacturer: m2; category: profi video
content4 - manufacturer: m3; category: hifi

In this point the exposed filter for user1 should display: m1, m2. Or for user2: m3. Or for user3: m1, m2, m3.
Answer: You need to set up relationships between the three content types in your View. Essentially, you have to set up relationship user->category, then relationship category->manufacturer, at which point the manufacturer fields will become available in the exposed filters section. There is a great tutorial here how to do this.
Comment: Thank you @longboardnode for reply. The problem was that I still wanted to add filter `Content: "name_of_field" (reference filter)` but only reference field where I could choose my view was `User: Name (reference filed)`. But the problem is that it shows only one option. Even if in the reference display view is everything ok. What is the problem now? How can I get different filter field where I can choose my reference display view?
Comment: I'm sorry but I'm not quite clear from your comment how you have things set up right now... Can you either add a screengrab of your view and relationships, or describe both as clearly as possible and in more detail? :)
Answer: Thanks. Im attaching image how the structure look like. The result should be the page for every content type except Manufacturer, where the user can filter the content by manufacturer. So there will be exposed filter with list of avaliable manufacturers who is assigned to current content type according to users categories.
Now just came in my had that Manufacturer can be taxonomy eather. But Im not shure if it will be easier to make this happen.

entity reference and exposed filter


Can I approximate this binomial problem with a standard normal?

Question: I'm looking at the diameters of trees, so let $X$ be the diameter random variable. Im told that the diameters are normally distributed with $\mu = 8.8$ and $\sigma =2.8$. Now if four trees are selected independently, what is the probability that at least one has a diameter exceeding 10 inches? I don't think this is a standard normal distribution problem but rather closer to a binomial distribution. I'm not entirely sure what it's asking.
Comment: Calculate with normal distribution the P that a tree doesnt exceed 10 in. Once you have that P, you can then apply binomial distribution with n=4, the P from above to calculate that no four trees are more than 10 in. Then subtract from 1 (complement rule)
Comment: That's what I tried and I got zero... It wouldn't make much sense that with a probability of zero that no trees out of four wouldn't even be near the mean.
Comment: What is the P that a single tree does not exceed 10 in? That's normal distribution. Raise that to the power 4. What did you get?
Answer: For a randomly selected tree, let $p$ be the probability it is $\le 10$. Then
$$p=\Pr(X\le 10)=\Pr\left(Z\le \frac{10-8.8}{2.8}\right),$$
where $Z$ is standard normal. You can look up this probability in a table of the standard normal.
The probability all $4$ are $\le 10$ is $p^4$. So the probability at least one is $\gt 10$ is $1-p^4$.
Comment: Ah I see it was much simpler than I thought. Thanks!
Comment: You are welcome. I we had a sample of $12$ and wanted the probability of $6$ of them being $\le 10$ we would use the binomial with the $p$ of the above answer.


How to add css framework into html file with existing basic css

Question: I'm wondering if you are able to add a css framework, specifically bulma, into an html sheet with existing css.  The current css is just a standard < link > style.css, and I'm wondering if I can just link the bulma https://cdnjs.cloudflare.com/ajax/libs/bulma/0.5.1/css/bulma.css
 with the same < link > tag just under or above it on the same sheet.
Comment: You should be able to yes.
Comment: Try it...and see the magic happen.
Comment: trying to do before questioning is a better option.
Answer: Simple Starter from Bulma Website
http://bulma.io/documentation/overview/start/
<!DOCTYPE html>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Hello Bulma!</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/font-awesome/4.7.0/css/font-awesome.min.css">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.5.0/css/bulma.min.css">

        Hello World

        My first website with Bulma!

Answer: <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.5.1/css/bulma.css">

in the ... of your html file and it will works
Answer: I think the question is probably missing a part.
The part where the question is raised : will this inclusion of the bulma css interfere with the existing style.css?
In that case.. probably.. as soon as pure HTML elements are described in the style.css and return in the bulma.css, for example: table {....} is present in the bulma css, it could cause some strange behavior.
There are several ways to try and bypass this behavior, some of them clean, others less clean :)
First you need to decide whether the bulma is the default css to be used or not...
If so, use your own style.css to overwrite the bulma.css.
If not, still call your style.css as last css in the row.. items that are not defined in your stylesheet will be taken care of by the bulma css
Comment: I'm a bit sad that my answer got a downvote... I just pointed out that inserting another css that contains same basic elements will cause styles to change. unless you apply !important to the once you don't want to change.


[a b]=cholcov function returns a=[] and b=NaN for a symmetric positive definite matrix

Question: I have this matrix:
S=   6.84370358358718e-006    -7.45833473076585e-007
    -7.45833473076565e-007     7.11723106043725e-006

It is symmetric:
S-S'=                     0    -2.00111533788828e-020
      2.00111533788828e-020                         0

and is positive definite:
eig(S)= 6.22219831321029e-006    and     7.73873633081414e-006

When I use [a b]=cholcov(S), it returns returns a=[] and b=NaN. It is written in MatLab help that[T,num] = cholcov(SIGMA) ... If SIGMA is not square and symmetric, num is NaN and T is empty.
Of course the chol(S) function decomposes this function without any error. I  don't know the difference between chol and cholcov and it is not important, since I don't have any choices. The error comes from mvnrnd(zeros(1,2),S) function, when I try to generate some random numbers:
??? Error using ==> mvnrnd at 118
SIGMA must be a symmetric positive semi-definite matrix.
Can anyone tell me what's wrong here?
thanks.
Answer: It is NEARLY symmetric. You yourself admit that it is NOT symmetric. Is being ALMOST pregnant the same thing as pregnant? You show that S~=S'. The fix is easy.
S = (S + S')/2;

Answer: You wrote:
S-S'=                     0    -2.00111533788828e-020
      2.00111533788828e-020                         0

That says that S is not symmetric.  It's ALMOST symmetric.  But... not quite.  If this is due to numerics, you might be able to fix this with:
symmetricS = mean(cat(3,S,S'),3);

Comment: Thanks a lot. I thought Matlab considers e-20 zero!


How to remove mouseover/mouseout event

Question: I have a site with three tabs that I'm trying to dynamically add mouseover/mouseout event depending on which tab is clicked, the problem is that it appears that the mouseover/out events are 'bound' to the tab after they're called.  Is there a way to 'unbind' the event from the tabs?
Here's what my js looks like
onTab1Clicked()
{
    $('#tab2').mouseover(function() {
        $('#tab2').addClass('outlineBorder');
        });
    $('#tab2').mouseout(function() {
        $('#tab2').removeClass('outlineBorder');
        });
    $('#tab3').mouseover(function() {
        $('#tab3').addClass('outlineBorder');
        });
    $('#tab3').mouseout(function() {
        $('#tab3').removeClass('outlineBorder');
        });
}

onTab2Clicked()
{
    $('#tab1').mouseover(function() {
        $('#tab1').addClass('outlineBorder');
        });
    $('#tab1').mouseout(function() {
        $('#tab1').removeClass('outlineBorder');
        });
    $('#tab3').mouseover(function() {
        $('#tab3').addClass('outlineBorder');
        });
    $('#tab3').mouseout(function() {
        $('#tab3').removeClass('outlineBorder');
        });
}

onTab3Clicked()
{
    $('#tab2').mouseover(function() {
        $('#tab2').addClass('outlineBorder');
        });
    $('#tab2').mouseout(function() {
        $('#tab2').removeClass('outlineBorder');
        });
    $('#tab1').mouseover(function() {
        $('#tab1').addClass('outlineBorder');
        });
    $('#tab1').mouseout(function() {
        $('#tab1').removeClass('outlineBorder');
        });
}

This works fine on page load but if I click away from tab1 (page load state) to another tab then back to tab1 the mouseover/out events still fire.
Thanks.
Comment: You should accept answers to your questions (via the check-mark beside the one that helped).  It'll help get answers in the future, and helps the next person searching with the same problem, finding your question.
Answer: You can simplify your approach overall by making some very simple changes here.  First, give those #tab1, #tab2 and #tab3 elements a class, e.g. class="tab" then you can do this:
$(".tab").click(function() {
  $(this).addClass('active'); //make this tab active
  $(".tab").not(this).removeClass('active'); //make other tabs inactive
});

Now when you click any tab, it gets the "active" class, and the others won't have it.  Then you can use .live() with the :not() selector to get the hover effect you want, like this:
$('.tab:not(.active)').live('mouseover mouseout', function() {
  $(this).toggleClass('outlineBorder');
});

This selector won't act upon a tab while it's .active, so the hover effect only works on the tab that isn't selected, like you originally had, but much easier to maintain.
Comment: +1 Massive code reduction, no binding/unbinding, clever use of `.live()`, I'd say you just about covered the bases. :o)
Answer: $('#tab1,#tab2,#tab3').click(function(){
    $(this).unbind('mouseout mouseover');
    // this will unbind mouseout and mouseover  events when click
    // e.g. onTab1Clicked, mouseout and mouseover events will be unbind on tab1
})

Comment: You can call `.unbind()` with multiple event as well, e.g. `.unbind('mouseout mouseover')`
Comment: Ahh, I was not sure about that Nick, thanks for clearing it to me.. ;)
Answer: Yep, you nearly had it!
$('#tab3').unbind('mouseout');

How to remove mouseover/mouseout event


Keep line breaks when printing custom plain text textarea field

Question: I'm using Drupal 7. I'm printing out a custom plain text textarea field in the node template using the following code:
print ($taxonomy_term->field_address['und'][0]['value']);

However, when I print it out I lose all the line breaks.  Is there a way I can keep the line breaks in when I print out the field value?
I have already changed the plain text format to add "limit allowed HTML tags" rule for

, and removed the "display any HTML as plain text" rule.
Many thanks,
Ben
Answer: I worked it out. I just needed to use the following code:
print nl2br($taxonomy_term->field_address['und'][0]['value']);

This uses the print nl2br function to convert line breaks to
 tags.


SQLite index benchmark

Question: The problem
I have the following schema in an SQLite DB:
table: Dispatches

rowid: primary key
identifier: string
route: integer
slot: integer

The most common query is:
SELECT * FROM dispatches WHERE route = 1 ORDER BY slot ASC

While attempting to optimize the indexes used I tried 4 approaches:

No indexes
Index by route
Composite index by route and slot
Separate index by route and slot

For each case I'm inserting n=10000 records. The route is either 0 or 1 (random) and the slot is a random number between 0 and n-1.
Shockingly, when results are ran and the approaches are ordered by speed, I'm getting the same order. In other words, the fastest one is the table with no indexes.
SQLite details
Here are the EXPLAIN QUERY PLAN outputs (edited):

SCAN TABLE dispatches USE TEMP B-TREE FOR ORDER BY
SEARCH TABLE dispatch_index_routes USING INDEX index_dispatch_index_routes_on_route (route=?) USE TEMP B-TREE FOR ORDER BY
SEARCH TABLE dispatch_index_composites USING INDEX index_dispatch_index_composites_on_route_and_slot (route=?)
SEARCH TABLE dispatch_index_separates USING INDEX index_dispatch_index_separates_on_route (route=?) USE TEMP B-TREE FOR ORDER BY

Test script
This was all implemented using ActiveRecord (I'm actually optimizing for an Android app, but I figured that it shouldn't be that much different when benchmarking in a PC):
#!/usr/bin/ruby

require 'active_record'
require 'sqlite3'
require 'benchmark'

db_name = 'test_db'
# Change the following to reflect your database settings
ActiveRecord::Base.establish_connection(
  adapter:  'sqlite3', # or 'postgresql' or 'sqlite3'
  host:     'localhost',
  database: db_name
)

#ActiveRecord::Base.connection.execute("CREATE DATABASE IF NOT EXISTS #{db_name}")
#ActiveRecord::Base.connection.execute("USE test_db")

class Dispatch < ActiveRecord::Base

end

class DispatchIndexRoute < ActiveRecord::Base

end

class DispatchIndexComposite < ActiveRecord::Base

end

class DispatchIndexSeparate < ActiveRecord::Base

end

class DispatchIndexCovering < ActiveRecord::Base

end

# Define a minimal database schema
unless ActiveRecord::Base.connection.table_exists?(:dispatches)
  ActiveRecord::Base.connection.create_table :dispatches, force: true do |t|
    t.string  :identifier
    t.integer :route
    t.integer :slot
  end
end

unless ActiveRecord::Base.connection.table_exists?(:dispatch_index_routes)
  ActiveRecord::Base.connection.create_table :dispatch_index_routes, force: true do |t|
    t.string  :identifier
    t.integer :route, index: true
    t.integer :slot
  end
end

unless ActiveRecord::Base.connection.table_exists?(:dispatch_index_separates)
  ActiveRecord::Base.connection.create_table :dispatch_index_separates, force: true do |t|
    t.string  :identifier
    t.integer :route, index: true
    t.integer :slot, index: true
  end
end

unless ActiveRecord::Base.connection.table_exists?(:dispatch_index_composites)
  ActiveRecord::Base.connection.create_table :dispatch_index_composites, force: true do |t|
    t.string  :identifier
    t.integer :route
    t.integer :slot

    t.index [:route, :slot]
  end
end

unless ActiveRecord::Base.connection.table_exists?(:dispatch_index_coverings)
  ActiveRecord::Base.connection.create_table :dispatch_index_coverings, force: true do |t|
    t.string  :identifier
    t.integer :route
    t.integer :slot

    t.index [:route, :slot, :id, :identifier], name: 'dispatch_covering_idx'
  end
end

classes = [Dispatch, DispatchIndexRoute, DispatchIndexComposite, DispatchIndexSeparate, DispatchIndexCovering]

classes.each do |c|
  ActiveRecord::Base.connection.execute("DELETE FROM #{c.table_name};")
  ActiveRecord::Base.connection.execute("VACUUM;")
end

puts "Creating items"
n = 10000
n.times.each do |i|
  name = SecureRandom.hex
  route = Random.rand(2)
  slot = Random.rand(n)
  classes.each do |c|
    c.new(identifier: name, route: route, slot: slot).save!
  end
  if i > 0 && (i % (n / 10)) == 0
    print i
  end
end
puts "Done creating items"

ActiveRecord::Base.connection.execute("ANALYZE;")

iterations = 1
classes.each do |c|
  table_name = c.table_name
  puts "Results for #{table_name}"
  query = "SELECT * FROM #{table_name} WHERE route = 1 ORDER BY slot ASC"
  # puts ActiveRecord::Base.connection.execute("EXPLAIN QUERY PLAN #{query}")
  Benchmark.bm do |bm|
    bm.report do
      iterations.times do
        ActiveRecord::Base.connection.execute(query)
      end
    end
  end
end

And the output:
Creating items
100020003000400050006000700080009000Done creating items
Results for dispatches
       user     system      total        real
   0.100000   0.010000   0.110000 (  0.108072)
Results for dispatch_index_routes
       user     system      total        real
   0.120000   0.000000   0.120000 (  0.123919)
Results for dispatch_index_composites
       user     system      total        real
   0.130000   0.000000   0.130000 (  0.138045)
Results for dispatch_index_separates
       user     system      total        real
   0.140000   0.000000   0.140000 (  0.138734)

Versions:
$ sqlite3 --version
3.8.5 2014-08-15 22:37:57 c8ade949d4a2eb3bba4702a4a0e17b405e9b6ace
$ ruby --version
ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-darwin14.0]

Question
I don't understand the results. Shouldn't the composite index be the fastest? I see that the first one is using SCAN instead o SEARCH, but still. How can it be faster?
Edit
At first glance, adding a covering index does not speed up the queries:
Results for dispatches
       user     system      total        real
   0.100000   0.000000   0.100000 (  0.104801)
Results for dispatch_index_routes
       user     system      total        real
   0.130000   0.010000   0.140000 (  0.133568)
Results for dispatch_index_composites
       user     system      total        real
   0.130000   0.000000   0.130000 (  0.143194)
Results for dispatch_index_separates
       user     system      total        real
   0.140000   0.000000   0.140000 (  0.144586)
Results for dispatch_index_coverings
       user     system      total        real
   0.140000   0.000000   0.140000 (  0.137730)

It does get better after running ANALYZE as @CL. suggested:
Results for dispatches
       user     system      total        real
   0.090000   0.000000   0.090000 (  0.098393)
Results for dispatch_index_routes
       user     system      total        real
   0.130000   0.000000   0.130000 (  0.137758)
Results for dispatch_index_composites
       user     system      total        real
   0.130000   0.010000   0.140000 (  0.134864)
Results for dispatch_index_separates
       user     system      total        real
   0.140000   0.000000   0.140000 (  0.142418)
Results for dispatch_index_coverings
       user     system      total        real
   0.100000   0.000000   0.100000 (  0.098227)

Trying with a less randomized route value doesn't help much, since only some options get a speedup:
Results for dispatches
       user     system      total        real
   0.220000   0.010000   0.230000 (  0.228870)
Results for dispatch_index_routes
       user     system      total        real
   0.170000   0.020000   0.190000 (  0.190722)
Results for dispatch_index_composites
       user     system      total        real
   0.230000   0.000000   0.230000 (  0.242936)
Results for dispatch_index_separates
       user     system      total        real
   0.240000   0.010000   0.250000 (  0.253333)
Results for dispatch_index_coverings
       user     system      total        real
   0.190000   0.000000   0.190000 (  0.205272)

Comment: Just running [ANALYZE](http://www.sqlite.org/lang_analyze.html) should help.
Answer:
The route is either 0 or 1 (random)

I think this is the key to why the performance is essentially the same -- or even that the indexes make things worse.  If I understand correctly, you need to read every other record.
A full-table scan reads the records and discards the unneeded ones. An index scan goes into the index and reads only the ones you need.  However, it is then bouncing between data pages for each record.  My guess is that the overhead of reading the record in the index and the non-locality reference is about the same as reading an additional row in the table.
So, an index for filtering is not going to help (actually, a clustered index might help).
That leaves the sorting.  I'm not sure how the SQLite optimizer works.  However, the following index might show some gain:  (route, slot, rowid, identifier).  This is a covering index, so there is no need to go to the data pages when processing the query.
Comment: Makes a lot of sense. I'll try the covering index and get back to you.
Comment: Here's a great article talking about this concept, which is called selectivity - [Selectivity in SQL Databases](http://www.programmerinterview.com/index.php/database-sql/selectivity-in-sql-databases/).

SQLite index benchmark