Função Javascript retornando Undefined

Question: Estou com um problema com uma função JavaScript. Eu preciso verificar a existência de um item no banco de dados e por isso uso uma função js com ajax chamada verificaExistente. O php me retorna um Json e nessa função eu verifico se existem elementos naquele retorno.
O retorno do banco está acontecendo como previsto, o problema é o return. Quando chamo essa função dentro do onclick do botão salvar, o valor retornado é undefined. Alguém já teve algum erro parecido? Agradeço a ajuda.
Abaixo o código:
$( "#btnSalvar" ).on( 'click', function(){
    if( $( "#frmDados" ).valid() !== false ){
        if(verificaExistente()) salvar();
        else alert("Dados já existentes");
    }
});

function verificaExistente(){
    $.ajax({
        url: "verificaItem.php",
        type: 'POST',
        dataType: 'json',
        cache: false,
        data:{
            id_tabela          : $("#id_tabela").val(),
            id_revestimento    : $("#id_revestimento").val(),
            operador           : 'AND'
        },
        error: function(){
            alert('Erro ao Tentar ação.');
        },
        success: function( retorno ){
            if(retorno.dados.length > 0) return false;
            return true;
        }
    });
}

Comment: Relacionada: [função $.Ajax() retornar valor?](http://pt.stackoverflow.com/q/29382/91)
Comment: Sim. O retorno `retorno` no `success` do `$.Ajax()` vem normal. Verifiquei com um `console.log` e o valor da verificação é realmente zero.
Answer: O que está acontecendo é que você está pegando o retorno da função verificaExistente e, como ela não retorna nada, o resultado é undefined. O que está retornando true/false é a função associada ao success do ajax e não a verificaExistente. Entretanto, a função de click não recebe o retorno da success, mas sim da função verificaExistente, que, como foi explicado anteriormente, sempre retornará undefined.
O correto seria você verificar se existe e só depois chamar uma função de retorno, como no exemplo a seguir:
$( "#btnSalvar" ).on( 'click', function(){
    if( $( "#frmDados" ).valid() !== false ){
        verificaExistente()
    }
});

function verificaExistente(){
    $.ajax({
        url: "verificaItem.php",
        type: 'POST',
        dataType: 'json',
        cache: false,
        data:{
            id_tabela          : $("#id_tabela").val(),
            id_revestimento    : $("#id_revestimento").val(),
            operador           : 'AND'
        },
        error: function(){
            alert('Erro ao Tentar ação.');
        },
        success: function( retorno ){
            if(retorno.dados.length > 0) alert("Dados já existentes");
            else salvar();
        }
    });
}

Entretanto, o fluxo correto seria fazer essa verificação no lado do servidor apenas, usando PHP ou a tecnologia que você está usando para fazer essa validação e apenas retornar um erro, caso os dados já existam ou uma mensagem de sucesso através do ajax.
Comment: Puts. Funcionou legal. Eu achava que os `return false` e  `return true` dentro do `success` seria repassado para a minha função. Valeu a ajuda @Felipe Avelar.
Comment: Consegui entender o raciocino, mas os `return true` ou `return false` não fariam a verificação no `if` funcionar?
Comment: Como eu disse, ele não tá retornando nada, vou editar pra ficar melhor essa parte da explicação...
Answer: Tens de ter em conta que uma chamada AJAX é assincrona e o resultado que pretendes vai aparecer dentro da função success. Quando corres a função verificaExistente ele vai iniciar o AJAX que vai correr paralelamente e acabar a função sem esperar pelo ajax.
O que sugiro é fazeres uma callback. Por exemplo:
$( "#btnSalvar" ).on( 'click', function(){
    if( $( "#frmDados" ).valid() !== false ){
        verificaExistente(function(retorno ){
             if(retorno.dados.length > 0) return alert("Dados já existentes");
             salvar();
        });
    }
});

function verificaExistente(fn){
    $.ajax({
        url: "verificaItem.php",
        type: 'POST',
        dataType: 'json',
        cache: false,
        data:{
            id_tabela          : $("#id_tabela").val(),
            id_revestimento    : $("#id_revestimento").val(),
            operador           : 'AND'
        },
        error: function(){
            alert('Erro ao Tentar ação.');
        },
        success: fn // usa a callback aqui
    });
}

Dessa maneira passas a função e o que queres fazer para a resposta do AJAX, que será executado quando o servidor responder.
Answer: Uma forma seria você criar um outro método e passar o valor para ele dentro do       "success", e assim ter o que deseja.
Exemplo:
$.ajax({
    url : site,
    type : 'GET',
    success : function(x) {
    //Faz algo
    metodoX(valor); //Aqui você chama outro método e passa o valor retornado do ajax.
    }
});

function metodoX(valor){
console.log(valor);
}

Lembrando que isso não é uma boa prática.

Função Javascript retornando Undefined


R-continuity of step function

Question: Let us define a function $f\colon\mathbb R^2\to\mathbb R$ by
$$
f(x,y) =
\begin{cases}
   1 & \text{if $xy\le 0$,} \\
   0 & \text{if $xy>0$.}
\end{cases}
$$
Does limit of $f$ as $(x,y)$ tends to $(0,0)$ exist?
Answer: No. In every neighborhood of $(0,0)$ you can find points $(x,y)$ with $xy>0$ as well as points with $xy\le 0$. Hence $f$ takes values $0$ and $1$ in every neighborhood of $(0,0)$ and is not continuous by the $\varepsilon$-$\delta$-definition of continuity.
Answer: For $t>0$ we have $f(t,t)=0$ and $f(t,-t)=1$. Conclusion ?

R-continuity of step function


Using canvas to draw a clock. Query about the save()

Question: I know the function of save() and restore() that they save and restore the current canvas states. But still have no profound understanding of these. When the program call the function to draw the clock, it has cleared all the canvas before redrawing. Why the save() should also be added here. Can anyone explain it? Thanks.

window.onload=function(){

   var oCan=document.getElementById('clock');
   var oCanPen=oCan.getContext('2d');
   drawClock();
   setInterval(function(){
    drawClock();
   },1000);
   function drawClock(){
    // oCanPen.save();
    oCanPen.clearRect(0,0,oCan.width,oCan.height);
    var time=new Date();
    var second=time.getSeconds();
    var minute=time.getMinutes();
    var hour=time.getHours()+minute/60;

   oCanPen.save();

   oCanPen.beginPath();
   oCanPen.arc(200,200,100,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   for (var i = 0; i < 60; i++) {
    oCanPen.moveTo(200,200);
    oCanPen.arc(200,200,100,i*6*Math.PI/180,i*6*Math.PI/180,false);
    }
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.fillStyle='#fff';
   oCanPen.arc(200,200,90,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.fill();

   oCanPen.beginPath();
    for (var i = 0; i < 12; i++) {
    oCanPen.moveTo(200,200);
    oCanPen.arc(200,200,100,i*30*Math.PI/180,i*30*Math.PI/180,false);
    }
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.fillStyle='#fff';
   oCanPen.arc(200,200,80,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.fill();

   oCanPen.beginPath();
   oCanPen.strokeStyle='red';
   var iSecond=(-90+second*6)*Math.PI/180;
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,85,iSecond,iSecond,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.lineWidth=4;
   oCanPen.strokeStyle='blue';
   var iMinute=(-90+minute*6)*Math.PI/180;
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,60,iMinute,iMinute,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.lineWidth=6;
   var iHour=(-90+hour*30)*Math.PI/180;
   oCanPen.strokeStyle='black';
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,40,iHour,iHour,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.restore();
  }

  };
 body{
  background: #333;
 }
 canvas{
  background: #f2f2f2;
 }
<!DOCTYPE html>

<meta charset="utf-8"/>
 <title></title>

<canvas id="clock" width="400" height="400"></canvas>

Answer: save and restore are not affected by the content of the bitmap - they simply deal with the state as you say, or "settings" such as color styles, line widths, current font etc.
When you clear the canvas the state remains unchanged. And vica versa, when restoring a state it won't affect any existing content on the canvas.
Typical usage is if you have a state you prefer but need to temporary change things like transformation, styles etc. draw, and then restore so you can get back to what you had without the need of resetting those values.
In your case you can change lineWidth and strokeStyles without worrying about those later.
On the other side, you can avoid save/restore if you are able to keep track of all the state values that changes. This is in many cases faster than save/restore as the latter saves a complete state, not just style and line-width.
In your case you can simply remove save/restore and use this new line of code to avoid it all together:
..
var hour=time.getHours()+minute/60;

oCanPen.lineWidth = 1;

oCanPen.beginPath();
..

window.onload=function(){

   var oCan=document.getElementById('clock');
   var oCanPen=oCan.getContext('2d');
   drawClock();
   setInterval(function(){
    drawClock();
   },1000);
   function drawClock(){
    // oCanPen.save();
    oCanPen.clearRect(0,0,oCan.width,oCan.height);
    var time=new Date();
    var second=time.getSeconds();
    var minute=time.getMinutes();
    var hour=time.getHours()+minute/60;

            oCanPen.lineWidth = 1;

   oCanPen.beginPath();
   oCanPen.arc(200,200,100,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   for (var i = 0; i < 60; i++) {
    oCanPen.moveTo(200,200);
    oCanPen.arc(200,200,100,i*6*Math.PI/180,i*6*Math.PI/180,false);
    }
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.fillStyle='#fff';
   oCanPen.arc(200,200,90,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.fill();

   oCanPen.beginPath();
    for (var i = 0; i < 12; i++) {
    oCanPen.moveTo(200,200);
    oCanPen.arc(200,200,100,i*30*Math.PI/180,i*30*Math.PI/180,false);
    }
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.fillStyle='#fff';
   oCanPen.arc(200,200,80,0,360*Math.PI/180,false);
   oCanPen.closePath();
   oCanPen.fill();

   oCanPen.beginPath();
   oCanPen.strokeStyle='red';
   var iSecond=(-90+second*6)*Math.PI/180;
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,85,iSecond,iSecond,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.lineWidth=4;
   oCanPen.strokeStyle='blue';
   var iMinute=(-90+minute*6)*Math.PI/180;
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,60,iMinute,iMinute,false);
   oCanPen.closePath();
   oCanPen.stroke();

   oCanPen.beginPath();
   oCanPen.lineWidth=6;
   var iHour=(-90+hour*30)*Math.PI/180;
   oCanPen.strokeStyle='black';
   oCanPen.moveTo(200,200);
   oCanPen.arc(200,200,40,iHour,iHour,false);
   oCanPen.closePath();
   oCanPen.stroke();

  }

  };
body{
  background: #333;
 }
 canvas{
  background: #f2f2f2;
 }
<!DOCTYPE html>

<meta charset="utf-8"/>
 <title></title>

<canvas id="clock" width="400" height="400"></canvas>


Exercise in well-ordered sets

Question: Definitions: Given a subset $A$ of a non-empty well-ordered set $X$ we define the supremum as follows:  $\text{sup(A)}:=\text{min}\bigg(\big\{\,y\in X \oplus\{ +\infty\}: \forall x\in A \;(x\le y)\, \big \} \bigg)$.
If $x\in X$ we define the successor of $x$ by the formula: $\text{succ}(x):= \text{min}\big( ( x, +\infty ]\big)$
Exercise: If $x$ is an element of a well-ordered set $X$ show that exactly one of the following statements hold:
1.- (Limit Case) $x=\text{sup} \big( \,[\text{min}(X),x)\, \big)$
2.- (Successor case) $x=\text{succ}(y)$ for some $y$.
Hi everyone I have trouble with the above exercise. I'd like to put my attempt but  everything I've tried is wrong (Basically my idea is split the proof in two cases one to show that at most one case hold assuming by contradiction the opposite, and the other  which shows that at least one hold ). Also I don't have a real intuition about what the limit case really is, I mean,  with the successor case is easy only thinking in the natural numbers. But what about the limit case.
(This is what I have so far)
Sketch proof: We have to show that at most (1) or (2) hold. Suppose for the sake of contradiction that both cases occurs simultaneously. Let $P(x)$ be the statement "$x$ is such that (1) and (2)  holds at the same time". We set $\{x\in X: \,P(x)\,  \}$, then by hypothesis is a non-empty subset of $X$ which is well-ordered, then it has a minimum element $x$. Thus for all $x'< x$ either (1) or (2) holds but not both.
We already know that $x= \text{min} ((y, +\infty])$, then $x$ is the least such that $x>y$. Thus $y$ is either in (1) or (2) but not at the same time.
If $y$ is in (1). This means that $y= \text{sup} ([\text{min}(X),y)$, i.e., $y$ is the least such as $y\ge w$ for all $w\in [\text{min}(X),y)$. Also we have that $y\in [\text{min}(X),x)$.
We claim that $[\text{min}(X),x)=[\text{min}(X),y) \cup \{y\}$. Let $z\in [\text{min}(X),x)$, i.e., $\text{min}(X)\le z<x$, we have to check the case when $y\le z< x $ if were some $z\not= y$ contradicts that $x$ is the successor of $y$, hence $z=y$, the case when $z<y$ is trivially true. Hence $[\text{min}(X),x)\subset [\text{min}(X),y) \cup \{y\}$ the other inclusion is trivially true. Therefore $y$ is indeed an upper bound for  $[\text{min}(X),x)$ and also is the least, i.e., $y = \text{sup} ( \,[\text{min}(X),x)\,)$ and $y\not= x$ a contradiction.
If $y$ is in (2), $y= \text{succ}(z)$. Then $y\ge w$ for all $w\in [\text{min}(X),y)= [\text{min}(X),z\,]$. Then $y$ is the least which would imply $y=\text{sup} ( \,[\text{min}(X),y)\, )$ and contradicts the minimality of $x$.
For the other case, I'm not sure of how prove that at least one always hold? Any suggestion, please? Thanks
Comment: I assume by $\oplus$ you mean the disjoint union?
Comment: Yes, it is correct.
Comment: I think you start out in the wrong direction and make things too complicated.
Comment: Could you explain it please? I took the exercise of some notes.
Comment: The key is that there is no $z$ such that $y < z < y^+$.
Comment: What is $y^+$? It is what I call $\text{succ}(y)$?
Comment: Yes, Jose. Sorry to confuse matters.
Answer: I will write $y^+$ to mean the successor of $x$ to save on typing.
Only one can be true
Suppose $x = y^+$. Then $y \in [\min X\,..\,x)$.
Let $z \in [\min X\,..\,x)$. Then $z<x = y^+$. Either $z\le y$ or $z>y$. Suppose $z>y$. Then $y<z<y^+$, a contradiction. So $z\le y$. As this holds for all such $z$, $y$ is an upper bound of $[\min X\,..\,x)$. As it is an element of that set, it is in fact the maximum, and therefore the supremum. Since a set can have only one supremum, and $y \ne x$, $x$ is not the supremum of $[\min X\,..\,x)$.
At least one is true
Let $s$ be the supremum of $[\min X\,..\,x)$. Either $s\in[\min X\,..\,x)$ or $s$ is strictly greater than each element of $[\min X\,..\,x)$. I think you can probably finish this argument.
Comment: The first part is similar, but yours is nicer. For the second one, the lemma every well-ordered set which is bounded above it has a supremum is not given, but I think using strong induction follows.
Comment: No need to reformulate it in those terms. Every non-empty subset of a well-ordered set has a minimum. In fact, I missed the fact that you stuck an extra $\infty$ in there, which makes things work out properly. I'm a bit tired.
Comment: Ok, thanks for your help. Let me check the entire argument again.
Comment: @JoseAntonio, that's a good idea, especially considering how tired I am. You should also write out the rest of the "one of them must be true" argument, perhaps as your own answer to this question.
Comment: Thanks again, I've follow your advice and I've put my own answer below.
Answer: First we'd like to show that at most one hold. Suppose for the sake of contradiction that both cases occurs simultaneously. We have $x= \text{succ} (y)= \text{sup} ([\text{min}(X),x))$. Let $z$ be an arbitrary element of $([\text{min}(X),x))$, since $y\in [\text{min}(X),x)$ and the set is total, either $z>y$ or $z\le y$. For the former $y<z<x$ which contradicts that $x=\text{succ} (y)$. For the latter, since $z$ was arbitrary $y\ge z$, $\forall z\in[\text{min}(X),x)$, i.e., is an upper bound, it is the least because $y\in [\text{min}(X),x)$. Since $y\not=x$ we have a contradiction.
To conclude the proof we shall show that at least on of the cases always happen. Let $x\in X$ be arbitrary. Then the set $[\text{min}(X),x)$ is bounded above. Let $\mathcal{S}$ be the set of all the upper bounds of $[\text{min}(X),x)$, clearly is non-empty and therefore it has a least element that we call $s$. Then $s= \text{sup}([\text{min}(X),x))$ and either lies in the set or not.
If were the case that $s\in [\text{min}(X),x)$, then is the maximal element and then we claim $x=\text{succ}(s)$. If were not the case, let $z=\text{succ}(s)$. Now since $z<x$, we have $z\in [\text{min}(X),x)$ but  by construction $s<z$, a contradiction. Hence $x=\text{succ}(s)$.
On the other hand, if we have that $s\notin [\text{min}(X),x)$, then we claim that $s= x$. If were not the case then either $s<x$ or $s>x$. If $s<x$, then $s\in [\text{min}(X),x) $ a contradiction. If $s>x$, we already know that for all $w\in [\text{min}(X),x) $, $w<x$ then is an upper bound which is least than the least upper bound, a contradiction. Hence $x= \text{sup}([\text{min}(X),x))$.

Exercise in well-ordered sets


How to choose prior in Bayesian parameter estimation

Question: I know 3 methods to do parameter estimation, ML, MAP and Bayes approach. And for MAP and Bayes approach, we need to pick priors for parameters, right?
Say I have this model $p(x|\alpha,\beta)$, in which $\alpha,\beta$ are parameters, in order to do the estimation using MAP or Bayes, I read in the book that we'd better pick a conjugate prior $p(\alpha,\beta)$, which is a joint probability of $\alpha,\beta$, right?
I have 2 questions:

Do we have other choices picking the prior other than this conjugate one?
Can we pick priors for $\alpha$ and $\beta$ respectively like $p(\alpha)$ and $p(\beta)$, other than put them together in a joint one?
Comment: Depending on what software you'll use, priors certainly don't have to be conjugate to the likelihood function... first and foremost, you should make sure that your priors represent your prior beliefs about the distribution of the parameters
Comment: Look up *prior elicitation*, e.g. [here](http://stats.stackexchange.com/questions/1/eliciting-priors-from-experts)
Comment: So I could just pick priors respectively for the parameters,right ? ACTUALLY I just try to understand baysian linear regression, no specific software considered
Comment: How to show that a Beta distribution is a conjugate prior for a Binomial distribution https://youtu.be/Q4HFHXs2m7Y
Answer: As stated in comment, the prior distribution represents prior beliefs about the distribution of the parameters.
When prior beliefs are actually available, you can:

convert them in terms of moments (e.g. mean and variance) to fit a common distribution to these moments (e.g. Gaussian if your parameter lies to the real line, Gamma if it lies to $R^+$).
use your intuitive understanding of these beliefs to propose a given prior distribution and check if it really fits your purpose and that it is not to sensitive to arbitrary choices (performing a robustness or sensibility analysis)

When no explicit prior beliefs are available, you can:

derive (or simply use if already available, a great ressource is http://www.stats.org.uk/priors/noninformative/YangBerger1998.pdf) a Jeffreys (e.g. uniform for a location parameter) or a reference prior (especially in case of multivariate parameters).
sometimes such choices are impossible or quite difficult to derive and in this case, you can try to choose among one of the many "generic" weakly informative prior (e.g. uniform shrinkage distribution for scale parameters of hierarchical model or $g$-prior for gaussian regression).

Having said that, there is no restriction to use a joint or an independent prior ($p(a,b)$ Vs $p(a) \cdot p(b)$). As a complement, I would say that in my humble opinion, there are three major things to take care when choosing a prior:

take care that your posterior is integrable almost everywhere (or proper), which is always true if you use an integrable prior (see Does the Bayesian posterior need to be a proper distribution? for more details),
limit the support of your prior only if you are highly confident on the support bounds (so avoid to do it).
and last but not least, make sure (most of the time experimentally) that your choice of prior means what you want to express. In my opinion, this task is sometimes the most critical. Never forget that when doing inference, a prior means nothing by itself, you have to consider the posterior (which is the combination of prior and likelihood).
Comment: I'm also self-studying machine learning stuff, I knew ML, but this bayesian approach of parameter estimation is new to me, hope you can show me some material to learn bayesian estimation and inference, ;-)
Comment: @loganecolss, [This](https://engineering.purdue.edu/kak/Trinity.pdf) is a good summary of the MLE, MAP and Bayesian inference. And this [link](http://stats.stackexchange.com/questions/47771/what-is-the-intuition-behind-beta-distribution) give s a good summary of how to incorporate a prior to Bayesian inference for a binomial distribution.
Comment: Thank you so much, could you please recommend me some tutorial stuff on how to do this kind of bayesian inference?
Comment: @loganecolss You're welcome, I was personnaly a bit lost a few months ago and this post is simply the summary of my self-study and I am happy if it can help somebody else. Concerning your question, what do you mean by "this kind of bayesian inference" ?
Comment: A minor elaboration: A proper prior represents *a* consistent set of beliefs about the parameters. They don't have to be *your* beliefs. Indeed models are often more persuasive when they are some else's.
Comment: If I want to fit an bayesian model to a small dataset...  Can I first apply that model to a large dataset and use its output as prior for the small one?
For example I want to fit a regression model to see if smoking has an effect on cancer using 30 observations. And I first apply my model to a public health service database with 1 million observations.
Answer: There is also empirical Bayes. The idea is to tune the prior to the data:
$$\text{max}_{p(z)} \int p(\mathcal{D}|z)p(z) dz$$
While this might seem awkward at first, there are actually relations to minimum description length. This is also the typical way to estimate the kernel parameters of Gaussian processes.
Answer: To answer the two questions above directly:

You have other choices to choose non-conjugate priors other than conjugate priors. The problem is that if you choose non-conjugate priors, you cannot make exact Bayesian inference (simply put, you cannot derive a close-form posterior). Rather, you need to make approximate inference or use sampling methods such as Gibbs sampling, Rejection sampling, MCMC, etc. to derive you posterior. The problem with sampling methods is that intuitively, it's like drawing a picture of elephant in darkness by repetitively touching it----you may be biased and incomplete. The reason people choose non-conjugate prior is that for certain likelihood, conjugate prior option is pretty limited, or to say, most are non-conjugate.
Yes, you definitely can. If α and β are independent, which is the idealistic condition, you can derive their joint distribution by p(α)p(β). If they are not independent, you may need to figure out the conditional probability and do integral to derive joint distribution.


Fastest method to read/write to SD card in Android?

Question: There are a lot of different classes that can be used in various ways to read/write to files in Android. For example, you can make use of java.nio.ByteBuffer, FileOutputStream and BufferedOutputStream. Are there any general guidelines for what to use to read/write quickly to the SD card? For example, BufferedOutputStream seems as if it should make things faster but I'm unsure the buffer size should be set for.
Specifically, I want to read/write byte arrays that are ~1Mb in size as quickly as I can.
Comment: Why not write a benchmark and then share the results?
Answer: This document benchmarks different ways to read files quickly in Java on a PC (the 'Conclusions' section is definitely worth a read). You might find it useful as guidance, but you should really just try different approaches and see what works quickest for you.


inconsistency between du -sh and df -h

Question: I know this is a common occurrence when the result of du -sh is smaller than that returned by the file system with df -h. (Because some files are still open in processes etc..) But in my case I have the opposite.
I'm running Ubuntu 12.04 and trying to get the used size of an NFS mount
df -h returns 270G used while du -sh of the mounted folder returns 320G used.
Update:
I'm mounting the partition with the following options:
nas-server:/path/to/mount /mnt/mount/point  nfs proto=tcp,rsize=8192,wsize=8192,hard,intr,exec

Does anyone know where this could come from? The correct amount on the disk should be 270G~
Thanks for your help. I'll provide any extra information necessary.
Comment: This difference can be triggered by block size on the NAS server. du shows number of byes, df - number of blocks * block size. Additionally, df asks NAS server for usage information, du counts it locally,
Comment: Hmm... how does NFS handle hard links? Is it possible that `du` is couning hard links as separate files due to NFS?
Comment: Does "correct amount" refer to the amount that nas-server reports locally (i.e. not over NFS, but local df or Web UI or similar)?
Comment: Yes. I've also copied all data onto an external drive to double check and both `df` and `du` returned the same value of about ~270G (locally)
Comment: I would check inode usage using `df -i` and run fsck on the filesystem.
Comment: Also I'd try to run `du -ms *` both on the share from the client perspective and locally on the server. By comparing the output, you might be able to narrow it down.
Comment: Another possibility might be sparse files. If NFS doesn't support them properly, `du` might believe that they use more space than they actually do.
Answer: du counts the blocks used by hardlinked files once, not once per hardlink.  However, there are some ways this de-duplication can fail:

The table du uses to de-dup hardlinked files is a fixed size.  If you have more hardlinked files than the table can store, de-duplication may not be successful.  (Some versions of du have a dynamicaly sized table and don't have this problem.)
The de-duplication is based on inode values.  If the NAS server shows different inode numbers for files that are hardlinked, then de-duplication is not possible.  Some NAS servers do a great job of presenting inodes because they use a filesystem that has inodes.  Others have to "fake it" and don't do a good job.

By the way...
du counts just the file data.
df counts blocks use for the file data plus all the meta data: directories, superblock, inode table, direct/indirect/doubleindirect blocks and so on.
Therefore, df should return a smaller "used" size than du.  If the opposite is happening, I would presume this de-duplication is broken or the NAS Server has done something that makes df display invalid information.
Comment: Voting you up, your conclusion was exact.
Answer: Thanks to everyone for the great answers and suggestions.
I'm posting to answer my own specific problem. Just on the offchance it helps someone (which it very well might not given the circumstances)
The NAS I was getting my filesystem information from has - if I'm not mistaking - a certain level of virtualisation as far as the partitions go (HP X9000). Therefore df should return an accurate "estimate" if all goes well.
However due to a bug with the NAS that has since then been fixed, the size of the virtual partition was not being updated and therefore df would return an invalid (outdated) value. Showing 270Go instead of the actual correct value of 320Go (I made a mistake in my comments)
All the above issues stemmed from this. Since then this has also occured on occasion when the NAS has been working in degraded mode (for whatever reason)
Thanks again guys.

inconsistency between du -sh and df -h


Dynamic footer link based on current url

Question: I've got a link to the mobile version of my site in the footer, but I need this link to be dynamic based off the page that the user is currently on.
Shown below are the options I have for the link.

Under the path there is this help text
The path for this menu link. This can be an internal Drupal path such as node/add or an external URL such as http://drupal.org. Enter <front> to link to the front page.
What I need is similar to the ability to use <front> to signify the homepage. I'd like to be able to use something like http://m.example.com/<node> so that this link will always go to the mobile version of the page that I'm currently on.
Is this possible? Or is what I want to do more complex than just using a keyword for the path?
Answer: The default menu system won't let you do that...but if you install the Menu Token module you should be able to use a path similar to the following to get what you need:
http://m.example.com/[current-page:url]

Comment: Perfect, does exactly what I need. I ended up using `[current-page:url:relative]` to get what I wanted. Many Thanks


Why does SQL Server require Office Web Components?

Question: After installing SQL Server 2005 (with SSNS, SSIS and SSAS) I found Office Web Components (OWC) installed as well. My customer is asking why OWC is required I have nothing to say. Any ideas? Is there any additional licensing requirement?
Answer: Sergejus,
Office components are installed are part of SSIS or SSRS and not with Business Inteligence Development Studio (BIDS). I just succeeded applying MS12-027 hot fix.
Answer: They are used for the reports
To be precise - Reporting Services
Comment: I haven't installed SSRS on this machine, only SSNSm SSIS and SSAS
Answer: OK, seems I know what's going on. OWC are installed together with Business Inteligence Development Studio (BIDS). It's required to connect to the OLAP cubes.


Excel ActiveX labels blink / change style upon clicking

Question: This, what seems to be a glitch, has annoyed me for years.  Some ActiveX controls, when placed on the worksheet, do odd things when clicked.  In the picture below, there is no code whatsoever and Label1 is disabled, but when Label1 is clicked, Label2 disappears.  I've also seen dark labels turn white upon clicking. The only way I know of to prevent this would be to use subclassing/hooking to intercept and cancel wm_mousedown messages when over the label but that is a ton of work for such a simple goal.  Does anybody out there have a simple solution for this?
*Edit: I believe what's happening is the z-order/z-index brings the clicked control to the top.  But if there's a way to prevent this I would love to know how.

Here is an example of the background color changing when a label is clicked.  See the label with the caption "Original Value:"
Comment: Really? Normally to me things stay in the order they were originally designed.  The background image of a webpage doesn't cover the entire page when you click it, for example.  You don't want a panel to cover all the items on a panel when the panel is clicked.  The order shouldn't change based on clicks.  If it's by design, I think it's a stupid design
Comment: You're saying that when you have physically overlapping labels, and you click the one that is behind, you want it to stay behind? The behavior you describe sounds by design. When you click something it changes the focus to the control, which normally you would want to be visible over anything that was placed over it.
Comment: Apples and oranges. When you have two different browser windows open on top of each other, and you click one, it takes focus and covers the second window. You clicked it, so Windows assumes you want to see it over the other window. That is by design. Textboxes and command buttons do the same thing as labels.
Comment: Windows sure, but for controls like labels that's retarded
Answer: Items should stay in the order they were laid out in, I have many projects with overlapping menus, shapes and labels and they all stay in the layer order they are organised in, the only time they move is if you programatically recreate them to update menues etc, then you need to write code to put them all back in order starting from the bottom up. If they are not staying in order then there is a problem and not a design flaw.


Prove $C^n(\mathbb{R})\cong C(\mathbb{R})$ (linear isomorphism)

Question: I need to prove that
$C^n(\mathbb{R})\cong C(\mathbb{R})$,
that is, there exists a bijective linear function from $C^n(\mathbb{R})$ onto  $C(\mathbb{R})$.
Answer: It's enough to show they have the same dimension (here the dimension is a certain infinite cardinal, namely the cardinality of a basis).
Since $C^n\subset C$ it's clear that $\dim(C^n)\le\dim(C)$. For the other inequality:

If $f\in C$ there exists $g\in C^1$ with $g'=f$.

Proof: $g(x)=\int_0^x f(t)\,dt$.
By induction:

If $f\in C$ there exists $g\in C^n$ with $g^{(n)}=f$.

So if we define $T:C^n\to C$ by $Tf=f^{(n)}$ then $T$ is surjective; hence $\dim(C^n)\ge \dim(C)$.
Here we used a basic fact about dimension:

If $T:X\to Y$ is linear then $\dim(TX)\le\dim(X)$.

Proof: Say $B$ is  a basis for $X$. Then $T(B)$ spans $TX$, so $T(B)$ contains a basis for  $TX$; hence $\dim(TX)\le\text{card}(T(B))\le\text{card}(B)=\dim(X)$.
Comment: Thank you very much!


doubts with queries

Question: I have a doubt about queries with LINQ library in C#.
I want to order by age, location and hobbies people that works in a company, and they've made calls. I have a class for those persons, and another class for the telephones, but they have no relation, I cannot go from one to another directly.
I've made a query to print all the calls that they've made;
private Model model = new Model();

( . . . )
ShowConsole(this.model.Calls
        .Where(call => this.model.Worker.Any(
                     worker => worker.TelephoneNumber.Equals(call.SourceNumber)))
        .Select(phonecall => new {
            Name = this.model.Employees.First(
            worker => worker.TelephoneNumber.Equals(phonecall.SourceNumber)),
            Duration = phonecall.Minutes
        }));

Now, I doubt how to add a condition with .Where, or another method, to check conditions (like, just print those persons which are from Japan).
How can I relate, and add filters and such to a query which plays with classes that are not related with each other directly?
Answer: Use OrderBy the Where and Select calls, then use ThenBy. If applicable use OrderbyDescending and ThenByDescending.
Just remember that using multiple OrderBy / OrderByDescending calls will result in multiple collections.
Answer: You can use the Join (eq SQL INNER JOIN) and GroupJoin (for other join kind) LINQ operators.

doubts with queries


Drum head choices for Hyperdrive toms (short)

Question: I have a Tama Starclassic Birch/Bubinga drumkit.  The toms are TAMA's Hyperdrive (short toms).  I am looking for new heads and ask the question what would be the best type of drum heads for short sized Toms, or how to choose? (1 ply, 2 ply, etc.) I have been using REMO Pinstripe heads but they appear to muffle these toms too much.
What are you thoughts and suggestions.
Tom sizes are as follows:
 8x6, 10x6 1/2, 12x7, 13x7 1/2
Comment: Note to close-voters. "Type" of skin doesn't have to include a brand recommendation per se - below I've used common brands for reference, but overall, all brands make their own version of all types.
Comment: Sorry, specific equipment recommendation questions are off-topic here. You could ask what the differences are between different types of drum heads and/or what effect the thickness of a head has on the sound, etc., and then use those answers to help you decide for yourself what kind of head to get. While different brands will have slightly different characteristics, mainly the type of construction and the thickness of the head are the two most important factors. If you want, you can edit this question by clicking the word "edit" in the lower left of the question.
Answer: Pinstripes - or any dual skin/oil-head do one job & do it quite well...
They dampen ring & overtone.
That's fine if you don't want ring & overtone, of course.
If you want those back, then lose the extra skin & the oil-fill.
That leaves you with a "bog standard" Remo Ambassador, clear, uncoated.
Cheap, cheerful, ubiquitous until the pinstripes took over.
Basically it's just a single fairly thin skin with lots of ring & lots of overtone.
Without going into different manufacturers &/or model designations, once you've got down to that single clear skin, the thinner it is the more overtone it will have, but the easier it is to tune out the ring. Thick single-skins will ring like a bell if you're not careful - which presumably is why people got into the pinstripes. Any idiot can tune them ;)
A secondary consideration is you can under- or over-tune them to get a huge variety of tones out of them.
A personal favourite of mine is to use thin skins on small drums, keeping the pitching "down a drum"...
lemme explain...
If you were to have 3 toms, 12, 14 & 18 with regular tuning, then instead you take 3 smaller, say 10, 13 & 16 & pitch them down to where you would have had the larger toms. Lots of smack, slap & depth, less definite 'note', little ring.
An additional note - for 'short' toms, this type of skin setup will hint at that 'rototom' sound in terms of 'slappiness' but with more depth because of the shell & optional 2nd head.


Why does $P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)P(\theta_1)$ hold?

Question: Suppose that in a Bayesian framework we have observed data $D$, using independent prior distributions on the parameters of the model, denoted by $\theta_1, \theta_2$. Then, the joint posterior distribution of $\theta_1, \theta_2$ can be written as:
$$
P(\theta_1, \theta_2 \mid D) \propto P(D \mid \theta_1, \theta_2)p(\theta_1)p(\theta_2)
$$
One way to obtain estimates from the posterior is use Gibbs Sampling. This sampler requires simulating from the full conditional distributions for both $\theta_1$, $\theta_2$. One relation for the full conditional that caught my eye in a book is the following:
$$
P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)p(\theta_1)
$$
I am wondering how exactly this proportional relation is derived. Would anyone have any ideas? It seems that there are extra assumptions here that are implicit.
Answer: The starting point is the proportionality symbol$$\propto$$as it takes different meanings in the few formulas in the question. In
$$
P(\theta_1, \theta_2 \mid D) \propto P(D \mid \theta_1,\theta_2)p(\theta_1)p(\theta_2)
$$
the function of $(\theta_1, \theta_2)$ on the l.h.s. is proportional to the function of $(\theta_1, \theta_2)$ on the r.h.s., meaning that they differ by a multiplicative constant $\kappa$. This is a constant w.r.t. the functions of $(\theta_1, \theta_2)$, meaning that $\kappa$ may depend on (and hence be a function of) other factors, like $D$:
$$\kappa=\kappa(D)$$
In fact, since $P(\theta_1, \theta_2 \mid D)$ is a probability density, this constant $\kappa$ is uniquely defined as the normalisation of the r.h.s. that turns it into a probability density, with total mass equal to one:
$$\kappa=\kappa(D)=1\Big/\int_\Theta P(D \mid \theta_1,\theta_2)p(\theta_1)p(\theta_2)\,\text{d}\lambda(\theta_1,\theta_2)$$
for the appropriate measure $\text{d}\lambda$.
In
$$P(\theta_1\mid D, \theta_2) \propto P(D \mid \theta_1, \theta_2)p(\theta_1)\qquad\qquad(1)$$
the function of $\theta_1$ on the l.h.s. is proportional to the function of $\theta_1$ on the r.h.s., meaning that $\kappa$ may depend on (and hence be a function of) other factors, like $D$ and now $\theta_2$:
$$\kappa=\kappa(D,\theta_2)$$
Note that
$$
P(\theta_1 \mid \theta_2, D) \propto P(D \mid \theta_1,\theta_2)p(\theta_1)p(\theta_2)
$$
is equally correct.
That the result (1) holds follows from the standard definition of the conditional density
\begin{align*}
P(\theta_1\mid D, \theta_2) &= \dfrac{P(D \mid \theta_1, \theta_2)p(\theta_1)p(\theta_2)}{\int_{\Theta_1} P(D \mid \theta_1, \theta_2)p(\theta_1)p(\theta_2)\,\text{d}\lambda_1(\theta_1)}\\
&= \dfrac{P(D \mid \theta_1, \theta_2)p(\theta_1)\times p(\theta_2)}{\left\{\int_{\Theta_1} P(D \mid \theta_1, \theta_2)p(\theta_1)\,\text{d}\lambda_1(\theta_1)\right\}\times p(\theta_2)}\\
&= \dfrac{P(D \mid \theta_1, \theta_2)p(\theta_1)}{\int_{\Theta_1} P(D \mid \theta_1, \theta_2)p(\theta_1)\,\text{d}\lambda_1(\theta_1)}\\
&= \dfrac{P(D \mid \theta_1, \theta_2)p(\theta_1)}{\kappa(D,\theta_2)}\\
&\propto P(D \mid \theta_1, \theta_2)p(\theta_1)
\end{align*}
Comment: Thank you, this is really clear and helped clear up some of my misconceptions!


Prove that $d(v_0)$ = $d(v_n) = 1$

Question: Given $P = \{v_0, v_1, \cdots,  v_n\} $ be  a longest path in tree $T$.  Show that the degree of $v_0$ and $v_n$ is 1.
My attempt:
I have drawn a tree and simulate some paths so I know which one is the longest. But, once I see the other paths,  I learn that the initial and terminal vertices also have the degree of 1. Every vertex in that same position must be 1. But,  why it doesn't go logic to the problem? Now,  what should I do?
Answer: Let's try to prove that there are exactly two vertices in $P$ such that those vertices have degree $1$. If we prove that, obviously, these vertices will be $v_0$ and $v_n$. And this will give us even more general result.
Suppose for a contradiction, there are more than two vertices such that degree of those vertices are $1$ in $P$. Then, by definition of path, all of those vertices must be either start or terminal vertices, which is a contradiction because a path in a tree has only one start and one terminal vertex (since it cannot be a circuit).
Now, suppose for a contradiction, there are less than two vertices with degree $1$ in the path. Since it is a tree, it cannot be zero because if it was zero, we would have a cycle. If the number of vertices with degree $1$ is one, then this is contradicting the assumption that $P$ is the longest path because the vertex with degree $1$ is $v_0$ and if there is no other vertex with degree $1$, then $d(v_n) > 1$ so it is not terminal at all. Therefore, there must be exactly two vertices with degree $1$ in longest path in tree and those are $v_0$ and $v_n$, obviously.
Comment: Is circuit and cycle the same thing?
Comment: No, they are slightly different things. Here: https://math.stackexchange.com/questions/655589/what-is-difference-between-cycle-path-and-circuit-in-graph-theory. But these definitions can be changed from book to book or even among some people. For example cycle definition here is not like the definition I learned. That's why if you are learning graph theory in university or high school, I suggest you to adapt to definitions as they are taught in your school or university.
Comment: Really thank you for your answer.  Regards
Comment: You're welcome, thank you :)
Answer: Short answer: If your path ends in a vertex connected to another edge, then you can add this edge
and you path is longer than before.
More technical answer:
We have the longest Path $P=v_1,...,v_n$.
Let's suppose: $d(v_n) > 1$.
Then there is a vertex $v_{n+1}$ and the edge $e = (v_n, v_{n+1})$
that is not part of the Path $P$.
Otherwise we would have a circle in the tree.
The Path $v_1,...,v_n,v_{n+1}$ is longer than $P$.
Contradiction!
Hence, $d(v_n) = 1$.
The same applies to $d(v_1)>1$.

Prove that $d(v_0)$ = $d(v_n) = 1$


Como fazer que ao clicar numa tecla um submit é enviado

Question: Bom é o seguinte, eu tenho um chat no meu site, o chat tem um campo de texto e um botão send que é para enviar. Ou seja a pessoa digita no texto e depois tem que clicar no send.
Eu gostaria de saber como faria para que ao clicar-se enter, o submit fosse enviado. Ou seja que não fosse preciso ir clicar no botao submit de enviar, mas que bastasse apenas apertar enter para enviar a mensagem.
Código do submit:

 <button class="btn btn-primary" type="submit" data-loading-text=" WAIT..." id="send_massage" style="margin-right:6px;">SEND</button>
Answer: Pode fazer assim: (fonte)
$("#id_of_textbox").keyup(function(event){
    if(event.keyCode == 13){
        $("#id_of_button").click();
    }
});

Comment: -1, se possível adicione à [fonte dessa resposta](http://stackoverflow.com/a/155263/4843158). Após adicionar a fonte eu retiro o -1.
Comment: @Randrade eu não tinha visto isso. Bem, o mesmo pra mim.
Comment: Fonte adicionado, tinha esquecido de colocar. Bom que resolveu seu problema.
Comment: Obrigado, resultou perfeitamente.
Comment: @Gonçalo Então, marca a resposta dele como certa, bruh!!!
Comment: Agora sim! @Gonçalo Marca como resposta, cara!
Answer: Na verdade você pode enviar o form por qualquer tecla.
Veja esse exemplo no JSFIDDLE que eu montei. Você pode colocar qualquer código de tecla.
Os códigos podem ser vistos aqui, basta digitar a tecla específica no formulário e o código aparece ao lado.
CÓDIGO

$('input[name="texto_exemplo"]').keypress(function (e) {
  if (e.which === 32) { /* 32 é a tecla de espaço, use 13 para [ENTER]*/
    if($('form[name="exemplo"]').submit()){
      alert("Formulário enviado!");
    }
    return false;
  }
});

<form action="" name="exemplo" method="POST">
  <input type="text" accesskey="s" name="texto_exemplo"/>
</form>
Comment: Eduardo, sua resposta está boa, mas se puder, poste pelo menos o código js aqui, pois se os links se quebrarem, sua resposta fica inválida.
Comment: Obrigado, @Randrade! Boas práticas são boas práticas!!!
Comment: Editei apenas para adequação do snnipets, nenhum erro específico. Mas faltou você falar que está enviando com espaço. :)
Comment: Agora sim, +1. Mas que pena que já existe uma resposta assim, e essa será fechada. Mas isso não inválida sua resposta.
Comment: O importante é que a gente documente bem as perguntas e as respostas no SOF em Português, assim, nossa comunidade cresce e vira referência.
Comment: Valeu mesmo! Bem, eu adicionei no comentário do código. Acho que estamos documentando bem a pergunta hehee
Answer: Bom, com javascript você pode utilizar o evento onkeypress() para realizar se o botão apertado foi o enter, se for você chama o submit, ficaria desta forma:

  <input type="text" id="txtSearch" onkeypress="return searchKeyPress(event);" />
<input type="button" id="btnSearch" Value="Search" onclick="doSomething();" />

Exemplo no JSFiddle.
Agora, caso queira utilizar jQuery, basta utilizar o evento keyup() para fazer a mesma verificação, desta forma:

<input type="text" id="txtSearch"  />
    <input type="button" id="btnSearch" Value="Search"/>

Exemlo no JSFiddle.
Fonte: SOen


Long run behavior of a absorbing markov chain

Question: $$A=\begin{pmatrix}1&0&0&0\\0&1&0&0\\0.2&0&0.6&0.2\\0&0.2&0.2&0.6\end{pmatrix}.$$
In the above matrix how do I calculate the probability that in the long run the system will be in 1 of the absorbing states?In particular say the probability of being in the 1st absorbing state in the long run?Is it 0.5?
Comment: Surely it depends on whether you starts in 1,2,3 or 4...
Answer: The probability the Markov chain ends up in one of the absorbing states is $1$.
Let $p_i$ be the probability of absorption in state 1 when starting in state $i$
Note $p_2=0$ $p_1=1$
$$p_3 = 0.2p_1+0.6p_3+0.2p_4$$
why?!
write down the equation for $p_4$, then solve.
Let $q_i$ be the probability absorption in state 2, note that $q_i+p_i=1$ (why?)


How is this matrix called (two diagonals)?

Question: I need to write an algorithm for solving this matrix but I wanted to first make a search online and that's why I need its name.

Comment: This might help: http://en.wikipedia.org/wiki/List_of_matrices
Answer: You can rearrange terms, to get the following system
$$
\left [ \begin{array}{ccccccc}
d_1 & a_{2n+1} \\
a_1 & d_{2n+1} \\
& & d_2 & a_{2n} \\
& & a_2 & d_{2n} \\
& & & & \ddots \\
& & & & & d_n & a_{n+2} \\
& & & & & a_n & d_{n+2} \\
& & & & & & & d_{n+1}
\end{array}\right ] \left [ \begin{array}{c}
x_1 \\ x_{2n+1} \\ x_2 \\ x_{2n} \\ \vdots \\ x_n \\ x_{n+2} \\ x_{n+1}
\end{array}\right ] = \left [ \begin{array}{c}
b_1 \\ b_{2n+1} \\ b_2 \\ b_{2n} \\ \vdots \\ b_n \\ b_{n+2} \\ b_{n+1}
\end{array}\right ]
$$
which is equivalent to solving $n$ $2\times 2$ linear systems and $1$ algebraic equation of $1^{st}$ order.
Comment: Would n = height/2 - 1? And is the height dn?
Comment: @HeyheyHeyhey sorry, I'm nor sure I'm following.
Comment: The number of n 2x2 matrices is dn?
Comment: No, number of matrices is $n$, as I specifically pointed out, so did you btw.


$E[(\beta_n-\beta)^2|\mathbf{X}]=\sigma^2(\mathbf{X}^T\mathbf{X})^{-1}$, what about $E[(\beta_n-\beta)^k|\mathbf{X}]$ for $k=3,4$?

Question: Ultimately my goal with this post is to find the orders of magnitude of $E[(\beta_n-\beta)^k]$ for $k\ge 3$ as $n \to \infty$ so that I can be sure that is safe to drop higher order terms in a Taylor expansion based approximation. As a first step towards $E[(\beta_n-\beta)^k]$ I consider $E[(\beta_n-\beta)^k|\mathbf{X}]$ in what follows.
My real interest is in the case of  multiple regression (including an intercept term) but I think it is easier to focus on a simpler case first. Suppose we have the simple intercept-free linear regression model
$$
Y_i = \beta X_i + \varepsilon_i, \quad \quad i=1,\dots,n,
$$
where $E[\varepsilon_i|\mathbf{X}] = 0$, $\text{Var}[\varepsilon_i^2|\mathbf{X}] = \sigma^2$, and $E[\varepsilon_i\varepsilon_j|\mathbf{X}] = 0$ for $i\neq j$.
It is known that
$\hat \beta = \beta + A\varepsilon$
where
$$
A = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'.
$$
In this simple setting the data matrix $\mathbf{X}$ is actually a vector:
$$
\mathbf{X} = [x_1,\dots,x_n]^T.
$$
So $A\varepsilon$ is simply a scalar in our simple model.
Now consider evaluating $E[(\hat \beta - \beta)^3|\mathbf{X}]$ or $E[(\hat \beta - \beta)^4|\mathbf{X}]$ using the same process (see p. 61, Econometric Analysis 8 ed. by Green or p.29, Econometrics by Hayashi) that is used for finding an expression for $E[(\hat \beta - \beta)^2|\mathbf{X}]$. That is
\begin{align}
E[(\hat \beta - \beta)^2|\mathbf{X}]
&=
E[(A\varepsilon)^2|\mathbf{X}] \\
& = A E[\varepsilon \varepsilon^T|\mathbf{X}]] A^T \\
& = A \sigma^2 A^T \quad \quad \quad \quad \quad \quad \quad \quad \text{(using assumption $E[\varepsilon \varepsilon^T|\mathbf{X}]] = \sigma^2$ )}\\
& = \sigma^2 (\mathbf{X}^T\mathbf{X})^{-1}.
\end{align}
Now lets apply this procedure to the higher moments:
\begin{align}
E[(\hat \beta - \beta)^3|\mathbf{X}]
&=
E[(A\varepsilon)^3|\mathbf{X}] \\
& = A E[\varepsilon (A \varepsilon) \varepsilon^T|\mathbf{X}]] A^T \\
\end{align}
and
\begin{align}
E[(\hat \beta - \beta)^4|\mathbf{X}]
&=
E[(A\varepsilon)^4|\mathbf{X}] \\
& = A E[\varepsilon (A \varepsilon)^2 \varepsilon^T|\mathbf{X}]] A^T \\
\end{align}
In both cases we get stuck with $A$'s inside the expectation. So is there any way to overcome this and obtain explicit expressions for $E[(\hat \beta - \beta)^3|\mathbf{X}]$ and $E[(\hat \beta - \beta)^4|\mathbf{X}]$?} This issue is overcome in the attempt below.
Further attempt:
We have
$$
A = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}' = ([x_1, \dots, x_n] [x_1, \dots, x_n]^T)^{-1} [x_1, \dots, x_n] = \frac{1}{||\mathbf{X}||^2}[x_1, \dots, x_n].
$$
Then
$$
Ae = \frac{1}{||\mathbf{X}||^2}[x_1, \dots, x_n] [\varepsilon_1, \dots, \varepsilon_n]^T = \frac{1}{||\mathbf{X}||^2} \sum_{i=1}^n x_i e_i.
$$
Next we have
$$
\begin{align}
E[(\hat \beta - \beta)^3|\mathbf{X}] &= E[(A\varepsilon)^3|\mathbf{X}] \\
&= \sum_{i=1}^n E\bigg[\frac{(x_i\varepsilon_i)^3}{||\mathbf{X}||^6}\bigg|\mathbf{X}\bigg] \\
\end{align}
$$
where all the other terms have been dropped by assuming $E[e_i e_j^2|\mathbf{X}] = 0$ for $i \neq j$.
Then
\begin{align}
E[(\hat \beta - \beta)^3|\mathbf{X}]
&= \frac{1}{||\mathbf{X}||^6} \sum_{i=1}^n x_i^3 E[\varepsilon_i^3|\mathbf{X}] \\
&= \mu_3 \frac{\sum_{i=1}^n x_i^3}{||\mathbf{X}||^6}.
\end{align}
Similarly for $k=4$, if we drop terms by setting conditions on the expectation of the various error terms we get
\begin{align}
E[(\hat \beta - \beta)^4|\mathbf{X}]
&= \mu_4 \frac{\sum_{i=1}^n x_i^4}{||\mathbf{X}||^8}.
\end{align}
So assuming these expressions are correct, the pattern for the moments is clear. I now need to get the order of magnitude with respect to $n$ of
$$
\begin{align}
E[(\hat \beta - \beta)^k] &= E[E[(\hat \beta - \beta)^k|\mathbf{X}]] \\
&= \mu_k E\bigg[\frac{\sum_{i=1}^n x_i^k}{||\mathbf{X}||^{2k}}\bigg] \\
&= \dots
\end{align}
$$
I am not sure how to evaluate this expectation and find its order of magnitude with respect to $n$ as $n\to \infty$.
Comment: That's a misunderstanding: the properties of the limit of a sequence do not need to pertain to individual members of the sequence.  It's easy to construct sequences of random variables that have no finite moments of any order (from the first on up) yet are asymptotically Normal.  All this is easier to analyze and understand when you strip away the regression trappings from the question and focus on the core issue of relating higher moments in a sequence of random variables to lower moments.
Comment: When the errors are iid Normal, that third moment is zero and the fourth moment can be computed from $X.$  But generally, as I have pointed out several times now, nothing can be said about those moments.  For instance, the third moment could be infinite for *all* $n.$
Comment: How are "$\beta_n$" and "$\hat\beta$" related?  What are you willing to assume about the error distribution?  (You must assume something quite specific, for otherwise almost nothing can be said about the higher moments of the estimator, any of which could be infinite or undefined.)
Comment: I fixed the typo and added the conditions on the error distribution at the start of the post. For the higher moments my post already specified conditions on the error so I left those unchanged.
Comment: Ultimately my motivation for this post is I am taking the expectation of a Taylor expansion of a well-behaved function $f(\hat \beta)$ around $\hat \beta = \beta$, and truncating at the second order: $E[f(\hat \beta)] = f(\beta) + \frac{1}{2}E[(\hat \beta - \beta)^2]f''(\beta) + \text{remainder}$. I need to be sure that it is safe to neglect the higher order moments as $n \to \infty$. While I know that $E[(\hat \beta - \beta)^2] = O(1/n)$ I have not been able to confirm the orders of the higher moments, in particular $E[(\hat \beta - \beta)^3]$.
Comment: The first step in finding the order of $E[(\hat \beta - \beta)^3]$ with respect to $n$ as $n \to \infty$ is finding the order of the conditional expectation $E[(\hat \beta - \beta)^3|\mathbf{X}]$, which is what I've attempted in my post.
Comment: I am considering the non-normal error case. If we assume the moments due to the error $\mu_k$ are finite, that leaves us with $E\bigg[\frac{\sum_{i=1}^n x_i^k}{||\mathbf{X}||^{2k}}\bigg]$. In the $k=2$ case this reduces to $E[(\mathbf{X}\mathbf{X})^{-1}]$ and a rate of of convergence in probability for $(\mathbf{X}^T\mathbf{X}/n)^{-1}$ can be [found](https://stats.stackexchange.com/questions/500431/rate-of-convergence-of-hat-q-xx-1-left-frac-mathbfxt-mathbfx) which can be converted into a rate of convergence in expectation.
Comment: So as an explicit order of magnitude can be found for the $k=2$ case I am hoping something similar can be possibly done for the $k=3,4$ cases.
Comment: We know $\hat \beta$ is asymptotically normal so I don't see how the third moment could ever be infinity .
Comment: That's very interesting, I will have to do some reading up on this phenomenon and the relationship between higher and lower moments in a sequence of random variables.
Answer: You have not assumed enough about $\epsilon_i$ to determine the answer. If you only want two moments of $\hat\beta$ is is enough to make assumptions about two joint moments of $\epsilon_i$, as you have done.  This is not enough for higher moments of $\epsilon$.

First, as @whuber has pointed out, you need to assume the higher moments of $\epsilon_i$ exist. There are distributions satisfying your assumptions that do not have a finite third moment, such as $\epsilon_i$ being iid $t_3$ variables. Existence of third moments might seem trivial, but it's not unusual for people to model long-tailed regression errors using $t$ distributions, and a $t_k$ distribution has only $k-1$ finite moments.

Second, you need control higher-order joint or conditional expectations if you're not willing to assume independence.  You have assumed $E[\epsilon_i\epsilon_j]=0$, which gives the same variance as if the $\epsilon_i$ were independent. It does not give the same third moments as if the $\epsilon_i$ were independent, and you can't conclude, eg, that
$$\left(\sum_i x_i\epsilon_i\right)^3= \sum_i (x_i\epsilon_i)^3$$
You might as well assume the $\epsilon$s are independent, since that's not much weaker than the assumptions you'll need about expectations of all higher-order products.

Similarly, you might as well assume $\epsilon_i$ are identically distributed, since you'll have to assume they have the same moments of all finite orders, and that's not usefully weaker.

So, if you assume $\epsilon_i$ are iid with finite moments of all orders, you can do the transformations you've done and we end up with the question of the size of
$$\mu_k \frac{\sum x_i^k}{\|X\|^{2k}}=\mu_k \frac{\|X\|_{k}^{k}}{\|X\|_2^{2k}}$$
For fixed $k$, and if the expectations of powers of $X$ exist (you need moment assumptons on $x$,too), this is of order $n^{-k}$ if $E[x_i]\neq 0$ (so that $\|X\|$ is proportional to $n$) and of order $n^{-k/2}$ if $E[x_i]=0$ (so that $\|X\|$ is proportional to $\sqrt{n}$)[this is where the omission of the intercept comes back to bite you].
The dependence on $k$, however, can be almost arbitrarily complicated, so you need more assumptions to say anything about the moments for large $(k, n)$.
Comment: A sum of $n$ independent things is of size $O_p(n)$ if they don't have mean zero (because even just the mean is that big), and a sum of $n$ independent mean-zero things is of size $O_p(\sqrt{n})$ because its variance is proportional to $n$
Comment: This is an excellent answer thanks, its great to finally confirm that my second-order Taylor expansion will be valid once these moments are controlled with appropriate assumptions. I'm still unsure about the evaluation of $E[||X||_k^k/||X||_2^{2k}]$, how are you finding that both $||X||_2$ and $||X||_k$ are proportional to $n$ when $E[x_i] \neq 0$, and $\sqrt{n}$ when $E[x_i] = 0$ so that ultimately we get $E[||X||_k^k/||X||_2^{2k}] = O(n^k/n^{2k}) = O(n^{-k})$?
Comment: ...for the $E[x_i] \neq 0$ case, and similarly we get $O(n^{-k/2})$ for the $E[x_i] = 0$ case?
Comment: For $||X||_2$ I have $||X||_2^2 = \sum_{i=1}^n x_i^2 \stackrel{}{\to} n E[x_i^2]$ which implies $||X||_2 \to (n E[x_i^2])^{1/2} = O(n^{1/2})$. For $||X||_k$ I have $||X||_k^k = \sum_{i=1}^n x_i^k \stackrel{}{\to} n E[x_i^k]$ which implies $||X||_k \to (n E[x_i^k])^{1/k} = O(n^{1/k})$. Then I get $E[||X||_k^k/||X||_2^{2k}] \to E[O(n^{1/k})^{k}/O(n^{1/2})^{2k}] = O(n/n^{k}) = O(n^{-k+1})$
Comment: So are you sure the norms are proportional to $n$ in the $E[x_i] \neq 0$ case? Should the final result be $O(n^{-k+1})$ instead of $O(n^{-k})$ or have I made a mistake? Actually, I don't see how $E[x_i] = 0$ or $E[x_i] \neq 0$ comes into play, I didn't use it in the above since we are dealing with $E[x_i^k]$ where $k > 1$.
Comment: Ok so in the $E[x_i] \neq 0$ case we have $||X||_{k}^{k}/||X||_{2}^{2k} = O_p(n)/O_p(n^k) = O_p(n^{-k+1})$. But we still have to take the expectation of that quantity, doing so we get $n^{-k+1}E[O_p(1)]$. So it remains to assert $E[O_p(1)] = O(1)$, but I'm not sure this is holds? Even if we assume uniform integrability for the sequence of random variables so that convergence in probability implies convergence in expecation, I'm not sure we can go from Big $O_p$ in probability to Big $O$ in expectation.
Comment: I actually just asked a related question [here](https://stats.stackexchange.com/questions/501113/can-we-go-from-x-n-mu-o-pn-1-to-ex-n-mu-on-1)
Comment: Yes but we are interested in the sum of $n$ independent things where these things are $x_i^k$, not $x_i$. So it seems the two cases of interest should be $E[x_i^k] = 0$ and $E[x_i^k] \neq 0$ instead of $E[x_i] = 0$ and $E[x_i] \neq 0$.
Comment: This post and the associated comments focusses on $O_p$ orders of magnitude which are for the random variables $(\hat \beta - \beta)^k$ whereas I am interested in the $O$ orders of magnitude of $E[(\hat \beta - \beta)^k]$. So while this post was very helpful I think the question is still not fully resolved and I am not certain it can be resolved using this $O_p$ approach as $O_p$ convergence doesn't automatically imply $O$ convergence in expectation (see posted linked in comment above).


nreco extract high quality thumbnail

Question: I use Nreco framwork. i call GetVideoThumbnail method, it extracts default resolution image
var ffMpeg = new NReco.VideoConverter.FFMpegConverter();
ffMpeg.GetVideoThumbnail(pathToVideoFile, "video_thumbnail.jpg");

How to custom option?
Answer: GetVideoThumbnail is a helper method that internally uses ConvertMedia to perform special conversion to the MJPEG output with only 1 frame.
You may use the following code snippet instead of calling "GetVideoThumbnail" method (and use any additional ffmpeg options you need):
Stream jpegOutputStream; // stream for thumbnail jpeg image output
var thumbSettings = new ConvertSettings() {
              VideoFrameRate = 1,   VideoFrameCount = 1, // extract exactly 1 frame
              Seek = 0, // frame position in seconds
              CustomOutputArgs = "" // any ffmpeg arguments that goes before output param
};
ffMpeg.ConvertMedia(pathToVideoFile, null, jpegOutputStream, "mjpeg", thumbSettings);

nreco extract high quality thumbnail


Proving that algebraic multiplicity and the geometric multiplicity of $\lambda$ are equal

Question: I'm having some trouble proving the following:

Let $f:V \to V$ be a linear isometry and $V$ a finite-dimensional euclidian vector space. Then, if $\lambda$ is a real eigenvalue of $f$, the algebraic multiplicity and the geometric multiplicity of $\lambda$ are equal.

I know that, because $f$ is an endomorphism:$$\text{geomtric multiplicity of } \lambda \leq \text{algebraic multiplicity of } \lambda$$
But I have no clue how to even approach this problem. How can I prove this?
Comment: The concept of isometry isn't defined on a general finite-dimensional real vector space.
Comment: V is an euclidian vector space@JoséCarlosSantos
Answer: Since $f$ is an isometry, $\lambda=\pm1$ for each eigenvalue $\lambda$. In fact, if $v$ is an eigenvector with eigenvalue $\lambda$,$$|\lambda|\|v\|=\|\lambda v\|=\|f(v)\|=\|v\|.$$
Suppose that the geometric multiplicity of $\lambda$ is smaller than the algebraic multiplicity. Then there is an eigenvector $v$ of $f$ with eigenvalue $\lambda$ (and we may assume without loss of generality that $\|v\|=1$) and there is a vector $w$ such that $f(w)=\lambda v+w$ (think about the Jordan normal form). Now, let $w'=w-\langle w,v\rangle v$. Then\begin{align}f\left(w'\right)&=f\bigl(w-\langle w,v\rangle v\bigr)\\&=f(w)-\langle w,v\rangle f(v)\\&=\lambda w+v-\lambda\langle w,v\rangle v\\&=\lambda w'+v.\end{align}But $\left\langle v,w'\right\rangle=0$. Now, note that\begin{align}\left\|f\left(w'\right)\right\|&=\left\|\lambda w'+v\right\|\\&=\sqrt{\lambda^2\|w'\|^2+\|v\|^2}\\&=\sqrt{\|w'\|^2+1}\text{ (since $|\lambda|=\|v\|=1$)}\\&>\|w'\|,\end{align}which is impossible, since $f$ is an isometry.
Comment: How can we conclude that $\exists w: f(w) = \lambda v + w$ from the Jordan Normal Form?
Comment: Form those blocks of the form$$\begin{bmatrix}\lambda&1&0&\ldots&0\\0&\lambda&1&\ldots&0\\\vdots&\vdots&\vdots&\ddots&\vdots\\0&0&0&\ldots&\lambda\end{bmatrix}.$$


What is the Likelihood function and MLE of Binomial distribution?

Question: I searched online, so many people mix up MLE of binomial and Bernoulli distribution.
They are saying:
"If $X$ is $Binomial(N,\theta)$ then MLE is $\hat{\theta} = X/N$."
And I don't agree with that, that is just MLE for Bernoulli in N trials, not for Binomial.
But I can't find correct Likelihood function and MLE of Binomial distribution online.
So I try to derive it myself, and seek for confirmation here.

For Bernoulli, I know that::
its Likelihood function is
$$\prod\limits_{i = 1}^n {{p_X}{{\left( {{x_i}} \right)}_{Ber\left( \theta  \right)}}}  = \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{n - \sum\limits_{i = 1}^n {{x_i}} }}} \right)$$
its MLE is
$${{\hat \theta }_{Ber\left( \theta  \right)}} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{n} = \bar x$$
${{p_X}\left( {{x_i}} \right)}$ is the pdf (or pmf)

For Binomial, I tried following::
its Likelihood function is
$$\left\{ \begin{array}{l}L\left( {\theta |{\bf{x}}} \right) = \prod\limits_{i = 1}^n {{p_X}{{\left( {{x_i}} \right)}_{Bin\left( {N,\theta } \right)}}}  = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right) \cdot {\theta ^{{x_i}}}{{\left( {1 - \theta } \right)}^{N - {x_i}}}}  \\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)}  \cdot \left( {\prod\limits_{i = 1}^n {{\theta ^{{x_i}}}{{\left( {1 - \theta } \right)}^{N - {x_i}}}} } \right)\\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)}  \cdot \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{\sum\limits_{i = 1}^n {\left( {N - {x_i}} \right)} }}} \right)\\ = \prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)}  \cdot \left( {{\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{{\left( {1 - \theta } \right)}^{nN - \sum\limits_{i = 1}^n {{x_i}} }}} \right)\\\left[ {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right){\text{ is just a constant when }}{x_i}{\text{ is given}}} \right.\\ \propto {\theta ^{\sum\limits_{i = 1}^n {{x_i}} }}{\left( {1 - \theta } \right)^{nN - \sum\limits_{i = 1}^n {{x_i}} }}\end{array} \right.$$
(I don't think we can have a general formula for the constant, so I drop it and just use the proportion, if you know please tell me.)
its MLE is
$$\left\{ \begin{array}{l}\ln L = \ln \left( {\prod\limits_{i = 1}^n {\left( {\begin{array}{*{20}{c}}N\\{{x_i}}\end{array}} \right)} } \right) + \sum\limits_{i = 1}^n {\left( {{x_i}} \right)}  \cdot \ln \left( \theta  \right) + \left( {nN - \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \ln \left( {1 - \theta } \right)\\\frac{{d\left( {\ln L} \right)}}{{d\theta }} = 0 + \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{\theta } - \frac{{nN - \sum\limits_{i = 1}^n {{x_i}} }}{{1 - \theta }}\\\frac{{d\left( {\ln L} \right)}}{{d\hat \theta }} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{{\hat \theta }} - \frac{{nN - \sum\limits_{i = 1}^n {{x_i}} }}{{1 - \hat \theta }} = 0\\\left( {1 - \hat \theta } \right) \cdot \sum\limits_{i = 1}^n {\left( {{x_i}} \right)}  = \left( {nN - \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \hat \theta \\\sum\limits_{i = 1}^n {\left( {{x_i}} \right)}  = \left( {nN - \sum\limits_{i = 1}^n {{x_i}}  + \sum\limits_{i = 1}^n {{x_i}} } \right) \cdot \hat \theta \\{{\hat \theta }_{Bin\left( {N,\theta } \right)}} = \frac{{\sum\limits_{i = 1}^n {\left( {{x_i}} \right)} }}{{nN}} = \frac{{\bar x}}{N}\end{array} \right.$$
Am I correct?

Also, n is the number of samples (trials), N is the number of times you flip it in 1 trial. ie::
in Bernoulli, you flip the coin n trials, you flip it 1 time each trial.
in Binomial, you flip the coin n trials, you flip it N times each trial.
(I guess this is why so many people mix these two up when calculating the Likelihood function)
Comment: For what I mean by mixing up likelihood function of Bernoulli & Binomial, you can look at the following links::
https://stats.stackexchange.com/questions/384296/deriving-likelihood-function-of-binomial-distribution-confusion-over-exponents
https://stats.stackexchange.com/questions/181035/how-to-derive-the-likelihood-function-for-binomial-distribution-for-parameter-es?noredirect=1&lq=1
Answer: These two models are statistically equivalent:
$$
X_1,\dots,X_n \sim \text{Ber}(\theta), \quad \text{i.i.d.}
$$
and
$$
T \sim \text{Bin}(n, \theta).
$$
the latter being the reduction of the former by sufficiency.
If you consider the following problem:
$$
Y_1,\dots, Y_n \sim \text{Bin}(N,\theta), \quad \text{i.i.d.}
$$
This is a different problem than either of the two above, a different model, not equivalent to the previous ones statistically. (This third model is equivalent to a version of the two above with $n$ replaced with $nN$.)
Saying "people mix up MLE of binomial and Bernoulli distribution." is itself a mix-up. There is no MLE of binomial distribution. Similarly, there is no MLE of a Bernoulli distribution. You have to specify a "model" first. Then, you can ask about the MLE. There many different models involving Bernoulli distributions. There are also many different models involving Binomial distributions. Once you fix a model, you can talk about the likelihood, etc.
Comment: I do think the model is default to be a set of n samples with a distribution (with parameters), instead of 1 sample, when we are talking about Likelihood functions. ie: $X \sim {\text{Distribution}}\left( \theta  \right)\& \left\{ {{x_1}, \cdots ,{x_n}} \right\}{\text{ are }}n{\text{ samples from }}X$

Shouldn't it?
Comment: It all depends on you and the problem you want to set up. The model can be whatever you want. A single Bernoulli observation $X \sim \text{Ber}(\theta)$ is a perfectly fine model.
Answer: Graphical comment: Suppose you have $X\sim\mathsf{Binom}(N, p),$ with $N=10$ and $x = 6$ successes.
p=seq(0,1, by=.01)
like = p^6*(1-p)^4
mle = p[like==max(like)];  mle
[1] 0.6

plot(p, like, type="l")
  abline(h=0, col="green2")
  abline(v = mle, col="red", lwd=2)

Suppose you have $X\sim\mathsf{Binom}(N, p),$ with $N=100$ and $x = 60$ successes.
p=seq(0,1, by=.01)
like = p^60*(1-p)^40
mle = p[like==max(like)];  mle
[1] 0.6
plot(p, like, type="l")
  abline(h=0, col="green2")
  abline(v = mle, col="red", lwd=2)

Notice that the curvature of the likelihood function at its maximum is much
tighter than for $N = 10.$
Comment: My point is, if you are calculating the MLE of Binomial distribution, in general, you should use eg:$\left\{ {{x_1} = 6,{x_2} = 7,{x_3} = 7, \cdots ,{x_n} = 5} \right\}$ as your samples each with 10 flips, instead of only using 1 sample {x=6} with 10 flips.
Or, I have to view it as 10 samples for a Bernoulli distribution instead of a Binomial distribution.
Because only using 1 sample for calculating a MLE of a distribution is generally not good.
Not sure if you get what I mean.
Comment: I guess my point is that you're making it more complicated than it needs to be. Only the sufficient statistic matters.
Comment: It might look complicated, but I think it add more clarification. I was trying to make a clarification that there is difference between MLE of Binomial and Bernoulli distribution. If people are referring to MLE of Bernoulli, then they should say its MLE of Bernoulli instead of saying that is MLE of Binomial (with n=1 sample).


Checking for existence for a field on an an entity_metadata_wrapper

Question: I am iterating through a field collection, and wrapping the field collection items with an entity_metadata_wrapper. I would like to check for the existence of field before calling its value method (which results in an error on fields that do not have a value), but I cannot find a way to do so.
$field_collection_item = field_collection_item_load($id);
$item_wrapper = entity_metadata_wrapper('field_collection_item', $field_collection_item);

// this results in an error if the field_contrib_headshot field is empty
$headshot = $item_wrapper->field_contributor->field_contrib_headshot->value();

My current workaround is to use field_get_items to see if the field is empty, but i'd love to be able to do this through the metadata wrapper if possible.
Answer: Simply call the PHP function isset():
$headshot = array();
if (isset($item_wrapper->field_contributor->field_contrib_headshot)) {
  $headshot = $item_wrapper->field_contributor->field_contrib_headshot->value();
}

EntityStructureWrapper implements the __isset() function according the principe of Overloading.
Comment: Yes, this is always how I've done it, less tied into the internals and easier to read IMO. Upvoted!
Answer: Any time there is an entity reference or field collection, isset() has never worked for me. What does seem to work any time we have an entity reference is doing:
if($wrapped_entity->entity_reference_field->getIdentifier()) {
  // This code only fires if there is an entity reference or field collection set.
}

Comment: This answer and the following post worked for me: http://dropbucket.org/node/1201
Answer: It looks like you have an entity_reference somewhere in there due to the method chaining. But, look at the __isset() method for EntityStructureWrapper.
Check like:
$has_headshot = $item_wrapper->field_contributor->__isset('field_contrib_headshot');

and then use an IF block to do your logic ...
EDIT:
$has_headshot is now valid check desired.
Comment: I'm not sure what you mean by "it looks like you have an entity reference somewhere", if you could explain that a little more i'd appreciate it. ``__isset`` works great on the field collection, although I need to add the full field name: ``field_contrib_headshot`` instead of ``contrib_headshot``
Comment: fields are values; a string, a number, whatever. `field_contributor` is referencing another field `field_contrib_headshot` ... you have nesting in there somehow. I'm assuming you're using entity_reference or something else like field_group to nest these things ... thats all I meant.
Comment: what was the __isset() piece of code that worked for you ?
Comment: It was: ``$item_wrapper->field_contributor->__isset('field_contrib_headshot')``
Comment: It feels weird to me to call the __isset() directly, why not:

`$has_headshot = isset($item_wrapper->field_contributor->field_contrib_headshot);`
Comment: did you read the code in the documentation i linked? http://drupalcontrib.org/api/drupal/contributions!entity!includes!entity.wrapper.inc/function/EntityStructureWrapper%3a%3a__isset/7  that code does call isset(), but if the Object is not fully loaded isset() will not work as expected. The __isset() magic method lazy loads the entities properties if they aren't loaded by the system yet to safely perform this check.
Answer: $wrapper = entity_metadata_wrapper('node', 123);
if($wrapper->__isset('field_middle_name')) {
    // Do something awesome with the middle name.
} else {
    // Don't do anything awesome, they don't have a middle name.
}

Entity reference fields and field collections
// also check if there is an identifier, __isset alone is not enough!
if ($wrapper->__isset('field_project_number') && $wrapper->field_project_number->getIdentifier()) {
    $number =  $wrapper->field_project_number->field_project_number_complete->value();
    return $number;
}

Copied and pasted straight from http://dropbucket.org/node/1201 but seems to be a better example than any other answer so far...
Answer: For EntityMetadataWrapper:
If you've got blocks of code running that shouldn't or if you've come across PHP errors look at some of the examples below. This example uses the property nid.
MAY ALL ERROR
if ($wrapper->__isset('nid')) {
  $var = $wrapper->nid->value();
}
else {
  // Do something it's FALSE;
}

OR
if ($wrapper->__isset('nid')) {
  $var = $wrapper->nid->getIdentifier();
}
else {
  // Do something it's FALSE;
}

OR
if ($wrapper->nid->value()) {
  // Do something it's TRUE.
}
else {
  // Do something it's FALSE;
}

You may find that using isset like so might evaluate to true even if the nid is not present. The ->getIdentifier(), or ->value(), or ->raw() may throw exceptions.
LIKELY ALWAYS TRUE
if (isset($wrapper->nid)) {
  // Do something it must be TRUE....Ah Hem wait.. this runs every time.
}

USE THIS INSTEAD
try {
  $var = $wrapper->nid->raw();
}
catch (EntityMetadataWrapperException $e) {
  $var = FALSE;
}

Answer: I've found that isset() cannot be used on the result of a function call. Since a Boolean is returned with the following statement, I was able to verify that the wrapped element, in fact, had a value.
if ( $wrapper->field_media_alternate[0]->value() ) {
  //...valid logic...
} else {
  //...not valid logic...
}

Answer: People often get this wrong. When you call isset on an entity metadata wrapper, you're checking whether the entity bundle supports that property. It has nothing whatsoever to do with the actual value of the field.
There isn't really an independent method for checking whether a value is set. You just have to evaluate value() or, preferably, raw(). (You can also call count() if it's a multi-value field, but raw() will return an empty array so it's not actually required.)
$w = entity_metadata_wrapper('node', $nid);
//Start by checking whether the node bundle supports field_entityref_myfield.
if (isset($w->field_entityref_myfield)) {
  //If we called value(), then that would cause the referenced entity
  //to be loaded, whereas raw() just gives us the entity_id if the value
  //is set, and Null if it isn't.
  if ($w->field_entityref_myfield->raw()) {
    //Do some things based on the entity-reference being set.
  } else {
    //Do some other things based on the entity-reference NOT being set.
  }
}

To reiterate, raw() is the value you should use to check whether a field exists. It's reliable and computationally cheap.
Answer: Here is my personal helper function to get a value from a potentialy not set chain of entity references:
function _get_wrapped_value_ifset($wentity, $field_chain){
  $root = $wentity;
  try{
    foreach ( $field_chain as $field_name ) {
      $root = $root->{$field_name};
    }
    return $root->value();
  }
  catch (EntityMetadataWrapperException $e){
    return null;
  }
}

$wentity is the wrapped entity ,
$field_chain is an array of field_names like so:
[
  'field_reference_from_the_wentity',
  'field_wanted_field_from_the_referenced_entity'
]

you may also do:
[
  'field_reference_from_the_wentity',
  'field_reference_from_the_referenced_entity',
  ...
  'field_wanted_field_from_the_N_referenced_entity'
]

it'll work !


Linux VPN Client with SecurID/RSA Tokens?

Question: I'm setting up a VPN that needs to have clients on Linux, Windows, and Mac.
I was planning to sue a PPTP client, with RADIUS to the SecurID RSA install, but I can't find any way to set up a linux client that has PPTP/SecurID support.
What have other people done in this situation? OpenVPN? Is there a guide to setting up SecurID tokens on a OpenVPN client?
Cisco VPNs aren't really a solution here, since I need to implement it in software on the server-side, but I could do a linux-based IPSEC VPN.
Any suggestions would be appreciated.
Answer: The SecurID tokens aren't supported because the validation software is non-free, as far as I know.  I've had a lot of fun with the yubikey hardware OTP generator for better-than-username-and-password authentication via PAM.  The yubikeys are also noticeably cheaper than SecurID tokens, and don't seem to have a limited lifespan.
Specifically, I've setup ssh using the yubikey for authentication, which opens up the possibility of using ssh-based VPNs.  My writeup's at http://www.teaparty.net/technotes/yubikey.html if it's of any use to you.  Everything involved is GPLed or better.
I've also seen people using PAM-based authentication steps with OpenVPN, which opens the possibility of getting OpenVPN to work with the yubikey.  The guys at Securix Live say they're working on a fully two-factor PAM module for the yubikey, and while I haven't been able to get it to work yet, that would give you the final piece of what you asked for.
If you do get OpenVPN working with a yubikey, do let us know - and write about it!


What sort of GLM should I use if my response variable is colony size?

Question: I want to model mean colony size as a function of predator density. Within each treatment level the smaller colonies (3-5 individuals) are a lot more common than the large ones (up to 600 individuals). Which error distribution should I use?
Comment: Could you describe distribution of colonies sizes in greater detail? Is it symmetric or skewed etc.?
Comment: most of the treatment replicates look similar to this graph I found on google http://www.statit.com/images/skewed_histogram.gif
Answer: There's not enough information to offer a definitive answer.
Colony size would be a count, I gather (rather than say an area, or a mass).
It's better not to think of them as representing an error term but rather the conditional distribution of the response.
Some possible models include negative binomial or log series distribution -- these are two distributions in the exponential-family which could be suitable for such counts, though they're not the only possibilities (some care must betaken with the zero-case; for example the negative binomial used in GLMs starts from 0, but colony sizes are presumably bounded below by 1).
In some cases you may be able to get some use from the Poisson but I expect that typically it won't be nearly skewed enough.
(The log series distribution may not be implemented in most software.)
You may need to consider more complex models that straight exponential family. You might need to consider truncated or shifted distributions, or if zeroes are possible, perhaps zero-inflated or hurdle models; you may also need mixed effect GLMs for example.
Comment: Its a count, sort of, but it is not really a sum of independent contributions, so the usual arguments leading to Poisson regression do not apply.
Comment: Thanks for your help. I've found that transforming the data like this: lm(sqrt(log(y))~x, ) makes it fit the model better but I wrote the code for the graph with CIs based on an untransformed model:
qplot(x, y, data= data, geom= c("point", "smooth"), method= "lm", )+theme_bw()
Any ideas of how I could change it so the new model is incorporated? I could put the method as glm with a log identity in but the sqrt makes the model fit much better.
Answer: About transformation, see :
O'Hara, Robert B., and D. Johan Kotze. 2010. « Do Not Log-Transform Count Data ». Methods in Ecology and Evolution 1 (2): 118‑22. doi:10.1111/j.2041-210X.2010.00021.x.
Summary :

1. Ecological count data (e.g. number of individuals or species) are often log-transformed to satisfy parametric test assumptions.
2. Apart from the fact that generalized linear models are better suited in dealing with count data, a log-transformation of counts has the additional quandary in how to deal with zero observations. With just one zero observation (if this observation represents a sampling unit), the whole data set needs to be fudged by adding a value (usually 1) before transformation.
3. Simulating data from a negative binomial distribution, we compared the outcome of fitting models that were transformed in various ways (log, square root) with results from fitting models using quasi-Poisson and negative binomial models to untransformed count data.
4. We found that the transformations performed poorly, except when the dispersion was small and the mean counts were large. The quasi-Poisson and negative binomial models consistently performed well, with little bias.
5. We recommend that count data should not be analysed by log-transforming it, but instead models based on Poisson and negative binomial distributions should be used.
Comment: +1 If this is you summarizing your own paper - hence the "we" in 4 and 5 - (then hey, hi and welcome ...) you should probably indicate that you're summarizing your own work, not quoting the paper without attribution (which it sort of looks like). [If instead that summary is a direct quote, you could indicate it by putting `>` at the start of each quoted paragraph, but the paper doesn't seem to have exactly those items, so I don't think it's that]
Answer: This is a partial answer only, but the graphical content makes a comment a poor alternative. In a comment, the OP talks about using sqrt(log()) as a transformation.
I'd advise against that on the general grounds that it is a very unusual and idiosyncratic transformation, so you will face puzzlement at all levels up to supervisors, examiners and paper reviewers. (Translate to the terminology of your own education and research set-up.)
I'd also advise that it really stretches out low colony sizes. Here is a plot for sizes 1(1)600.

Notice how on this transformed scale the interval from size 1 to size 5 is more than half the entire range from size 1 to size 600.
Implication: this transformation will create outliers for small colony sizes.  The better fit observed could be an artefact of over-transforming.
I wouldn't go further than a log link. At least that is standard and easier to think about.


How do I prove that each number is in a pigeonhole? (Edit: figured it out, thanks!)

Question: Consider a set of $n+1$ positive integers, each less than or equal to $2n$. Show there must always exist a pair of integers in the set, one dividing the other.
We use pigeonhole principle. For $1 \leq k \leq n$, let the $$k\text{th pigeonhole} = \{(2k-1)*2^t \mid t \text{ is a nonnegative integer, } 1 \leq (2k-1)*2^t \leq 2n\}$$ Since there are n "pigeonholes" and we must choose $n+1$ integers, there must be two integers, $i=(2k-1)*2^t$ and $j=(2k-1)*2^u$ such that $i<j$, that are in the same pigeonhole. Thus, $i | j$ since $j=i*2^{u-t}$, where $u-t$ is a positive integer.
(Edit: I've been told to use unique prime factorization, but I'm having trouble. On a gap year self-studying so any help would be great!)
(Edit 2: Essentially, I've placed numbers 1..2n into n pigeonholes, but do I prove that each number 1..2n is IN a pigeonhole? In other words, how do I prove that each number 1..2n can be expressed in the form (2k-1)*2^t. I'm not sure if I need to prove this but regardless, I'm curious.)
Comment: wait a second ... hope this is right, but cant you express any integer as (2^e)*s, where s is odd? thus by construction, my pigeonholes will contain all values 1..2n
Comment: Hi, so my question was essentially asking how to prove that every integer is of the form  $(2−1)2^p$.
Comment: It's not clear what your question is. Are you asking how to prove that each number is in the form $(2k-1)*2t$ for some $k,t$?
Comment: Hello! I'm asking how to prove that each number from 1...2n is in a pigeonhole. I first approached this problem by example (e.g. let n = 5) and realized that each number from 1..2n is in a pigeonhole, but I'm not sure how to prove it for an arbitrary n
Comment: edited my original post
Comment: What properties would a number need to have to _not_ be in any pigeonhole?
Comment: @It'sNotALie. the number must not be odd because if t = 0, then (2k-1)*2^0 = 2k-1 for k 1...n spits out all the odds from 1..2n-1. I'm currently thinking about the evens
Comment: Yes, you can. Every integer is of the form $(2k-1)2^p$. So your proof is finished.
Comment: But I understand now
Comment: Yep, you got it^^
Comment: @It'sNotALie. Thank you!!
Comment: @beginner Welcome to Math SE. Note a specific case of your question, where $n = 50$, is asked at [Pigeonhole Principle - numbers between $1$ and $100$](https://math.stackexchange.com/q/538129/602049), with its answer also being applicable to your case.
Answer: Factor out the largest number of twos that you can from $n$. If you factor out $t$ twos, then you've factored out $t^2$. Whatever's left is odd (if it were even, you could factor out another two), so it's in the form $2k+1$. For instance, take $56$. Is it odd? No, so divide by two. We now have $28$. Is that odd? No, so divided by two again. We now have $14$, which isn't odd. We divide by two again and get $7$, which is odd, so we can stop. Since $7$ is odd, we can express it as $2*3+1$. We divided by two three times, so we factored out an eight. $56 = (2*3+1)*2^3=7*8$
Another way of doing it: write $n$ in binary. Underline the rightmost 1. Let $t$ be the number of 0s to the right of that one, and let $k$ be the number that the digits to the left of that 1 represent (note that $t$ and $k$ can be zero). Then $n = (2k+1)*2^t$. Example: 56 in binary is 111000. Underline the rightmost 1:
$11\underline 1000$
There are three 0s to the right of the underline, so $t=3$. We take the digits to the left of the underline, which are $11$. That's $3$ in binary, so $k=3$.
$56 = (2k+1)*2^t = (2*3+1)*2^3 = 7*8$
The reason this works is that just as multiplying things by ten in base ten appends a 0, multiplying things in base two by two appends a 0. So when we multiply $11$ by two, we get $110$. Then when we add $1$, we get $111$, which gets us the original digits back. But then we also have to append some 0s, and doing this means multiplying by two. There are $t$ zeros, so we're multiply by two $t$ times, which is the same as multiplying by $2^t$.


Balancing sound of Acoustic and Electric Guitars

Question: My band has two guitar players. One often plays acoustic while the other one plays an electric guitar. There has always been a problem balancing the volume and the frequences. While both guitars play on a clean sound it sounds fine but when electric guitar changes to distorted, overdriven or crunched sound then even at a low volume the acoustic guitar is almost unheard. Is it a common problem or particulary our local one. Any solution?
By the way we tried playing bright tones on acoustic and tried EQing the distorted sound on the electric guitar.
Also, when a lead guitar is playing a solo part the second guitar is hardly heard too.
Both the acoustic and thr electric guitar go through an amplifier and into a mixer then line out to 4 speakers in corners of the room, so we have a quad sound as we are in the middle of tge room.
The acoustic guitar has bright steel strings and is mostly playing either chords or some arps. The electric guitar has very thin strings and mostly is a fender squire strat' and mistly plays either solo or power chords, sometimes rhythm on clean and some effects like tremolo, phaser and etc.
Sometimes there is need for a third and fourth guitar and i wonder how to balance that then.
Comment: Having four speakers won't necessarily produce 'quad sound'. If you use two stereo amps it might, but I guess you have the same sound coming out of each corner of the room. Why is this preferable to the more common two speaker p.a. set up?
Answer: I think there are different way to deal with this.

Maybe don't. I think it's totally okay to have the acoustic guitar
fade back when the electric one kick in with distortion or for a
solo.
Another idea would be to set a compressor of the electric guitar to control the volume and the dynamics.
You can also play with pan, it's common to pan one guitar on the left
and the other on the right. In your setup you could output one guitar
on two speakers and the other one the other two speakers.
You said you've tried EQing the guitar, maybe cut a little bit of the
bottom end of the electric guitar (between 200Hz and 500Hz) and add a
little bit of 2.5k - 3.5k on the acoustic to get a little bit more presence.
Comment: We tried adding treble to the acoustic guitar.. It sounds like a xylophone, only the highs are heard. Panning is a good idea. I remember doing it with several synthesizer tracks to make more space. Unfortunately we already have a compressor in the pedal-board and it doesn't help much. The dynamics are flat but the electric guitar is still screening the acoustic one even at a quite low volume. I guess it will still be so... whatever we do. I wonder how people manage 4 or more guitars all at once.
Answer: Often electric guitarists cannot play a distorted sound without cranking up the volume. Sometimes it's the limitations of the amp or pedals, but often it's the player himself. Can't control the volume! If your acoustic player is at the same volume for both scenarios, then that could be the problem.
As already stated, panning the separate tracks/guitars will help in the room, and even using one track of four for each. If each guitar is playing in the same pitch range, then they won't stand out against each other, but will blend into one.
There's also the Freddie Green scenario. He was guitarist playing rhythm in big bands (Count Basie). It was said that you couldn't really hear his playing, but if he stopped, you certainly knew he was playing before.
Answer: This is a musical problem rather than a technical one.   Guitarists engage overdrive when they WANT to dominate.  That's fine, but don't do it ALL the time!  Not if there's an acoustic guitar in the band and you want to hear it!  Everyone needs to listen, everyone has to be aware that another player may have something interesting to play, and therefore make space for it.   Like turn down or - horror - even stop playing for a bit!
Answer: I think the problem or behaviour of Electic stringed musical instruments is loudness with excitement. Once the guitar gets louder than the acoustic it becomes distorted Sound.  Compression would be the ideal action or Volume control from the
Comment: ...from the *what*, pray tell!!
Answer: My impression of the advice offered so far, is that compression on the electric guitar may solve the problem. However, I have found through experimentation and practice that a compressor used on an acoustic can highlight the more fragile aspects of it's sound, which when amplified can better compete with an electric guitar. I advise plenty of trial and experimentation before trying it in public, because compressors used incorrectly can create serious feedback headaches.


To Do Application involving React Native

Question: Main objective is creating a todo app with a FlatList and CheckBox. CheckBox must have a line pass through once the object has been clicked or pressed. I need help putting in the checkbox within the  FlatList and also the action of the checkbox as well with the line passing through once clicked.
import React, { useEffect, useState } from 'react';

import { StyleSheet, View, Text, Alert, Button } from 'react-native';

import { CheckBox } from 'react-native-elements';

export default function App() {

const [car, setCar] = useState(false);

const [tree, setTree] = useState(false);

const [boxing, setBoxing] = useState(false);

const tasks = []

const click = () => {

if (car === true) {

tasks.push("car")
}

if (tree === true) {

tasks.push("tree")

}

if (boxing === true)
{

tasks.push("boxing")

 }

Alert.alert("Tasks have been completed")

}

return <View style={StyleSheet.container}>

<CheckBox title={<Text style={{ textDecorationLine: "line-through" }}>"Change car oil"</Text>} checked={car} onPress={() => setCar(!car)} />
<CheckBox title={<Text style={{ textDecorationLine: "line-through" }}>"Cut tree"</Text>} checked={tree} onPress={() => setTree(!tree)} />
<CheckBox title={<Text style={{ textDecorationLine: "line-through" }}>"Go boxing"</Text>} checked={boxing} onPress={() => setBoxing(!boxing)} />

}
Answer: Have you looked at Flatlist's docs? I would encourage you to try to use it before asking how to use it. We can take a look at your code if it didn't work, no problem.
About the code you posted, have you thought about using only one state? It could be an array like this:
const [tasks, setTasks] = useState([]);

inside you can set the strings of the activated checkboxes, something like:
const check = (checkedTask) => {
    setTasks(prev => {
        if (prev.find(task => task === checkedTask) {
            return prev.filter(task => task !== checkedTask)
        }
    return [...prev, checkedTask]
  })
}