Am I using the wrong statistical test?

Question: I am using $\chi^2$ to analyze a set of data gathered over 4 time periods. The two variables are independent--each 'event' is categorized as 'good' or 'bad'. We want to prove that the intervention caused an increase in the good events and decrease in the 'bad' events, or at least that the increase in good events outpaces the bad events.
Here is a sample of what the data might look like:
<code>       good    bad
m1     1035    2278
m2     1152    2643
m3     1189    2917
m4     1125    2974
</code>
A $\chi^2$ test returns 15 which would have me believe the result is significant, but the change doesn't look very significant at all. In fact, I don't think there is a case to be made that there wasn't any improvement at all. I think that this may be due to the fact that my $N$ varies in each measurement (though I thought $\chi^2$ controlled for that).
It also seems strange to me that if I simply move the decimal over two places (thereby changing my N but not the proportion of good results to bad) then I get $\chi^2=.15$ which is closer to what I would expect.
I know I need to change my approach drastically. How can I make this comparison? Should I be using two independent student's t-tests? Or maybe just use the correlation coefficient for each column and the coefficient of determination?
Thanks in advance for your help and please let me know if anything is unclear.
EDIT:
Here is a thought: should I set up benchmarks for expected improvement, something like 3-5% increase (proportional to $N$) in each measure? This way my expected values in my $\chi^2$ might more accurately reflect what I am trying to show.
EDIT for clarification:
the % of good events compared to bad over the measures:
31%, 30%, 28%, 27%
Clearly there has not been improvement with regard to the proportion of good to bad so what test should I be using to demonstrate this?
Comment: Actually, if the samples are independent, that's a highly significant difference.  Consider m1 and m4: $p_1=0.312$ and $p_4=0.274$, but with those sample sizes, the std. error of $p_1$ is only about 0.008 and of $p_4$ about 0.007.  Consequently the two are roughly 3.6 std. errors (of the difference) apart.
Comment: How exactly is the "intervention" reflected in the data?  How can one identify "increases" in good and bad events from a table like the one presented here?
Comment: Thanks for the replies.

The workers that produce the data are rewarded based on the number of 'good' events they produce. They were also given a tool that helps them produce more good events.

I still can't see why, when I have about the same mean during the entirety of the study, my test claims the improvement is significant.
Answer: I agree with @jbowman, taking your data at face value, you have a highly significant effect.  This might be a good time to review what statistical significance is.  The p-value of a statistical test gives the probability of finding data as extreme or more extreme than your data if the null hypothesis is true.  Generally, people will believe that the null hypothesis is not true if the p-value is less than some threshold, $\alpha$.  In practice, $\alpha$ is almost always arbitrarily set at .05.  It is exceedingly unfortunate that this idea had been given the label 'significant'.  That word automatically connotes importance in peoples' minds, however, importance is basically unrelated to statistical significance.  The p-value simultaneously measures two things: the effect size, and $N$.  The reason your situation seems perplexing is that you have a small effect, but an $N$ that is so large you get a highly 'significant' p-value anyway.  Years ago, Kirk coined the term 'practical significance' to denote the idea of whether the effect was large enough to be meaningful.  A Google search led me to this, which looks good upon skimming it and may be helpful for familiarizing yourself with the ideas.
On a different note, it is not clear what can or should be concluded from your data.  It appears that you do not have a control group, or an estimate of the trend prior to the intervention.  From a logical point of view, these facts make drawing any conclusions about the effect of the intervention problematic.  Furthermore, I'm guessing that your observations are not independent, as they appear to be sequential measurements taken over time.  Note that independence is one of the most important assumptions of the $\chi^2$ test.  Thus, from a statistical point of view, using the $\chi^2$ test, or other simple statistical test, for inference may be problematic as well.  There are ways of dealing with such situations, but they are more advanced.  You may want to work with a statistical consultant.  (Although if the data are not logically capable of answering your questions, it may not be worth it--unfortunately this case reminds me of a famous quote by the statistician Sir Ronald Fisher.)
Comment: The existence of the baseline is comforting, but without a control group, I think you would need to be able to estimate the prior *trend*. Think of it this way, what if the proportion of good results had been going down beforehand at a faster rate, & after the intervention it is still going down, but at a slower rate? Then, the intervention was positive, even though the post intervention slope is negative. I don't see how you can rule that out. Moreover, standard correlations & t-test will be subject to the same problems, so I don't think that will help you. I hope it works out.
Comment: I do have a baseline measure (which I should have identified). We then took subsequent measures as the intervention was implemented. Thanks for the clarification on independent observations- I understand now why the test was failing to demonstrate what I was seeing. Could I simply use the correlation coefficient and student's t? I just need to identify a trend and be able to defend intelligently the strength of that trend.
Comment: Also, in response to your quote from Sir Ronald Fisher, to quote Monty Python, "I'm not dead yet!"
Comment: You're absolutely right. Thankfully, the study is only for performance improvement and doesn't have to be perfectly scientific. We began the study because we knew this behavior was a problem so we'll assume it was already dismal and stable (which I understand, is a big assumption). Thanks for your help.
Comment: @user12234 gung is right that you really needed prior info or a control group to make statements about the impact of the intervention.  But - you do have plenty of information to estimate the trend over the course of the study.  I think a logistic regression (proportion good events ~ time, weighted by total events) would let you say pretty definitively that the proportion of good events is decreasing in a way that is not due to chance (but again, nothing about the intervention).

Am I using the wrong statistical test?


Adicionar Objetos dentro de uma Array em JavaScript

Question: preciso chegar nesse resultado aqui:
[
   {
      alu_id: 10,
      alu_nome: 'Edson Santana dos Santos',
      notas: {
                dis_id: 1, dis_nome: 'Português', nota: 25
             }
   },
   {
      alu_id: 11,
      alu_nome: 'Larissa Manoela da Silva',
      notas: { dis_id: 3, dis_nome: 'História', nota: 28 },
             { dis_id: 1, dis_nome: 'Português', nota: 29 },
             { dis_id: 2, dis_nome: 'Matemática', nota: 25 }
]

Porém está saindo esse resultado:
[
   {
      alu_id: 10,
      alu_nome: 'Edson Santana dos Santos',
      notas: { dis_id: 1, dis_nome: 'Português', nota: 25 }
   },
   {
      alu_id: 11,
      alu_nome: 'Larissa Manoela da Silva',
      notas: { dis_id: 2, dis_nome: 'Matemática', nota: 25 }
   },
   {
      alu_id: 11,
      alu_nome: 'Larissa Manoela da Silva',
      notas: { dis_id: 3, dis_nome: 'História', nota: 28 }
   },
   {
      alu_id: 11,
      alu_nome: 'Larissa Manoela da Silva',
      notas: { dis_id: 1, dis_nome: 'Português', nota: 29 }
   }
]

CÓDIGO FONTE:
const data = {
                disciplinas: [],
                alunos: []
             }

for (let i = 0; i < results.length; i++) {
   data.disciplinas.push({
      dis_id: results[i].dis_id,
      dis_nome: results[i].dis_nome
   })
   data.alunos.push({
      alu_id: results[i].alu_id,
      alu_nome: results[i].alu_nome,
      notas: {
                dis_id: results[i].dis_id,
                dis_nome: results[i].dis_nome,
                nota: results[i].notas_valor
             }
   })
}

DADOS DO BANCO:
Answer: Tenha uma variável para controlar se você está fazendo outro data.alunos.push() para o mesmo aluno. Declare essa variável antes do for. Exemplo:
var alu_id_anterior = null

Assim que for tratar os dados dos alunos, faça antes o teste:
if (results[i].alu_id === alu_id_anterior) {
    // outra nota para o mesmo aluno
} else {
    // primeira vez que trata o aluno
}

Quando cair no bloco do if faça o push apenas da nota:
data.alunos[data.alunos.length - 1].notas.push({
    dis_id: results[i].dis_id,
    dis_nome: results[i].dis_nome,
    nota: results[i].notas_valor
})

Quando cair no bloco do else, faça o push do aluno juntamente com a nota, conforme você já estava fazendo.
Como última instrução do for, guarde o id do aluno que você acabou de processar.
alu_id_anterior = results[i].alu_id

Para que tudo isso funcione, é fundamental que o resultado da sua consulta à tabela esteja ordenado por alu_id.
Answer: Estás a usar data.alunos.push para inserir novos dados na array, quando devias usar .push no aluno específico. Para isso notas tem de ser uma array e tens de encontrar o aluno no qual queres inserir as notas.
Podes fazer isso assim:

const results = [{
    alu_id: 10,
    alu_nome: 'Edson Santana dos Santos',
    notas: {
      dis_id: 1,
      dis_nome: 'Português',
      nota: 25
    }
  },
  {
    alu_id: 11,
    alu_nome: 'Larissa Manoela da Silva',
    notas: {
      dis_id: 2,
      dis_nome: 'Matemática',
      nota: 25
    }
  },
  {
    alu_id: 11,
    alu_nome: 'Larissa Manoela da Silva',
    notas: {
      dis_id: 3,
      dis_nome: 'História',
      nota: 28
    }
  },
  {
    alu_id: 11,
    alu_nome: 'Larissa Manoela da Silva',
    notas: {
      dis_id: 1,
      dis_nome: 'Português',
      nota: 29
    }
  }
];

function organizarNotas() {
  const data = results.reduce((alunos, entrada) => {
  const id = entrada.alu_id;
    const aluno = alunos[id];
    if (!alunos[id]) {
      alunos[id] = {
        ...entrada,
        notas: []
      }
    }

    alunos[id].notas.push(entrada.notas);
    return alunos;
  }, {});
  return Object.values(data);
}

console.log('Notas:', organizarNotas(results));
Comment: Muito obrigado Sergio, deu muito certo sua função, está me retornando o que eu preciso!


MySQL Query in PHP - Not Correct?

Question: What is wrong with this query? It appears to be correct to me:
mysql_query("UPDATE culture SET cult_desc=$culture WHERE cult_id is $UID");

Modified it, NetBeans is still giving me an error. Here's my total code for the page:
$culture = $_POST["culture"];
if (isset($_POST["id"]))
    $UID = $_POST["id"];
    mysql_query("UPDATE culture SET cult_desc='$culture' WHERE cult_id=$UID");
else
     mysql_query("INSERT INTO culture
             VALUES(cult_desc='$culture')");

Comment: Why do you think there is something wrong?
Comment: because there obviously was something wrong
Answer: what's the value of $culture?
If it's a string, you'll need to encapsulate it with quotes.
Same thing for $UID.
Also, The 'is' in the where-condition should be '='
Also: watch our with this code. Make sure that $culture and $UID can not contain any malicious values (e.g. malicious input from users)
Comment: This is an internal admin system, so there won't be malicious values.
Answer: cult_desc probably string so need to wrap with ' '
mysql_query("UPDATE culture SET cult_desc='$culture' WHERE cult_id = $UID");

Answer: if SET cult_desc is a string then
mysql_query("UPDATE culture SET cult_desc='$culture' WHERE cult_id = $UID");

or
mysql_query("UPDATE culture SET cult_desc=$culture WHERE cult_id = $UID")

Answer: Seeing the newly edited code, your update-statement is now correct, but your insert statement now is wrong.
Try:
mysql_query("INSERT INTO culture (culture_desc)
             VALUES ('$culture')");

Answer: your problem in the { and } of if else statement
$culture = $_POST["culture"];
if (isset($_POST["id"])){
    $UID = $_POST["id"];
    mysql_query("UPDATE culture SET cult_desc='$culture' WHERE cult_id=$UID");
}else{
     mysql_query("INSERT INTO culture
             VALUES(cult_desc='$culture')");
}

Answer: $sql = "UPDATE 'culture' SET `cult_desc` = '$culture' WHERE `cult_id` = '$UID'";

Basically, you're using is instead of =
Answer: Depending on the data type of $culture and $UID you might be missing quotes. Cult_desc sounds like a string and thus $culture should be enclosed in quotes.
Answer: You should always check the output of mysql_error.http://php.net/manual/en/function.mysql-error.
I also usually use = instead of 'is' and also wrap all of my input data in quotation marks. eg
$sql = "UPDATE 'culture' SET cult_desc = '".$culture."' WHERE cult_id = '".$UID."'";

MySQL Query in PHP - Not Correct?


How to initialize Graphics? because it says it was not initialized

Question: I have this part of my code but it will not initialize and I do not know how to do it. It keeps giving me an error such as
    Exception in thread "main" java.lang.NullPointerException
    at memor.main(memor.java:131)
import java.awt.*;

import javax.swing.JFrame;
import javax.swing.JPanel;
import java.awt.GridLayout;
import javax.swing.JButton;
import javax.swing.JLabel;
import javax.swing.SwingConstants;

import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

public class memor extends JFrame
{
private static final long serialVersionUID = 1L;

public static void main(String args[]){

final JPanel pan;
GridLayout h=new GridLayout(3,3);

pan =new JPanel(h);
JButton button1= new JButton("1");
pan.add(button1);
final JLabel label1= new JLabel("hi");
label1.setVisible(false);
pan.add(label1);
JButton button2= new JButton("2");
pan.add(button2);
final JLabel label2= new JLabel("hi");
label2.setVisible(false);
pan.add(label2);
JButton button3= new JButton("3");
pan.add(button3);
final JLabel label3 = new JLabel("hi");
label3.setVisible(false);
pan.add(label3);
JButton button4 = new JButton("4");
pan.add(button4);
final JLabel label4 = new JLabel("hi");
label4.setVisible(false);
pan.add(label4);
JButton button5= new JButton("5");
pan.add(button5);
final JLabel label5= new JLabel("hi");
label5.setVisible(false);
pan.add(label5);
JButton button6= new JButton("6");
pan.add(button6);
final JLabel label6= new JLabel("hi");
label6.setVisible(false);
pan.add(label6);
JButton button7= new JButton("7");
pan.add(button7);
final JLabel label7= new JLabel("hi");
label7.setVisible(false);
pan.add(label7);
JButton button8= new JButton("8");
pan.add(button8);
final JLabel label8= new JLabel("howdy");
label8.setVisible(false);
pan.add(label8);
JButton button9= new JButton("9");
pan.add(button9);

final JLabel label9= new JLabel("hi");
label9.setVisible(false);
pan.add(label9);
JLabel l=new JLabel("grid layout");
l.setHorizontalAlignment(SwingConstants.CENTER);
button1.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label1.setVisible(true);
    }
});
button2.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label2.setVisible(true);
    }
});
button3.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label3.setVisible(true);
    }
});
button4.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label4.setVisible(true);
    }
});
button5.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label5.setVisible(true);
    }
});
button6.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label6.setVisible(true);
    }
});
button7.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label7.setVisible(true);
    }
});
button8.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label8.setVisible(true);
    }
});
button9.addActionListener(new ActionListener() {
    public void actionPerformed(ActionEvent arg0) {
        label9.setVisible(true);

    }
});

if (button1.isEnabled()){
Graphics g= pan.getGraphics();
g.setColor(new Color(156, 93, 82));
g.fill3DRect(21,3,7,12, true);
g.setColor(new Color(156,23,134));
g.fillOval(1,15,15,15);
g.fillOval(16,15,15,15);
g.fillOval(31,15,15,15);
g.fillOval(7,31,15,15);
g.fillOval(22,31,15,15);
g.fillOval(16,47,15,15);

}}

}

Comment: For better help sooner post an [SSCCE](http://sscce.org/)
Comment: Your code above does not run. Well, it runs but it displays no GUI and it really doesn't make sense. All your code is one big static main method. What GUI could you be painting from a static main method? How about posting a functioning *small* example program that actually runs and displays a GUI?
Answer: If this is a Swing application, then you should either be drawing to a BufferedImage, in which case you'd get the BufferedImage's Graphics object via getGraphics(), or you would be doing this drawing in a JPanel's paintComponent(Graphics g) method and would use the Graphics object passed into the method by the JVM.
To tie this in to a JButton's being enabled or not, I'd add a ChangeListener to the button's model, and call repaint if the enabled property changes, and then would have an if block in the JPanel's paintComponent method and base whether or not to paint the shapes on the button's enabled state.
For more detailed help, consider posting an sscce.


Characteristic polynomial of a matrix is monic?

Question: Given a $n \times n$ matrix A, I need to show that its characteristic polynomial, defined as $P_A (x) = det (xI-A)$ is monic. I am trying induction. But no clue after induction hypothesis.
Answer: You have
$$c_A(\lambda) = \det(\lambda I - A)
= \sum_{\sigma\in \Sigma_n} {\rm sgn}(\sigma)
\prod_{i=1}^n
(\lambda \delta_{i\sigma(i)} - A_{i\sigma(i)}),$$
where $\Sigma_n$ is the permutation group on $n$ letters.  The only term in the sum giving a $\lambda^n$ term is the identity permutation.  Hence the polynomial is monic.
Comment: Should the $j$ on the second line be $i$?
Answer: $$A=\begin{pmatrix}a_{11}&a_{12}&\ldots&a_{1n}\\a_{21}&a_{22}&\ldots&a_{2n}\\\ldots&\ldots&\ldots&\ldots\\a_{n1}&a_{n2}&\ldots&a_{nn}\end{pmatrix}\implies $$
$$|xI-A|=\begin{vmatrix}x-a_{11}&-a_{12}&\ldots&-a_{1n}\\-a_{21}&x-a_{22}&\ldots&-a_{2n}\\\ldots&\ldots&\ldots&\ldots\\-a_{n1}&-a_{n2}&\ldots&x-a_{nn}\end{vmatrix}=\prod_{i=1}^n(x-a_{ii})+\ldots=x^n+\ldots$$
and this is enough since we know $\;\deg p_A(x)=n\;$ ...
Answer: If you already know that $P_A$ is a polynomial, and if the matrix has real or complex coefficients, here is a nice trick:
$$ \lim_{|x|\to+\infty} \frac{P_A(x)}{x^n}
= \lim_{|x|\to+\infty} \det \left( I - \frac{1}{x} A \right)
= \det I
= 1, $$
because $\det$ is continuous.
From this, you can deduce that $P_A$ is monic of degree $n$.
Answer: If we use the permutation definition of the determinant, there will be only one term in the sum that is a polynomial of degree $n$, namely the product of all diagonal elements. (The rest of the terms form a polynomial of degree strictly less than $n$.)
This term is of the form $(x - a_{11}) \dots (x - a_{nn})$, which is clearly monic.
Answer: Let $A=(a_{ij})$ so we have
$$P_A(x)=\left|\begin{array}\\
x-a_{11}&\cdots&\cdots&-a_{1n}\\
-a_{21}&x-a_{22}&\cdots&-a_{2n}\\
\vdots&&\vdots\\
-a_{n1}&\cdots&\cdots&x-a_{nn}
\end{array}\right|$$
We expand the determinant relative to the first line we have
$$P_A(x)=(x-a_{11})\Delta_{11}+\sum_{j=2}^n(-1)^{1+j}a_{1j}\Delta_{1j}$$
where $\Delta_{1j} $ is a polynomial with degree less or equal $n-2$ and then we see that the term on $x^n$ of $P_A(x)$ is in  $(x-a_{11})\Delta_{11}$ and by induction in
$$(x-a_{11})(x-a_{22})\cdots(x-a_{nn})$$
and since the constant term of $P_A(x)$ is $(-1)^n\det A$ hence we have
$$P_A(x)=x^n-\underbrace{\left(\sum_{k=1}^na_{kk}\right)}_{=Tr A}x^{n-1}+\cdots+(-1)^n\det A$$
Answer: For $n=1$,
\begin{equation}
P_A(x)=det(xI_1-A)=|s-a_{11}|=s-a_{11}.
\end{equation}
So, $P_A(x)$ is monic for $n=1$.
Assume $P_A(x)$ is monic for $n=k$, i.e.
\begin{equation}
P_A(x)=det(xI_k-A)=\begin{vmatrix}
                   (x-a_{11})&-a_{12}&...&-a_{1k}\\
                   -a_{21}&(x-a_{22})&...&-a_{2k}\\
                   \vdots&\vdots&\ddots&\vdots\\
                   -a_{k1}&-a_{k2}&...&(x-a_{kk})
                   \end{vmatrix}=x^k+...
\end{equation}
We are now left to show that; $P_A(x)$ is monic for $n=k+1$, so
\begin{equation}
\begin{split}
P_A(x)=&det(xI_{k+1}-A)\\
=&\begin{vmatrix}
                   (x-a_{11})&-a_{12}&...&-a_{1k}&-a_{1,k+1}\\
                   -a_{21}&(x-a_{22})&...&-a_{2k}&-a_{2,k+1}\\
                   \vdots&\vdots&\ddots&\vdots\\
                   -a_{k1}&-a_{k2}&...&(x-a_{kk})&-a_{1,k+1}\\
                   -a_{k+1,1}&-a_{k+1,2}&...&-a_{k+1,k}&(x-a_{k+1,k+1})
\end{vmatrix}\\
=&\underbrace{(-a_{k+1,1})(-1)^{(k+1)+1}\begin{vmatrix}
                   -a_{12}&...&-a_{1k}&-a_{1,k+1}\\
                   (x-a_{22})&...&-a_{2k}&-a_{2,k+1}\\
                   \vdots&\ddots&\vdots\\
                   -a_{k2}&...&(x-a_{kk})&-a_{1,k+1}
                   \end{vmatrix}}_{order\ of\ x\ is\ k-1}\\
+&\underbrace{(-a_{k+1,2})(-1)^{(k+1)+2}\begin{vmatrix}
                   (x-a_{11})&-a_{13}&...&-a_{1k}&-a_{1,k+1}\\
                   -a_{21}&-a_{23}&...&-a_{2k}&-a_{2,k+1}\\
                   -a_{31}&(x-a_{33})&...&-a_{3k}&-a_{3,k+1}\\
                   \vdots&\ddots&\vdots\\
                   -a_{k1}&-a_{k3}&...&(x-a_{kk})&-a_{1,k+1}
                   \end{vmatrix}}_{order\ of\ x\ is\ k-1}\\
+&...\\
+&(x-a_{k+1,k+1})(-1)^{\overbrace{(k+1)+(k+1)}^{2(k+1)}}det(xI_k-A)\\
=&(x-a_{k+1,k+1})(x^{k}+...)+...\\
=&x^{k+1}+...
\end{split}
\end{equation}
So, $P_A(x)$ is also found monic for $n=k+1$.
$\therefore$ $P_A(x)$ is monic.$\square$
Answer: The characteristic polynomial is the product of monic degree-1 or degree-2 terms, corresponding to eigenvalues and eigenpairs, respectively.
The set of monic polynomials is closed under multiplication. That is your induction hypothesis.
done.
Comment: That's what I thought at first, @NicholasR.Peterson, yet that weird division of the char. poly. in "monic pol's of degree 1 and two..." Perhaps the answerer was thinking of what happens in the real field, where any pol. is factorable in a product of linear and quadratic pol's ? But even then, why should those factors be monic??
Comment: What's an "eigenpair"?
Comment: @DonAntonio I would imagine that he means pairs of complex eigenvalues, given the context.


After inserting data Cannot forward after response has been committed

Question: "Here is code after inserting data its cannot forwarding the index page please help me.....
and after that i am not able to fetch data also...
"
'
public class insertData extends HttpServlet {

String query;

static Connection conn;
//Statement st;
PreparedStatement pst;
ResultSet res;
ConnectionManager dbconn;

protected void processRequest(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {
    response.setContentType("text/html;charset=UTF-8");
    try (PrintWriter out = response.getWriter()) {

        dbconn= new ConnectionManager();

        String id=request.getParameter("id");
        String fname=request.getParameter("fname");
        String lname=request.getParameter("lname");
        String pass=request.getParameter("pass");
        String mobile=request.getParameter("mbl");
        String email=request.getParameter("email");

        conn=dbconn.getConnection();
         //pst=conn.createStatement();
        pst=conn.prepareStatement ("INSERT INTO reg(`id`, `fname`, `lname`, `pass`, `mbl`, `email`) VALUES (?,?,?,?,?,?)");
       // query="insert into reg(id('"+id+"','"+fname+"','"+lname+"','"+pass+"','"+mobile+"','"+email+"')";
        pst.setString(1, id);
        pst.setString(2, fname);
        pst.setString(3, lname);
        pst.setString(4, pass);
        pst.setString(5, mobile);
        pst.setString(6,email);

        int i=pst.executeUpdate();

    }
    catch(Exception e)
    {
      //System.out.println("Exception"+e.getMessage());
      request.setAttribute("Error",e);
      RequestDispatcher rd=request.getRequestDispatcher("/error.jsp");
      rd.forward(request, response);

    }
    finally
    {
        RequestDispatcher rd=request.getRequestDispatcher("/index.jsp");
        rd.forward(request, response);

    }
}

'
so......how can i fix this please help me...thanks in advance for help
Comment: Try closing `out` as `out.close()` at the end of the `try` block.
Comment: Thanks For response but still.getting this....Warning:   StandardWrapperValve[insertData]: Servlet.service() for servlet insertData threw exception
java.lang.IllegalStateException: Cannot forward after response has been committed
 at org.apache.catalina.core.ApplicationDispatcher.doDispatch(ApplicationDispatcher.java:448)
 at org.apache.catalina.core.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:428)
 at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:378)
 at insertData.processRequest(insertData.java:66)
 at insertData.doGet(insertData.java:84)
Comment: Are facing any error? If so, please post the stacktrace
Comment: You should declare variables ```con```, ```pst``` and ```dbconn``` in the method.
Answer: When an exception occur as per above code it will forward the request using RequestDispatcher to error.jsp. Once this happen your Request is committed and now you cant do anything more with it.
But you have a finally block - which will get executed irrespective of the fact try block being successful or throwing an exception. You are again trying to get an RequestDispatcher there and forward to index.jsp and hence getting the above error.
Not sure of your exact use case but feel you can redirect to your index.jsp as last step of Try block itself.And remove the finally block altogether.
One more thing to note is its not advised to write your JDBC/DB related code in Servlet itself you can create a separate DAO class for that and it will lead to better abstraction and modularity.


How do I view the string pulled from feeds image importer?

Question: I am importing images from a feed using xPath parser. I think that I have the correct attribute value, media:content@url, but I am getting a Unable to handle the provided embed string or URL. error. My first instinct is to check that I'm getting the proper string. How do I do this? Is there a way through devel to view the imported string? Devel?
The url string I want to pull is https://cdn2.orgsync.com/images/os-photos/s5o0zqk8o29mi46_710.jpg
Answer: I haven't solved my problem, but I found the hook needed to spit out the pre-parse data.
\**
* Importer testing.
*/
function wpi_events_feeds_after_parse(FeedsSource $source, FeedsParserResult $result) {
dpm($result);
}

This spits out the source strings, which is great, because it tells me my xpath selector is wrong.


Why JVM does not support forced class/classloader unloading?

Question: Disclaimer: I know how classes are loaded in JVM and how and when they are unloaded. The question is not about the current behaviour, the question is, why JVM does not support "forced" class/classloader unloading?
It could have the following semantics: when classloader is "forced unloaded", all classes it loaded are marked by "unloaded", meaning no new instances will be created (an exception will be thrown, like "ClassUnloadedException"). Then, all instances of such unloaded classes are marked as "unloaded" too, so every access to them will throw InstanceUnloadedException (just like NullPointerException).
Implementation: I think, this could be done during garbage collection. For example, compacting collector moves live objects anyway, so it can check if class of current object was "unloaded" and instead of moving object change the reference to guarded page of memory (accessing it will throw the abovementioned InstanceUnloadedException). This will effectively make object garbage, too. Or probably that could be done during "mark" phase of GC. Anyway, I think this is technically possible, with little overhead when no "unloading" occurs.
The rationale: Such "hardcore" mechanism could be useful for runtimes where a lot of dynamic code reloading occurs and failure of particular application or part of it is tolerable whereas failure of whole JVM is undesirable. For example, application servers and OSGi runtimes.
In my experience, dynamic redeployment in 9 cases of 10 leads to PermGenSpace due to the references not being cleaned up correctly (like ThreadLocal in static field filled in long-living thread, etc). Also, having an explicit exception instead of hard-to-debug leak could help polishing the code so no references are leaked into the long-living scope uncontrolled.
What do you think?
Comment: I thought about this for a while too, but the cost of having the program attempt to deal with a class that could suddenly "go missing" would create a lot of extra logic and new complexities.  is the solution making the problem worse?  Not sure...
Comment: You've always got the option of writing your own classloader if you want different behavior.
Comment: @Jonathan B: Yeah, but, for example, you can get OutOfMemoryError at any time. Moreover, with OOME you can't control when you will get it and you cannot limit it to some module. With "forced unloading" you are sure that only unloaded module and its clients could fail. With well-thought modularity enforced by runtime, you can limit the who will fail.
Answer: This feature would just cause havoc and confusion. Forcing the unload of a class would bring a lot of problems, like deprecated Thread.stop() had, except that would be many more times worse.
Just for comparing, Thread.stop() tends to leave a lot of objects in inconsistent states due to the abrupt thread interrupting, and the thread could be executing any type of code. Coding against that in practice is impossible, or at least an tremendous extreme effort. It is considered between almost impossible and completely impossible to write a correct multithreded code in that scenario.
In your case, that sort of feature would have similar bad side-effects, but in much worse scale. The code could get the exception anywhere unexpectedly, so in practice would be very difficult or impossible to code defensively against it or handle it. Suppose that you have a try block doing some IO, and then a class is abruptely unloaded. The code will throw a ClassUnloadedException in an unexpected place, potentially leaving objects in inconsistent states. If you try to defend your code against it, the code responsible for that defense might fail as well due to another unexpected ClassUnloadedException. If you have a finally block that tries to close a resource and a ClassUnloadedException is thrown inside that block, the resource could not be closed. And again, handling it would be very hard, because the handler could get a ClassUnloadedException too.
By the way, NullPointerException is completely predictable. You got a pointer to something null and tried to derefence it. Normally it is a programming error, but the behaviour is completelly predictable. ClassCastException, IllegalStateException, IllegalArgumentException and other RuntimeExceptions are (or at least should be) happening in predictable conditions. Exceptions which are not RuntimeException may happen unexpectedly sometimes (like IOException), but the compiler forces you to handle or rethrow them.
On the other hand, StackOverflowError, OutOfMemoryError, ExceptionIninitializerError and NoClassDefFoundError, are unpredictable things that may happen anywhere in the code and very rarely there is something possible to do to handle or recover from them. When some program hits that, normally they just go erratic crazy. The few ones that try to handle them, limits to warning the user that it must be terminated immediatelly, maybe trying to save the unsaved data. Your ClassUnloadedException is a typical thing that would be a ClassUnloadedError instead. It would manifest itself like a ExceptionIninitializerError or NoClassDefFoundError which in 99% of the cases means just that your application is completely broken, except that it would be much worse because it has not a fail-fast behaviour, and so it gets still more randomness and unpredictableness to it.
Dynamic redeployment is by its very nature, one of the most ugly hacks that may happens in a JVM/Container since it changes abruptely the code of something that is already running on it, which tends to get you to a very erratic random buggy behavior. But it has its value, since it helps a lot in debugging. So, a defense against erratic behavior that the container implements, is to create a new set of classes to the running program and shares memory with the older one. If the new and the old parts of your program don't communicate directly (i.e., just by compatible serialization or by no communication at all), you are fine. You are normally safe too if no structural changes occurs and no living object depends of some specific implementation detail that changed. If you do follow these rules, no ClassUnloadedError will be show. However there are some situations where you may not follow these rules and still be safe, like fixing a bug in a method and changing its parameters and no live object exists which depends on the bug (i.e., they never existed or they are all dead).
But if you really wants a ClassUnloadedError being thrown if an object of the older part is accessed, as this behaviour flags that one of that isolament rules were broke, and then bring everything down. So, there is no point in have new and old parts of the program in the same time, it would be simpler to just redeploy it completely.
And about the implementation in the GC, it does not works. A dead object is dead, no matter if its class object is dead too or alive. The living objects of unloaded classes can't be garbage collected, because they are still reachable to other objects, no matter if the implementation of every method will magically change to something that always throws an Exception/Error. Backtracking the references to the object would be a very expensive operation in any implementation, and multithreading this would be still worse, possibly with a severe performance hit in perfectly living objects and classes.
Further, dynamic loading classes are not intended for production use, just for developer tests. So, it is no worth to buy all that trouble and complexity for this feature.
Concluding, in practice your idea creates something that combines something similar to Thread.stop() with something similar to NoClassdefFoundError, but is stronger than the sum of the two. Like a queen is a combination of a bishop and a rook in chess, but is stronger than the sum of the two. It is a really bad idea.
Comment: >>Backtracking the references to the object would be a very expensive operation in any implementation, and multithreading this would be still worse, possibly with a severe performance hit in perfectly living objects and classes.

That's not completely true. My point was, GC **does this** when relocates live objects. It's the same, just an extra check if this object should be relocated or dropped.
Comment: As for the other part of the post, that is true if you program defensively. However, defensive programming is not the only way to build robust systems, look at Erlang. http://www.erlang.se/doc/programming_rules.shtml#HDR11

Fail fast! That's another approach. Also, the point was I am fine if one particular application would crash. The application server itself (or it may be runtime running in the embedded device which you cannot access easily and you need to update it from time to time with new code) should continue running.
Comment: The fail-fast that i meant to say is like the Iterator.next() of the Collections class. If something wrong is detected, it fails quickly immediately. In NoClassDefFoundError, the class could not be loaded and it immediatelly fails. In your case, it would fail unexpectedly long after it started to run.
Comment: My ClassUnloadedError will still fail faster than OutOfMemoryError will occur, maybe, a day later in some completely separate piece of code. I agree the proposed mechanism is very hardcore, I just think it could have its business value. The same with Thread.stop(), by the way. It is dangerous to use Thread.stop() in your logic, but it could be useful to kill resistant threads. The same does Linux on shutdown, kill -TERM first, then kill -KILL for those who failed to shutdown.


Why is my right click/start menu behind my application?

Question: My menu appears behind my window? any solutions?
image 1
image 2
Answer: Does this happen with every application or just some? If your window is "Always On Top" then right clicks on apps behind it will not render correctly in some situations.


Check for values of $x$ that are not a basis for $V$ with basis $B$

Question: I have a vector space $V$ with basis $B = \{v_1, v_2, v_3\}$.
I have a set $Z = \{v_1 + v_2, v_2 + v_3, v_3 + av_1\}$.
What value of $a$ makes $Z$ not a basis for $V$?
I know that $Z$ needs to be linearly independent to be a basis. But I'm lost on what to do after this...
Answer: We know that $dimV=3$ also $|Z|=3$ so $Z$ forms a basis if and only if elements of $Z$ are linearly independent
if $B = \{v_1, v_2, v_3\}$ is a basis then
$c_1v_1+c_2v_2+c_3v_3=0 \Rightarrow c_1=c_2=c_3$
$a_1(v_1+v_2)+a_2(v_2+v_3)+a_3(v_3+av_1)=0 \Rightarrow (a_1+aa_3)v_1+(a_1+a_2)v_2+(a_2+a_3)v_3=0$
$(a_1+aa_3)v_1+(a_1+a_2)v_2+(a_2+a_3)v_3=0 \Rightarrow (a_1+aa_3)=(a_1+a_2)=(a_2+a_3)=0 $
$(a_1+a_2)=(a_2+a_3) \Rightarrow a_1=a_3$
$a_1=a_3 \Rightarrow a_1+aa_3=(a+1)a_1=0$
$(a+1)a_1=0 \Rightarrow a_1=0$ or $a=-1$
$a_1=0 \Rightarrow a_1=a_2=a_3=0$
$a=-1 \Rightarrow -a_2=a_3=a_1$ is a solution which means they are linearly dependent
Answer: Hint: Evaluate the determinant:
$\det\left(\begin{array}{ccc}
1 & 0 & a\\
1 & 1 & 0\\
0 & 1 & 1\\
\end{array}\right)$.
The vectors are given by the columns of the matrix.
$\det = 1 + a$ and so $\det$ is zero (i.e., vectors are linearly dependent and so not a basis) if $a=-1$ as in the other answer.


UFW route not responding

Question: Hi i tried to make a server where every software running on it is in a vm.
i tried using ufw to route the requests and it worked ... locally.
when i tried to connect to one of my vm using a vpn i saw this in tcpdump(ran outside a vm
dump where we see the request but the ICMP for the response fail
Without the vpn and pinging the external ip of my router i get this successful exchange between the server and the router):
my routes are defined in /etc/ufw/before.rules as such
<code>*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -p tcp --dport 25565 -j DNAT --to-destination 192.168.122.232:25565
-A PREROUTING -p tcp --dport 25522 -j DNAT --to-destination 192.168.122.232:22
COMMIT
</code>
i did my test with the 25565 port.
and my server is running fedora server.
my VM can connect to the internet so it should be able to send the response back but the icmp fail at my host server.
the strangest part in all of that is that when i run nc $SERVERIP 25565 i get connection refused even if the port is open in my router
sorry for my approximative english i not a native speaker.
Answer: i followed this guide : https://www.cyberciti.biz/faq/kvm-forward-ports-to-guests-vm-with-ufw-on-linux/
and when i saw
<code>## NOTE replace 192.168.2.0/24 with your public IPv4 sub/net   ##
</code>
i used 192.168.0.14 instead of my router.

UFW route not responding


Exporting Client Data from Groupwise 6.5 to Outlook 2010 without Crashing

Question: My employer has recently moved from Novell GroupWise 6.5 to Exchange 2010.
We've imposed mailbox limits on staff but we still need to move their old messages, contacts, calendars, etc. over to Outlook 2010.
Our problem however is this, utilizing the Novell MAPI client is slow within Outlook 2010 and upon exporting messages to a PST file (for later re-attachment, and offline backup purposes) crashes the GroupWise server.  Connecting to the server in Outlook via IMAP to export messages to PST is faster and apparently more stable but also crashes the server.
We'll be keeping our GroupWise server online internally until then end of the year but I have staff with mailboxes approaching 12 gigabytes, which is fine if we're going to move the data to offline storage (DVD set) but if I keep crashing the server every time I try to get the data I'll just be spinning my wheels.  In my first attempts, I tried to move mail for a staff member with 3GB of data. The transfer lasted roughly 8 hours before crashing.
I'm wondering if there is an open source solution to my problem.  Paid solutions exist but we're a not-for-profit organization and have too many staff to justify the costs of per seat licenses just to migrate mail.
Comment: Maybe see if a mail archival solution could help you here. It could potentially suck in the mail off the Groupwise server directly and then provide you an archive location for your Exchange environment moving forward. Look into MessagingArchitects
Answer: You might look at imapsync though, frankly, if Outlook is crashing GroupWise when export IMAP then it's likely imapsync will crash it, too.


Null hypothesis in permutation inference for the general linear model

Question: I'm currently studying a paper by Winkler et. al on permutation inference for the general linear model and am confused by their reasoning and notation.
They first introduce a formulation for the general linear model:

Y
=
M
ψ
+
ϵ     (Eq. 1)
where Y is the N × 1 vector of observed data, M is the full-rank N × r
design matrix that includes all effects of interest as well as all
modelled nuisance effects, ψ is the r × 1 vector of r regression
coefficients, and ϵ is the N × 1 vector of random errors.

Regarding the null hypothesis, it is stated that:

Our interest is to test the null hypothesis that an arbitrary
combination (contrast) of some or all of these parameters is equal to
zero, i.e., H0 : C′ψ = 0, where C is a r × s full-rank matrix of s
contrasts, 1 ≤ s ≤ r.

The general formulation (Eq. 1) in the first quote can be transformed into a partioned one:

Y
=
X
β
+
Z
γ
+
ϵ     (Eq. 2)
where X is the matrix with regressors of interest, Z is the matrix
with nuisance regressors, and β and γ are the vectors of regression
coefficients.

When discussing the Freedman-Lane procedure for permutation inference in the following it is stated that:

The rationale for this permutation method is that, if the null hypothesis is true, then β = 0 [...]

These paragraphs pose two questions to me:

What does C′ in H0 : C′ψ = 0 denote mathematically speaking, given that C is a matrix?

How does C′ψ = 0 imply β = 0? Is the reasoning C′ψ = 0, therefore Y = M0 + ϵ = X0 + Z0  + ϵ = ϵ valid?

Thanks a lot in advance, I'd be very grateful for help or advice on how to approach/present this problem in a better way.
Answer: The C matrix just indicates which slopes, or linear combinations of slopes, is equal to 0.  If you want to test that the first slope is 0 (but the remaining can be anything) then C will just be a vector with the first element $1$ and the remaining elements are $0$.  If you want to test that the first 3 slopes are all 0, then C is a 3x3 identity matrix with the remaining rows equal to $0$.  This is the simple case which leads to $\beta=0$ for a set of the slopes.  But is also more general in that you could test if $\beta_1 = \beta_2$ by setting the first column of C to $1$ and $-1$.
Comment: @AdrianMak, the null can be a scalar if you only have one contrast ($s=1$), but in general the right side of the null hypothesis is a vector of $0$'s, each contrast is equal to 0.
Comment: The prime "'" indicates the transpose of  the C matrix.
Comment: Thanks a lot for that answer! I do however have a basic follow up question:
If  `C` is a `r x s` matrix that indicates which slopes are equal to zero and `ψ` a `r x 1` vector that contains the slopes/regression coefficients themselves, how can `C′ψ` result in a scalar as stated in the null hypothesis (`C′ψ = 0`). Shouldn't the result always be of form of a `r x 1` vector?
Comment: Okay thanks, allow me one last question: In `C′ψ = 0`, what does ` ′ ` indicate? Because from all of your answers, it just seems like `C`and `ψ` are just multiplied in the null hypothesis.


Getting Expected Value from Monte Carlo Simulation

Question: There are two independent uniform continuous random variables $X$ and $Y$ (such that $0 \leq X  \leq  10$, $0  \leq  Y \leq 10$). The function $f$ is the difference between the two random variables ($|X-Y|$).
What is the expected value of $f(X,Y)$?
My analytic solution was $10-\sqrt{50}$. I wanted to check my answer with Monte Carlo Simulation.
I randomly chose $1M$ numbers each for $X$ and $Y$ and calculated one million $f(X,Y)$. One intuitive way of getting the expected value is averaging the million values ($\approx$ 3.33). Another way is getting the median(the distributional balance) value ($\approx$ 2.93) among the million values.
The second one was close to my analytic solution.
Maybe I am confusing different concepts. What is the right way of calculating the expected value through the MC simulation? Is my analytic answer correct? If not, how do I calculate it analytically?

updated:
After some research, it seems like the correct analytic solution should be instead(for simplication, the uniform distributions of X, Y are U(0,1) instead of U(0,10):
$E(|X-Y|) = \int_{y=0}^{1}\int_{x=0}^{1}|x-y|2(1-|x-y|)dxdy = \frac{1}{3}$
Because the p.d.f. of Z is $2(1-|x-y|)$, which is derived from the convolution function between $X$ and $-Y$.
Answer: The right way of calculating the expected value of a function by Monte Carlo simulation is to calculate the (sample) average of the function value on all n (one million in your write up) replications.  You should also look at the sample standard deviation divided by sqrt(n) or sqrt(n-1), to understand the uncertainty in your estimate.
You need to correctly generate the values of X and Y, accounting for dependency, if any, between X and Y. I.e., draw from their joint distribution. if they are independent, this amounts to drawing separately from their own distributions. You have specified Uniform([0,10]) distributions for X and Y, and based on the results of the Monte Carlo simulation, you have assumed they are independent.
Assuming independence of X and Y, then indeed your averaging is the correct thing to do to get expected value of abs(X-Y).  Your analytical calculation of 10 - sqrt(5) does indeed correspond to the median of abs(X-Y), not its expected value
Plot the sorted values of abs(X-Y) from the simulation, and you'll see that the median is not equal to the expected value.  You may be too used to symmetric distributions, and perhaps in particular the Normal distribution. Median = expected value in such cases, but not for abs(X-Y) in your case.
Comment: @Tim I calculated the geometric probability for 0.5 point(finding a, where |X-Y| is larger than value "a" by the probability of 0.5).

(10*10 - (10-a)^2 )/(10*10) = 0.5

However, now it seems it is not the expected value.
Comment: That is the median. The median does not equal the mean when the distribution is not symmetric.  |X-Y| where X and Y are independent Uniform[0,10] is not symmetric.Look at the link I provided in the comment above to see how to calculate E(|X-Y|). If you don't understand that, you need to read a probability book or take a course in introductory calculus-based probability.
Comment: Thank you Mark. Then how do I calculate the expected value analytically, if mine was wrong?
Comment: @Saju maybe first tell us how did you calculate it..?
Comment: http://math.stackexchange.com/questions/48822/calculate-the-probability-density-function-of-the-absolute-difference-of-two-ran


Is this an unbiased estimator for standard deviation of normal distribution?

Question: Suppose we have $n$ samples, with mean $\mu$.
Calculate the average absolute distance from $\mu$, i.e.,
$$
y = \frac{1}{n} \sum_{i=1}^n |X_i - \mu| \>.
$$
Then, take as an estimate of the standard deviation
$$ \tilde \sigma = y\frac{1}{1-1/n}\sqrt{\pi/2}$$
Is it unbiased?
Comment: Good start, stevie. You have the scale factor correct, but not the conclusion regarding the unbiasedness. Your follow-up question is a good one, with a fairly deep answer. You might consider adding it to the question to make it more prominent. :)
Comment: So, presumably $\mu$ is known *a priori*?
Comment: **Hint**: Can you find the expectation $\mathbb E |X - \mu|$? It *is* a scaled version of $\sigma$. Find the scale factor.
Comment: I think I proved it was unbiased estimator.  E|X-u| = standard deviation * sqrt(2/pi)
Comment: I was wondering though why we don't see this formula that often for estimating standard deviation?
Comment: I've made some edits to your post to markup the math. Please check that I have not introduced any errors.
Comment: We calculate u like sample mean.  The formula you have for is correct.
Answer: The proposed estimator is not unbiased, at least if we indeed know the true mean, $\mu$, and if we are dealing with a normal sample as the title says, where the distribution is symmetric and unimodal and the mean equals the median. Informally, knowing the true mean, makes the mean absolute deviation equal in value to the probability limit of the same expression with $\bar X$ instead of $\mu$. We have
$$ y=\frac{1}{n} \sum_{i=1}^n |X_i - \mu| = \frac{1}{n}\left[\sum_{X_i\geq \mu} (X_i - \mu)+\sum_{X_j< \mu} (\mu - X_j)\right]$$
Denote $m_1$ the count for the first sum, and $m_2$ the count for the second sum (both are random variables). Then, using also Wald's equation
$$E(y) = \frac{1}{n}\Big[E(m_1)E(X\mid X\geq \mu) - E(m_1)\mu + E(m_2)\mu - E(m_2)E(X\mid X\leq \mu)\Big]$$
Since we have the true mean, which is equal to the median, we get $E(m_1)=E(m_2) = n/2$, so the two middle terms cancel, while substituitng for the expected values of the counts, taking common factors and simplifying we arrive at
$$E(y) = \frac{1}{2}\Big[E(X\mid X\geq \mu)  - E(X\mid X\leq \mu)\Big]$$
For the truncated normal distribution, these expected values are
$$E(X\mid X\geq \mu) = \mu + \sigma \frac{\phi(0)}{1-\Phi(0)} = \mu +\sigma\sqrt{2/\pi}$$
$$E(X\mid X\leq \mu) = \mu - \sigma \frac{\phi(0)}{\Phi(0)} = \mu -\sigma\sqrt{2/\pi}$$
So
$$E(y)=\frac{1}{2}\Big[\mu +\sigma\sqrt{2/\pi}  - \mu +\sigma\sqrt{2/\pi}\Big] = \sigma\sqrt{2/\pi}$$
So the correction factor in $\tilde \sigma$ should be $\sqrt{\pi/2}$ only, for it to be unbiased.
I note that since $X_i - \bar X = (1-1/n)X_i - (1/n)\sum_{j\neq i}X_j$ one suspects that we should examine the case where we do not know $\mu$ and we use the sample mean instead, which I may find the time to do later.
Comment: Simulations indicate that the form with the sample mean is biased (as we might expect, since the variation about the sample mean will tend on average to be lower than the variation around the true mean), though of course as n gets large the bias rapidly disappears.


Problem with rendering Hugo/blogdown site

Question: I've seen the post about blogdown::serve_site() no longer serving the site and read the release notes for blogdown 0.21, but it didn't help with my problem.
My workflow is/was to write a post, then click "Serve Site" in RStudio and check out the newly generated files in the public/ folder of my project. I have a symbolic link of that folder in my ShinyApps directory so I can view my site via the Shiny server. This is great, because then my colleagues who also use the server can see my site as well.
Now this doesn't work anymore. While I get the updated site in RStudio directly, the files displayed by the Shiny server are not being updated. The only explanation I can find is this one:

The global option blogdown.generator.server has been deprecated. Now blogdown::serve_site() always use the Hugo server (which corresponds to options(blogdown.generator.server = TRUE) in previous version of blogdown), instead of the server created via the servr package (which corresponds to the default options(blogdown.generator.server = FALSE) before).

I don't know much about Hugo but I found that hugo server doesn't update the public/ directory, is that correct? What can I do now to update that?
Comment: Please see https://github.com/rstudio/blogdown/issues/495
Comment: Thank you! That seems to cover all of my issues.
Answer: The question was already answered on GitHub:
I need to build the site with blogdown::build_site(local=TRUE).
Answer: Edit: Turns out that the below was not the solution either for me. Therefore I posted an own question with a possible workaround:
Problem (and solution?) with rendering Hugo/blogdown site

Earlier (old) post:
With me this did not help. When updating blogdown and starting-up my R project, blogdown:::preview_site(startup = TRUE) automatically runs. Something I don't recall from earlier start-ups. I now always receive the same Error message:

Launching the server via the command:
hugo server --bind 127.0.0.1 -p 4321 --themesDir themes -t hugo-academic -D -F --navigateToChanged
sh: line 0: kill: (3262) - No such process
Error: It took more than 30 seconds to launch the server. There may be something wrong. The process has been killed. If the site needs more time to be built and launched, set options(blogdown.server.timeout) to a larger value.

Running blogdown::build_site(local = TRUE) results in an even longer Error message starting with:

ERROR 2020/11/13 15:55:56 render of "page" failed: execute of template failed: template: _default/single.html:6:5: executing "_default/single.html" at <partial "page_header.html" .>: error calling partial: execute of template failed: template: partials/page_header.html:92:7: executing "partials/page_header.html" at <partial "page_metadata" (dict "page" $page "is_list" 0 "share" true)>: error calling partial: "/Users/frederick/Dropbox/EUR/R_work/r_website/r_website_project/themes/hugo-academic/layouts/partials/page_metadata.html:63:31": execute of template failed: template: partials/page_metadata.html:63:31: executing "partials/page_metadata.html" at <.>: range can't iterate over R

Solution for me
For me, it helped to roll-back to blogdown version 0.20 like so:
packageVersion("blogdown")
#> [1] '0.21'

library("devtools")
#> Loading required package: usethis
install_version("blogdown", version = "0.20", repos = "http://cran.us.r-project.org")
#> Downloading package from url: http://cran.us.r-project.org/src/contrib/Archive/blogdown/blogdown_0.20.tar.gz

packageVersion("blogdown")
#> [1] '0.20'

Created on 2020-11-13 by the reprex package (v0.3.0)
Now everything is back to "normal".
Comment: Could this have anything to do with your theme? See "When creating a new site with blogdown::new_site(), the theme gcushen/hugo-academic is automatically redirected to wowchemy/starter-academic, because the original Github repo has moved and become a repo of Hugo modules." https://github.com/rstudio/blogdown/releases
Comment: Yes that could be. Only I have no clue how to fix this.
Comment: You could file an issue in either the blogdown or hugo-academic repo, I guess. @Yihui is usually around to help, but I don't know how to tag him here.


AVQueuePlayer AVPlayerItemDidPlayToEndTimeNotification fails to call

Question: Im using AVQueuePlayer to loop through an array of AVPlayerItems.
The way I'm looping it, I listen to the AVPlayerItemDidPlayToEndTimeNotification and every time its called, I add the current object to the end of the queue.
heres the code:
    viewWillAppear
{
    ...

    [[NSNotificationCenter defaultCenter] addObserver:self
                                                 selector:@selector(playerItemDidReachEnd:)
                                                     name:AVPlayerItemDidPlayToEndTimeNotification
                                                   object:[_queuePlayer currentItem]];

        [_queuePlayer play];
}
- (void)playerItemDidReachEnd:(NSNotification *)notification {
    AVPlayerItem *p = [notification object];
    AVPlayerItem *fakeNew = [[AVPlayerItem alloc] initWithAsset:p.asset];
    if (_queuePlayer.items.count == 1)
    {
        [p seekToTime:kCMTimeZero];
        [_queuePlayer play];
    }
    else
    {
        [_queuePlayer insertItem:fakeNew afterItem:[_queuePlayer.items lastObject]];
    }
    NSLog(@"array of items to play:%lu", (unsigned long)_queuePlayer.items.count);

}

The problem is, that the method is called only for the first video that plays, after that, the method stops getting called, so if for example i have 2 movies in the array, it would play them both+the first one again, any idea why is this happening?
More Info:
also tried to make a new player every time and set it to layer. failed to send the notification more than once just the same
- (void)playerItemDidReachEnd:(NSNotification *)notification {
    AVPlayerItem *p = [notification object];
    [self.playList removeObjectAtIndex:0];
    [self.playList addObject:p];
    AVPlayer *newPlayer = [[AVPlayer alloc] initWithPlayerItem:[self.playList objectAtIndex:0]];
    _player = newPlayer;
    self.AVPlayerLayerView.layer.player = self.player;
    [_player play];

}

Answer: After a lot of messing around, apparently for whatever reason, the view unregistered as observer every time, I just removed and added observer after every notification:
- (void)playerItemDidReachEnd:(NSNotification *)notification {
    AVPlayerItem *p = [notification object];
    AVPlayerItem *fakeNewItem = [[AVPlayerItem alloc] initWithAsset:p.asset];
    [self.playList removeObjectAtIndex:0];
    [self.playList addObject:fakeNewItem];
    AVPlayer *newPlayer = [[AVPlayer alloc] initWithPlayerItem:[self.playList objectAtIndex:0]];
    _player = newPlayer;
    self.AVPlayerLayerView.layer.player = self.player;
    [[NSNotificationCenter defaultCenter] removeObserver:self];
    [[NSNotificationCenter defaultCenter] addObserver:self
                                             selector:@selector(playerItemDidReachEnd:)
                                                 name:AVPlayerItemDidPlayToEndTimeNotification
                                               object:[_player currentItem]];
    [_player play];

}

Comment: because you are resisting a notification for a AVPlayerItem, and you are initializing an AVQueuePlayer with an array, hence you need to register for AVPlayerItemDidPlayToEndNotification for each playerItem in that array, not only currentItem
Answer: For a clean approach to resolve this issue. I approached with the next piece of code instead
The first is you have to add the code necessary to receive a feedback from the AVPlayer when the reproduction time changes.
- (void)addPeriodicTimeObserverForReproductionChanges {
    @weakify(self);
    [self.player
    addPeriodicTimeObserverForInterval:CMTimeMakeWithSeconds(kBQDefaultTimeIntervalReproductionChanges, NSEC_PER_SEC)
    queue:self.eventsQueue
    usingBlock:^(CMTime time) {
    @strongify(self);
    [self dispatchBlockOnMainQueue:^{
        if ([self.delegate respondsToSelector:@selector(playerItemController:didChangeReproductionTime:)])
             [self.delegate playerItemController:self
                       didChangeReproductionTime:time];

         [self checkForItemPlayToEnd];
     }];
 }];

}

- (void)checkForItemPlayToEnd
{
    CMTime durationScaled = CMTimeConvertScale(self.duration,self.player.currentTime.timescale, kCMTimeRoundingMethod_Default);

    if (CMTIME_COMPARE_INLINE(durationScaled, ==, self.player.currentTime)) {
        [self playerDidFinishReproducingItem];
    }
}


Re-expressing the Schrodinger Equation as a first order expansion.

Question: I am reading an online text on quantum computing and the author expands and re-expresses the Schrodinger equation. I am not really sure as to the intermediate steps he used or what happened to the derivative when going from the first to second equation. Thanks in advance.
$$\frac{d} {dt} |ψ(t)>  =  −iH(t)|ψ(t)> $$
"re-express this equation, to first order in the infinitesimal quantity dt".
$$|ψ(t + dt)> = (I − iH(t)dt)|ψ(t)>$$
Answer: You could use a forward difference approximation for the derivative:
$$
\frac {d \psi}{dt} = \frac{\psi(t + dt) - \psi (t) }{ dt } $$
Here I am treating dt now as an infinitesimal. Insert this and solve for $\psi (t+dt) $.


gfortran linking c libraries with conda

Question: I am trying to compile a C++/Fortran program using conda on an Ubuntu 18.04 server where I do not have superuser rights.
I am able to compile correctly the program with the same source code on my Ubuntu 18.04 PC (using conda too), but on the server I get a bunch of errors. At the moment, I am stuck with a "library not found - undefined reference" error:
gfortran -o glm -Wl,--export-dynamic obj/glm_globals.o obj/glm_util.o obj/glm_csv.o obj/glm_mobl.o obj/glm_mixu.o obj/glm_wqual.o obj/glm_layers.o obj/glm_surface.o obj/glm_input.o obj/glm_plot.o obj/glm_output.o obj/glm_ncdf.o obj/glm_lnum.o obj/glm_init.o obj/glm_flow.o obj/glm_mixer.o obj/glm_deep.o obj/glm_stress.o obj/glm_bird.o obj/glm_model.o obj/glm_types.o obj/glm_const.o obj/glm_debug.o obj/glm_main.o obj/glm_zones.o -L/local/XXX/my_name/PycharmProjects/glm/glm_source/GLM/../libutil/lib -lutil -L/local/XXX/my_name/anaconda3/lib -lnetcdf -L/usr/lib -L/local/XXX/my_name/PycharmProjects/glm/glm_source/GLM/../libplot/lib -lplot -lgd -lpng -ljpeg -lm -lX11    -lgfortran

/local/XXX/my_name/anaconda3/bin/ld: warning: libpthread.so.0, needed by /local/XXX/my_name/anaconda3/lib/libnetcdf.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libxcb.so.1, needed by /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libdl.so.2, needed by /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: librt.so.1, needed by /local/XXX/my_name/anaconda3/lib/./libhdf5_hl.so.100, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: warning: libresolv.so.2, needed by /local/XXX/my_name/anaconda3/lib/././libgssapi_krb5.so.2, not found (try using -rpath or -rpath-link)
/local/XXX/my_name/anaconda3/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so: undefined reference to `dlopen@GLIBC_2.2.5'
/local/XXX/my_name/anaconda3/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libX11.so: undefined reference to `xcb_wait_for_event'
/local/XXX/my_name/anaconda3/bin/ld: /local/XXX/my_name/anaconda3/lib/././libkrb5.so.3: undefined reference to `__res_nsearch@GLIBC_2.2.5'
...

collect2: error: ld returned 1 exit status
Makefile:277: recipe for target 'glm' failed
make: *** [glm] Error 1

I can see the shared libraries libpthread.so.0, librt.so.1, libdl.so.2, libresolv.so.2 not found by the linker are part of glibc. So I assume there is some kind of problem linking to that library.
In a previous attempt I tried installing glibc with conda. This resulted in conda crashing with a "segmentation fault (core dumped)" when I tried to compile. I had to reinstall anaconda to make conda work again.
In the last days I also have tried to add -rpath as suggested in the warnings, to add an LD_LIBRARY_PATH, to add the directory containing the shared libraries with -L. Nothing worked.
I feel quite lost at the moment. Any idea of what could have gone wrong?
PS: The difference between compiling on my computer and on the server is that I installed missing libraries in the system on my computer, but on conda on the server. Thus, I had to add the location of those libraries with -L when compiling on the server.
Answer: This answer is a suggested alternative workflow that aims at avoiding the problem rather than an exact diagnosis of the issue in OP.
In my experience, I have found the Conda Forge compiler packages help simplify the creation and use of custom environments for compilation. As an example, here is the YAML definition for the environment I use to build the kallisto software. For your case, I would expect something like
fortran-compiler.yaml
name: fortran-compiler  # name it what you want
channels:
  - conda-forge
  - defaults
dependencies:
  - fortran-compiler
  - cxx-compiler
  - libpng
  - libgd
  - jpeg
  - libnetcdf
  - openlibm
  - xorg-libx11
# include the other required libraries

Create the environment with
conda env create -f fortran-compiler.yaml

then run your compilation with this environment activated. Activating the environment should automatically manage where the linker will look so that the libraries in the environment will be found (this is, in part, what the Conda Forge *-compiler packages provide). The idea is to have as many libraries as possible come from the environment itself.
I find this approach keeps the amount of manual locating and including of library paths to a minimum. However, it does require having to track down the libraries in Conda Forge (e.g., search for libx11), which unfortunately is not always a one-to-one mapping from library name to package name. The advantage is that a compilation environment defined in this way can facilitate transferring across platforms - e.g., the example YAML I have for kallisto works on both osx-64 and linux-64 without any changes - because it explicitly defines that Conda will provide the shared libraries.
Perhaps there are simpler ways to do this, but this is at least what I've found works after having multiple bad experiences trying to use the gcc or clang Anaconda packages directly.
Comment: Thanks, that worked great. In addition to your solution, I only had to modify my gfortran call to indicate the location of the libraries within the virtual environment with -L. No more errors.

gfortran linking c libraries with conda


If $\lambda=\omega_\overline\alpha$, $k_{\alpha<\lambda}\neq0$ and $k=\sup k_{\alpha<\lambda}$ then $\sum_{\alpha<\lambda}k_\alpha=k*\lambda$?

Question: I carry on following what is written at the 9th chapter of "Introduction to Set Theory" by Karel Hrbacek and Thomas Jech
$\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit$
1.3 Theorem
Let $\lambda$ an infinite cardinal, let $k_{\alpha<\lambda}$ be nonzero cardinal numbers, and let $k=sup[k_{\alpha<\lambda}]$. Then
$\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \sum_{\alpha<\lambda}k_\alpha=k*\lambda$.
Proof. On the one hand, $k_\alpha\le k$ for each $\alpha<\lambda$, and so $\sum_{\alpha<\lambda}k_\alpha\le\sum_{\alpha<\lambda}k=k*\lambda$. On the other hand, we notice that $\lambda=\sum_{\alpha<\lambda}1\le\sum_{\alpha<\lambda}k_\alpha.$ We also have $k\le\sum_{\alpha<\lambda}k_\alpha$: the sum $\sum_{\alpha<\lambda}k_\alpha$ is an upper bound of the $k_\alpha$'s and $k$ is the least upper bound. Now since both $k$ and $\lambda$ are $\le\sum_{\alpha<\lambda}k_\alpha$, it follows that $k*\lambda$, which is the greater of the two, is also $\le\sum_{\alpha<\lambda}k_\alpha$. The conclusion of Theorem 1.3 is now a consequence of the Cantor-Bernstein Theorem.
$\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit\diamondsuit$
Well I don't understand why since both $k$ and $\lambda$ are $\le\sum_{\alpha<\lambda}k_\alpha$, it follows that $k*\lambda$, which is the greater of the two, is also $\le\sum_{\alpha<\lambda}k_\alpha$. Could someone explain to me this formally?
Anyway it seems to me that it could be possible to prove it in this way: since $\lambda$ is an infinite cardinal, $k$ must have this property and then it's $k*\lambda=k+\lambda \le\sum_{\alpha<\lambda}k_\alpha+\sum_{\alpha<\lambda}k_\alpha=\sum_{\alpha<\lambda}k_\alpha$; is it correct?
Comment: Sorry, could you explain better?
Comment: Why? Sorry but I don't understand.
Answer: Exercise.
Show if a and b are infinite cardinals, then a×b = max{a,b}.
That will answer your question.
This his comment "which is the greater of the two."
Comment: Of course; I already knew this: if $\alpha$ and $\beta$ are infinite cardinals then $\alpha*\beta$=$max${$\alpha,\beta$}=$\alpha+\beta$ so  the results is immediate considering that $\lambda$, $k$ and $\sum_{\alpha<\lambda}k_\alpha$ are infinite cardinals: infact it's result $k*\lambda=k+\lambda\le\sum_{\alpha<\lambda}k_\alpha+\sum_{\alpha\lambda}k_\alpha=\sum_{\alpha<\lambda}k_\alpha$; or as you said $k*\lambda=$$max${$k,\lambda$}$\le\sum_{\alpha<\lambda}k_\alpha$. Maybe I was wrong?
Comment: You are right, a+b = the larger.  You way requires extra, unneeded steps.


Comparando campos de varias TextBox para checar se há duplicidade

Question: Estou desenvolvendo um programa básico em C# onde o usuário vai inserir valores através de um leitor de códigos de barra. O mesmo será salvo em uma planilha de Excel para um futuro relatório.
O meu problema é que existe situações em que o usuário bipa o mesmo código duas vezes em textbox diferentes.
Queria saber se, ao clicar no meu button Salvar, tivesse um evento para comparar todos os meus textbox e checar se há valores repetidos, impedindo de salvar as informações duplicadas.
Segue abaixo código do meu Button Adicionar:
private void BtnAdicionar_Click(object sender, EventArgs e)
{
    Microsoft.Office.Interop.Excel.Application Excel = new Microsoft.Office.Interop.Excel.Application();
    Microsoft.Office.Interop.Excel.Workbook sheet = Excel.Workbooks.Open("C:\\Users\\Core i3\\Desktop\\EtiqMasterBoxTornozeleira.xlsx");
    Microsoft.Office.Interop.Excel.Worksheet x = Excel.ActiveSheet as Microsoft.Office.Interop.Excel.Worksheet;

    Excel.Range userRanger = x.UsedRange;

    int countRecords = userRanger.Rows.Count;
    int add = countRecords + 1;
}

Answer: Uma maneira simples de fazer isso é você usar um Array
        Int64[] valores = new Int64[3];
        valores[0] = Convert.ToInt64(textBox1.Text);
        valores[1] = Convert.ToInt64(textBox2.Text);
        valores[2] = Convert.ToInt64(textBox3.Text);

        int quant_campos = valores.Length;
        var groups = valores.Distinct().ToList();

        if(quant_campos > groups.Count)
        {

            MessageBox.Show("Existe Campos Duplicados");
        }

Comment: Obrigado pela ajuda, funcionou aqui, agora só preciso implementar um método pra ele não deixar salvar mesmo com campos duplicados, que é oque ainda esta acontecendo, mas ele ja acusa que há campos duplicados. Obrigado
Answer: Algumas coisas não ficaram muito claras, mas vou tentar ajudar.
Você comenta sobre iterar os textboxs, e por isso, vou concluir que você não precisa confirmar as informações que já foram gravadas anteriormente na sua planilha.
Pois bem:
Salve cada código de barras (string) em uma coleção de strings da sua escolha; mas antes, confirme se dentro dessa lista ainda não contem este "código de barras" que será salvo através do método Contains:
ICollection<string> codesCollection = new List<string>();

private void AddBarCode(string barcode)
{
    bool isDuplicatedCode = codesCollection.Contains(barcode);

    if(isDuplicatedCode)
        throw new ArgumentException("O código de barras já foi adicionado.").

    codesCollection.add(barcode);
}

Comment: O seu tambem funcionou amigo, obrigado pela ajuda.


python - multilayer perceptron, backpropagation, can´t learn XOR

Question: i am trying implement multilayer perceptron with backpropagation, but still i cant teach him XOR, i will also often get math range error. I looked in books and google for learning rules and error back propagation methods, but still i have no idea where are my mistakes
def logsig(net):
    return 1/(1+math.exp(-net))

def perceptron(coef = 0.5, iterations = 10000):
    inputs = [[0,0],[0,1],[1,0],[1,1]]
    desiredOuts = [0,1,1,0]
    bias = -1
    [input.append(bias) for input in inputs]
    weights_h1 = [random.random() for e in range(len(inputs[0]))]
    weights_h2 = [random.random() for e in range(len(inputs[0]))]
    weights_out = [random.random() for e in range(3)]
    for itteration in range(iterations):
        out = []
        for input, desiredOut in zip(inputs, desiredOuts):
              #1st hiden neuron
            net_h1 = sum(x * w for x, w in zip(input, weights_h1))
            aktivation_h1 = logsig(net_h1)
              #2st hiden neuron
            net_h2 = sum(x * w for x, w in zip(input, weights_h2))
            aktivation_h2 = logsig(net_h2)
              #output neuron
            input_out = [aktivation_h1, aktivation_h2, bias]
            net_out = sum(x * w for x, w in zip(input_out, weights_out))
            aktivation_out = logsig(net_out)
              #error propagation
            error_out = (desiredOut - aktivation_out) * aktivation_out * (1-    aktivation_out)
            error_h1 = aktivation_h1 * (1-aktivation_h1) * weights_out[0] * error_out
            error_h2 = aktivation_h2 * (1-aktivation_h2) * weights_out[1] * error_out
              #learning
            weights_out = [w + x * coef * error_out for w, x in zip(weights_out, input_out)]
            weights_h1 = [w + x * coef * error_out for w, x in zip(weights_h1, input)]
            weights_h2 = [w + x * coef * error_out for w, x in zip(weights_h2, input)]
            out.append(aktivation_out)
    formatedOutput = ["%.2f" % e for e in out]
    return formatedOutput

Answer: The only thing I notice is that you're updating weights_h1 and weights_h2 with error_out instead of error_h1 and error_h2.  In other words:
weights_h1 = [w + x * coef * error_h1 for w, x in zip(weights_h1, input)]
weights_h2 = [w + x * coef * error_h2 for w, x in zip(weights_h2, input)]

Answer: The math range error is likely coming from the math.exp(-net) calculation, for net a large negative number.


Empty collection check returning false with no entries (Ruby on Rails)

Question: I have both a Customer model and Device model, and the Customer model has_many :devices and the Device model belongs_to :customer. I'm trying to show either a form to add a device on a customer's home page if customer.devices.empty? is true or simply show customer's device and accompanying details if customer.devices.empty is false.
My problem is that customer.devices.empty? is always returning false. With some tests, I've seen that customer.devices.count will always display the correct number of devices, however I only get the desired behavior out of customer.devices.empty? while using the Rails console.
I can simply check the value of customer.devices.count, but I would really would like to use the empty? or any? checks as (I think) they are intended.
The problem itself has been described but if you'd like to see code...
   <% if customer.devices.count == 0 %>
     Count is 0
   <% end %>
   <% if customer.devices.empty? %>
     Customer has no devices!
   <% end %>
   <% if customer.devices.any? %>
     Customer has <%= pluralize(customer.devices.count, "device") %>.

   <% end %>

Almost forgot my manners -- Thanks in advance to any and all answers.
-MM
Comment: Have you loaded (but not saved) any devices on this customer?  The `.count` method will always trigger a database query, while `empty?` will not if the association has already been loaded.  Please post the relevant controller code...
Comment: So basically, have I associated a device to a customer (say at some point `@device = customer.devices.build(params[:device]`) without actually performing a `@device.save`? I created a completely new user that, by default, has no associated devices and I realized something strange. I'm using a set of helper functions that let me keep track of the customer across different pages (with a cookie), and the site "home" page that references the helper function, displays the wrong result, whereas if I go to customers/3 it has direct access to the `@customer` variable, and displays everything correct..
Comment: Yup, that explains this behavior.  If you want to see unpersisted data, use the Enumerable methods (`any?`, `empty?`), otherwise use the query API.
Answer: Use exists? instead of empty?:
customer.devices.exists?

The difference is that exists? checks the database via the query API, while empty? checks the association contents as a standard Enumerable (which may be dirty/modified).
Comment: This works, but if you look at my response to your comment above, why would `empty?` work on the customer's page (i.e. after visiting customer/3 and having direct access to the @customer variable) but not when I'm using a helper function that gets the @customer variable by `@customer ||= Customer.find_by_remember_token(cookies[:remember_token])`?
Comment: @MandM: Where you haven't yet manipulated the `@customer.devices` collection, `empty?` will have no choice but to check the database.  If the collection was already instantiated (via a separate database call, or by manual manipulation as you are doing) it will simply check the collection in its current state.
Answer: As per your comment exists and count will trigger DB query to check associated devices. When you use build it is not saved in DB so exists returns false and count returns 0. When you use blank it will return false that means it has some devices
customer.devices.blank?

Comment: `exists?` is slightly more efficient since the database can use EXISTS which is more performant (where available), while `blank?` will call COUNT.  However, both will return the same result.
Comment: Would like to accept both answers, but Pinny slid in a few minutes before. Thanks!


Applying the T-test

Question:
So far I have calculated the sample means $\overline{A} = 0.75$ and $\overline{B} = 2.33$. Using these I computed the sample variances: $S_A^2 \approx \frac{28.805}{9} = 3.2005$ and $S_B^2 = \frac{36.081}{9} \approx 4.009$.
After that, I calculated the pooled variance $S^2_{pooled} = \frac{28.805 + 26.081}{18} = 3.6047$. But now I'm not sure how to proceed.
Is the t-test I'm supposed to use this one: $$t = \frac{\overline{x}-\overline{y}}{s_{pooled}\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}$$ How do I proceed?
Thank you in advance for explanations!
Comment: Since your groups are correlated (they patients are essentially the same across A and B), this is a case of Repeated Measure Design. I would suggest you do a a bit more research on the same before proceeding any further.
Comment: The wikipedia article on the t-test lists several types. You appear to have edited your question down so radically that it's been automatically flagged by the system as low quality, which puts it in review to be closed, and I don't see how it can remain open in its current state. You might do better to roll it back and try to actually answer your original question as far as you can yourself.
Comment: I see you've also invalidated an already received answer; as a result I will roll back to a version where the already-existing answer makes sense. You should then improve that question (consistent with the restrictions on self-study questions) as far as you can.
Answer: Since this is homework, some pointers:
You use t-tests outside of regression mainly for three things:

Testing the mean of a group of samples against a number, e.g. do French men tend to be larger than 1.7 m.
Testing whether two groups differ in some way, e.g. are French men larger than French women?
Testing whether two groups differ, if you have a clear one-to-one link of samples of one group to samples of the other group, e.g. are French husbands larger than their wives? Husbands and wives are linked by being married to each other.

For these cases you use the one-sample t-test, the two-sample t-test and the paired t-test respectively. Your questions shows that you still need to determine which of these three scenarios you have.

Applying the T-test


Permission tmp problem: Change of tmp directory

Question: I recently installed one new Drupal (7.32) on a server that doesn't belong to me. Since then, I have a permission problem, I can't edit CHMOD rights and when I try to add new content (with image(s)), I can't. Here's the mistake returned by Drupal:
Warning: move_uploaded_file(): open_basedir restriction in effect.
File(/home/tmp/php/phpBKRRLYL) is not within the allowed path(s): (/home/tmp/sessions/:/home/XXX/public_html/:/home/XXX/shared) in drupal_move_uploaded_file() (line 1639 of /home/XXX/www/public_html/YYY/drupal-7.32/includes/file.inc).

Warning: move_uploaded_file(): open_basedir restriction in effect.
File(/home/tmp/php/phpBKRRLYL) is not within the allowed path(s): (/home/tmp/sessions/:/home/XXX/public_html/:/home/XXX/shared) in drupal_move_uploaded_file() (line 1639 of /home/XXX/www/public_html/YYY/drupal-7.32/includes/file.inc).

Warning: move_uploaded_file(/home/tmp/php/phpBKRRLYL): failed to open stream:
Operation not permitted in drupal_move_uploaded_file() (line 1639 of /home/XXX/www/public_html/YYY/drupal-7.32/includes/file.inc).

Warning: move_uploaded_file():
Unable to move '/home/tmp/php/phpBKRRLYL' to '/home/XXX/www/public_html/YYY/drupal-7.32/sites/default/files/anime.jpg' in drupal_move_uploaded_file() (line 1639 of /home/XXX/www/public_html/YYY/drupal-7.32/includes/file.inc).

As we can see, Drupal apparently uses the server's tmp directory (in which I have no rights) and I would like it to use another tmp directory in "/home/XXX/www/public_html/YYY/drupal-7.32/" so that I can add/edit content properly.
How could I do this? By editing the "/includes/file.inc" file (if so, which line(s) shall I modify to do this?)? Something else?
NB: The system administrator is unavailable for an extended period so he can't change my rights for now. That's why it's important that I could change this tmp directory.
Thank you!
Answer: You can edit the relevant setting in Configuration > Media > File system > Temporary directory (/admin/config/media/file-system).
You can choose any directory in your document root skipping the forward slash. I would recommend you to specify tmp. Take care of the permissions.
Comment: The temp directory has to be writable by the web server. So if you use 775 then either the owner or group of the folder must be that of the webserver.
Comment: I tried by changing it to "/home/XXX/www/public_html/YYY/drupal-7.32/tmp" or "/sites/default/files/tmp", even "tmp", same problem. :/
Comment: Manually create a folder called tmp in your drupal root, then save the setting as tmp.
Comment: It's not working either. Just to be sure I'm not mistaken, I created a "temp" directory here : "/www/public_html/captainetwork/drupal-7.32/temp" with 775 rights and change the temp directory in file-system to "temp", is that the right way to do?
Comment: If your base url is public_html then possibly (not sure) you need to specify `captainetwork/drupal-7.32/temp` or change your $base_url in `settings.php`.
Comment: If you give it 777 permissions and it still doesn't work then you at least know that it isn't a permissions problem, in which case it is probably for some reason looking in the wrong place.
Comment: Indeed, I gave it 777 permissions and I still have the error, but the same error (with /home/tmp/php/phpBKRRLYL, not "temp").
Comment: Another track perhaps? If the change of the temp directory on file-system is not changing the tmp directory where Drupal stores its data, what else can I do? Is there a file I can edit (if so, which line(s)?), one extra module to modify it? Else? Thanks for your previous helps!

**Edit :** I just see that there is no .htaccess file. And when I try to upload it, it's automatically deleted after the upload. Is there a possible link between this and my problem?
Comment: @Kiritsu, this is a Q+A website, sorry but we cannot troubleshoot for you. The tmp dir is specified as stated above and in rooby's comments, and this pretty much answers your question. There may possibly be something not properly setup with your web server or your Drupal installation.
Comment: I understand ;) You all helpt me a lot already, I found the issue when I found that the .htaccess was automatically deleted. I will try to correct that problem now!

Thanks again!


Can I programmatically find a Sitecore item's parent matching a certain template id?

Question: I have a custom Sitecore command that I would like to expand upon to determine if the current item has a parent item matching a certain template id.
When the command/button is clicked, I will have access to the current Item.
My initial attempt, since I had issues with escaping the FullPath for use in a query, was to loop through the parent item (and then that item's parent, etcetera) to determine if any matched the template.
However, there seems like there should be a better way to do this.
So given an item, what's the best way to find whether it has a parent item that is of a certain template?
Comment: Per [a meta question](http://meta.stackoverflow.com/q/322356/11912), I am asking this more focused variant of [this question](http://stackoverflow.com/q/36991247/11912).
Answer: SXA way
 public static Item GetParentOfTemplate(this Item item, ID templateId)
    {
        if (item == null)
        {
            return null;
        }
        ItemRelationCacheValue itemRelationCacheValue = ParentOfTemplateCache.GetCacheForDatabase(item.Database.Name)?.Get(item, templateId);
        if (itemRelationCacheValue != null)
        {
            return item.Database.GetItem(itemRelationCacheValue.OutputID);
        }
        Template template = TemplateManager.GetTemplate(templateId, item.Database);
        if (template == null)
        {
            return null;
        }
        ID iD = item.ID;
        while (item != null)
        {
            Template template2 = TemplateManager.GetTemplate(item);
            if (template2 != null && template2.DescendsFromOrEquals(template.ID))
            {
                ItemRelationCacheValue value = new ItemRelationCacheValue(iD, item.Database.Name, templateId, item.ID);
                ParentOfTemplateCache.GetCacheForDatabase(item.Database.Name)?.Set(value);
                return item;
            }
            item = item.Parent;
        }
        return null;
    }

Answer: There's not really a good way to do it. In a regular system you'd do it in SQL, but Sitecore has obfuscated their 'Nexus' code which I believe is where all of this data access/mapping logic happens thus I don't really know how that part of Sitecore works so it's a difficult to try to reverse engineer a query to do this efficiently.
What I use in my projects I use a recursive function like this.
I use a delegate to determine if the item is a match for what I'm looking for, and just recurse up the tree - dropping out to return null if it reaches the root node.
public delegate bool ItemSelectionCondition(Item sitecoreItem);
public static Item GetAncestor(Item sitecoreItem, ItemSelectionCondition selectionCondition)
{
    Item parent = sitecoreItem.Parent;
    if (selectionCondition(parent))
    {
        return parent;
    }
    else if (Sitecore.ItemIDs.RootID == parent.ID)
    {
        return null;
    }
    else
    {
         return GetAncestor(parent, selectionCondition);
    }
}

So to work with this you'd need to define a delgate something like:
public static bool IsTemplateX(Item item)
{
   return item.TemplateID == Constants.TemplateXId;
}

It's quite a simple solution that's flexible enough to work with any Ancestor or Parent type things that I've needed to do.
Be warned though it will take several seconds to run the first time if Sitecore hasn't cached all of the items it needs to look at, but subsequent times it will take a bearable amount of time.
EDIT: Since this answer has gotten a few downvotes I thought I'd explain why it's the best way to do this in Sitecore.
The reson why a recursive approach is better that iterative is two-fold. First, it's less code so it looks nicer - but if that's not enough the second reason is that if you're careful about how you've structured your code and are compiling your project you'll be able to take advantage of tail recursion.
The reason why using Item.Axes.Ancestors (Or Item.Axes.GetAncestors() in later versions of Sitecore) is bad is because it has five steps:

Allocate a List of Items
Iteratively add All Ancestors to that list
Reverse the order of the List
Convert the list to an array
Use Linq to filter the array

This means that you're doing twice as much allocation as if you were to just recurse to the top of the content tree, but you're also ruling out the possibility of exiting the recursion at the first relevant item - and avoiding unecessary calls to the database.
I do have to conceed that the comment about using a Lucene Index however is right - that's the way that is potentially the fastest in almost all scenarios just because it can reduce your time finding the item down to a single Sitecore Query - or no Sitecore Query if you index the data about that ancestor that you need.
Comment: How about something like `item.Axes.Ancestors.Where (ancestor => ancestor.Template I'd == "xxx")`
Comment: @HishaamNamooya whether you use the Axes or a recursive function, you have the same number of ancestors. I would use the query, as long as its not a "fast" query, it will benefit from Sitecore's caching layers. Now if it was going the other way, a descendants call, then you are going to have performance issues. You would have to have a very deep tree for ancestors to really start causing a problem.
Comment: tbh - I can't remember the last time I used a Sitecore query. I almost always use the index to find items now. It is a much faster method :)
Comment: "My initial attempt was to loop through the parent item (and then that item's parent, etcetera) to determine if any matched the template." What is difference from @James Skemp initial approach?
Comment: @MartinDavies, using the Axes.Ancestors, will this not cause performance issue if it has many ancestors?
Comment: @RichardSeal, yeah the axes.descendants causes performance issue. We had experienced this when developing our project


Where is the gambler's fallacy in this coin flipping binomial distribution?

Question: Suppose we have a biased coin where:
$p(Heads) = 0.6$
$p(Tails) = 0.4$
If $X$ is the number of heads obtained in 10 flips, the binomial distribution says:
$p(X = 9) = 0.04$
$p(X = 10) = 0.006$
Suppose we are now flipping this biased coin, and we have flipped it 9 times so far. On all 9 flips, the coin landed on heads. Is heads or tails more likely on the next flip?
Answer 1: Since $p(X = 9) > p(X = 10)$, $X = 9$ is the more likely outcome. Therefore, the next flip is more likely to be tails.
Answer 2: Since $p(Heads) > p(Tails)$, the next flip is more likely to be heads.
I think the first answer is wrong because it looks like the gambler's fallacy, but I can't explain it in mathematical terms. Can someone explain how the reasoning in the first answer is faulty? How do I refute the reasoning given in the first answer?
Comment: Besides, $p(X= 9 ) > p(X = 10)$ is true. But what you actually want on the RHS, is the quantity $P(X = 10 | \mathrm{all\ of\  the\  first\  nine\  tosses\  are\  heads})$. So you are not even comparing the correct quantities in the first part. The second part basically says that by independence, this conditional expectation is just equal to the probability that the $10$th coin is heads (which is the correct way to reason), so a head is more likely than a tail.
Comment: @TeresaLisbon Thank you for the explanation. Could you perhaps copy your comment into an answer?
Comment: Sure, but I will mould it into an answer.
Comment: I have done it, thanks for asking me to do it!
Answer: Answer 1 is wrong. Suppose the coin is fair (the intuition here will be clearer).
Still, answer 1 would say that the probability of $X=9$ is greater. The reason is that each series of H and T has the same probability, and there are 10 series which give 9H1T compared to only a single series giving 10H.
BUT! Since you already tossed the coin 9 times, you ruled out 9 of the possible 9H1T series, which means that you either in a series of 10H or a series of 9H followed by 1T. Both are equality probable.
Answer: The term you are looking for is independence.
In probability, independent events are ones where the occurrence of one event does not affect the probability of occurrence of the other.  For example, flipping heads on a coin does not make it more or less likely to roll a $6$ on a die.  We say that these events are independent.
To refute the first answer, we want to rephrase the question to "is the next flip more likely to be Heads or Tails [given the first nine flips were Heads]?"  We can now apply what we know about conditional probability.
Answer: Independence is of course important, but there is also dependence here that one should make note of. Once you know the result of the first $9$ coin flips, your probability distribution for $X$ gets skewed. For example, the likeliest value for $X$ is $6$, given no prior information. But once you see that the first 9 coins have been heads, you cannot still possibly hope that $X$ will be $6$ at the end. Similarly, even though $X=9$ is likelier than $X=10$, given no prior information, if you know that the first $9$ coins landed heads, the probability that $X=9$ and $X=10$ becomes equal.
Answer: $Pr(9 heads)$ includes $THHHHHHHHH,  HTHHHHHHH$ and so on.  But the only one left that is possible is $HHHHHHHHHT$


How to re-create vanilla field configuration strings programmatically for any field?

Question: After a D7 update and migration of fields, most of our problems with converted fields result from invalid or incompatible field settings. Field settings are stored in the
field_config_instance.data column in the database as serialized php objects.
How can I, for any given field instance, generate a fresh field configuration string, as if the field was just created in D7, while leaving the already stored field data intact?
My manual approach seems to work but is very cumbersome:

create a new content type
add an existing field to the new content type, generating new settings string
search the new and old field in the database
using sql: replacing the string in the data column of the old field (with the same name) with the new data content.

There must be a way to automate this a bit.
Answer: I guess you will need a combination of field_update_field() and field_create_field().
During field_create_field(), the settings are loaded from the default field definition as each field has defined in its own hook_field_info().
$field['settings'] += field_info_field_settings($field['type']);

What you are looking for is to apply this logic to field_update_field().
Basically you will need to load all fields:
$fields = field_info_fields();

Optionally limit the fields to your selection:
foreach ($fields as $field_name => $field) {
  if ($field['type'] == 'some type') {} // Filter by type
  if ($field_name == 'some name') {} // Filter by name
}

And update the fields:
$field['settings'] = field_info_field_settings($field['type']);
field_update_field($field);

So I guess it should look something like this:
$fields = field_info_fields();
foreach ($fields as $field_name => $field) {
  if ($field['type'] == 'some type') {
    $field['settings'] = field_info_field_settings($field['type']);
    field_update_field($field);
  }
}

Comment: Ah I see what you mean, but that union will make sure all non-defined values in the settings array will be set with the values of the previous field. Since we redefine the whole settings array with its defaults, I assumed it should be good. If the result is not as expected you can place the contents of the complete `field_update_field()` function in the snippet above and remove the union.
Comment: It might also be that modules implement `hook_field_update_forbid`. So you could attempt to bypass that as well.
Comment: Thank you for your answer, I think its very good information. I have run the suggested script and am printing the field settings before and after for all fields, however, the field settings appear unchanged whenever I run the script. Looking into the code of field_update_field. this line: `$field['settings'] += $prior_field['settings'];` computes the union of new and old settings. If I am not mistaken, this will always guarantee I will not loose any settings. In my case I need to remove the full settings record unfortunately, or at least remove some keys.
Comment: That is why I changed it into a regular = in the final snippet ;)
Comment: I see, but what I am saying is that field_update_field contains the same logic inside (using +=), so I will never be able to delete anything from the settings when using this function, if I am correct. So whenever a field is missing in the new settings but present in the old settings, field_update_field will keep the old settings.
Comment: Thank you, I am trying to modify the functions to circumvent all protection mechanisms.