É possível executar múltiplas queries numa única execução?

Question: Eu precisava de alguns dados de um banco de dados no mysql. Porém, os dados estavam em tabelas diferentes, logo, a solução mais óbvia era de filtrar usando as tabelas relacionais, com os respectivos índices. Mas eu não conheço muito de mysql, até tentei usar as tabelas relacionais no phpmyadmin, mas não soube como fazer.
O que fazer? Obviamente não ficaria de braços cruzados na frente do pc, logo tentei bolar uma solução com um script em PHP, já que sou mais familiarizado.
Eis o script:
<?php

function queryDB($user, $pass)
{
    $connect = new PDO('mysql:host=localhost;dbname=foobar', $user, $pass);

    foreach($connect->query("SELECT id, titulo FROM produtos WHERE publicado > 0") as $produto)
    {
        print($produto['id'].' -> '.$produto['titulo'].'
');

        $id = $produto['id'];

        $assuntos = $connect->query("SELECT id_assunto FROM produtos_tem_assuntos WHERE id_produto = $id");

        foreach($assuntos as $assunto_id)
        {
            foreach($connect->query("SELECT titulo FROM assuntos WHERE id = $assunto_id[0]") as $final)
            {
                print('
');
                var_dump($final[0]);
                print('
');
            }
        }
    }
}

print(queryDB('root', 'admin'));

O objetivo era:
1 - filtrar o produto de uma tabela, e pegar sua id;
2 - usar a id do respectivo produto para filtrar a id do assunto (que estava em outra tabela);
3 - usar a id do assunto para, enfim, mostrar o assunto (que também estava em outra tabela).
Com o script apresentado, eu consegui retornar os resultados esperados, mas como estou estudando boas práticas de código, penso que este script não está da maneira como deveria ser para executar as tarefas descritas. Eu mesmo chamei isso de gambiarra.
Então:
Qual seria a forma correta de fazer estas queries?

como eu poderia ter feito no phpmyadmin/mysql
como eu poderia ter feito o script (caso o atual seja redundante)
como poderia ser feito a refatoração deste código.
Comment: Um bom começo => [Qual é a diferença entre INNER JOIN e OUTER JOIN?](http://pt.stackoverflow.com/q/6441/91)
Comment: poderia me dar alguma referência destes métodos? um link seria ótimo!, e como vc faria a refatoração deste código usando o JOIN?
Comment: opa! vlw! eu até ja estava lendo http://dev.mysql.com/doc/refman/5.7/en/join.html, mas acho os docs do mysql muito confusos! Me ajudou :)
Answer: 1 . Primeiramente vamos ver como ficaria seus SELECT's se usarmos JOIN:
SELECT pr.id, pr.titulo, pt.id_assunto, as.titulo
FROM produtos pr
INNER JOIN produtos_tem_assuntos pt ON pr.id = pt.id_produto
INNER JOIN assuntos as ON pt.id_assunto =  as.id
WHERE pr.publicado > 0;

Um bom material de estudo para relacionamentos entre tabelas é o link abaixo:
http://blog.thiagobelem.net/relacionamento-de-tabelas-no-mysql/
2 . Agora vamos aplicar o SELECT na função, deixando-a bem mais enxulta:
function queryDB($user, $pass)
{
    $connect = new PDO('mysql:host=localhost;dbname=foobar', $user, $pass);

    $sql = "SELECT pr.id, pr.titulo, pt.id_assunto, as.titulo
            FROM produtos pr
            INNER JOIN produtos_tem_assuntos pt ON pr.id = pt.id_produto
            INNER JOIN assuntos as ON pt.id_assunto =  as.id
            WHERE pr.publicado > 0";

    $retorno = $connect->query($sql);

    // Testamos o retorno para não dar Warning
    if(count($retorno) > 0)
    {
        foreach( $retorno as $linha )
        {
            // Aqui fica sua lógica do que fazer com o retorno...
            print('
');
            var_dump($linha);
            print('
');
        }
    }
}

3 . Sobre refatorar o código, é sempre bom deixar a conexão em um arquivo separado, ou em uma variável global, evita você estar passando os dados de conexão em várias partes do código.
Na idéia abaixo, criamos um arquivo connection.php apenas para definirmos a conexão:
$user = "seu-usuario";
$pass = "sua-senha";
$connect = new PDO('mysql:host=localhost;dbname=foobar', $user, $pass);

Ai então nas paginas que vão ter acesso ao banco de dados, basta incluir a pagina connection.php e utilizar a variável connect para acesso ao banco:
include_once "connection.php";

function queryDB()
{
    // Pegamos a connecção global
    global $connect;

    $sql = "SELECT pr.id, pr.titulo, pt.id_assunto, as.titulo
            FROM produtos pr
            INNER JOIN produtos_tem_assuntos pt ON pr.id = pt.id_produto
            INNER JOIN assuntos as ON pt.id_assunto =  as.id
            WHERE pr.publicado > 0";

    $retorno = $connect->query($sql);

    // Testamos o retorno para não dar Warning
    if(count($retorno) > 0)
    {
        foreach( $retorno as $linha )
        {
            // Aqui fica sua lógica do que fazer com o retorno...
            print('
');
            var_dump($linha);
            print('
');
        }
    }
}

*** Obs: A solução apresentada não foi testada, pois eu não tinha a sua base de dados e tabelas criadas aqui... Caso apresente erros me informe que corrigimos.
Comment: perfeito! Obrigado :)


Gluon Applicatiopn Error Android Support Packages

Question: I have a problem with Gluon on Intellij I installed all teh Packages at the Android SDK Manager. And linked the android SDK Path in my Gluon Project build.gradle.
But i throws this Error all the times:
FAILURE: Build failed with an exception.

* What went wrong:
You must install the Android Support Repository. Open the Android SDK Manager and choose the Android Support Repository from the Extras category at the bottom of the list of packages.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 0.386 secs
You must install the Android Support Repository. Open the Android SDK Manager and choose the Android Support Repository from the Extras category at the bottom of the list of packages.
18:06:45: External task execution finished 'androidInstall'.

Here is my build.gradle
buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'org.javafxports:jfxmobile-plugin:1.3.4'
    }
}

apply plugin: 'org.javafxports.jfxmobile'

repositories {
    jcenter()
    maven {
        url 'http://nexus.gluonhq.com/nexus/content/repositories/releases'
    }
}

mainClassName = 'me.gentlexd.gluon.Main'

dependencies {
    compile 'com.gluonhq:charm:4.3.1'
    androidRuntime 'com.gluonhq:charm-android:4.3.1'
    iosRuntime 'com.gluonhq:charm-ios:4.3.1'
    desktopRuntime 'com.gluonhq:charm-desktop:4.3.1'
}

jfxmobile {
    downConfig {
        version = '3.2.4'
        // Do not edit the line below. Use Gluon Mobile Settings in your project context menu instead
        plugins 'display', 'lifecycle', 'statusbar', 'storage'
    }
    android {
        manifest = 'src/android/AndroidManifest.xml'
        androidSdk = "F:/Android"
    }
    ios {
        infoPList = file('src/ios/Default-Info.plist')
        forceLinkClasses = [
                'com.gluonhq.**.*',
                'javax.annotations.**.*',
                'javax.inject.**.*',
                'javax.json.**.*',
                'org.glassfish.json.**.*'
        ]
    }
}

Comment: Have a look at this [question](http://stackoverflow.com/questions/42143300/how-can-i-install-android-support-library-to-deploy-a-gluon-mobile-application/42143741#42143741)
Answer: You must create (but if not exists) gradle.properties in Users//.gradle/gradle.properties and add ANDROID_HOME=PathToAndroidSDK
This is my ANDROID_HOME=U:/AndroidStudio.


Views: Ranking users by pageviews this month, last month

Question: I have created a view to rank Users by the total numbers of pageviews their nodes have received. I now want to clone that view and create 2 blocks:

Totals views their nodes have received this month
Total view their nodes received last month.

I have tried using contextual filters and I can create a view to show total pageviews in a date range, but, that would mean me updating the filter every month. Is there another way I can do this? I don't want to limit the results to "last 30 days". Rather, January, December etc
Thanks!
Comment: Would [aggregation](http://drupal.org/node/1398368) help here? Or a [PHP Filter Criterion](http://drupal.org/project/viewsphpfilter)?
Answer: Take a look at the "archive" example view, which is provided by default (though disabled) as part of the views module.  It can be found under "Archive" on the admin/structure/views page.
This view groups the content created on the site by year and month; the contextual filter, filter criteria, and sort criteria are already configured and working correctly.  Study this example view and hopefully it will help you create what you want to do.
Comment: wow, after messing for half an hour with this I finally found the solution! Thanks a ton!!!


Soft Delete Nhibernate

Question: I want to make a soft delete on my db table...i have apply following statement (as described here http://nhibernate.info/blog/2008/09/06/soft-deletes.html and in a lot of question on SO).
Fattura is my table where i want apply logical delete (there is no trigger on it)
Fattura.hbm.xml
<?xml version="1.0" encoding="utf-8"?>
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" assembly="Paggentola.Gestionale.DL.Model"   namespace="Paggentola.Gestionale.DL.Model">
  <class name="Fattura" table="Fattura"  where="Cancellato=0">
    <id name="Id_Fattura" column="Id_Fattura">
      <generator class="native" />
    </id>
    <property name="Tipo_Fattura" column="Tipo_Fattura" />
    <property name="Cancellato" column="Cancellato" />
  </class>
</hibernate-mapping>

Fattura.cs - Model Object
using System;

namespace ModelObject
{
    public class Fattura : ISoftDeletable
    {
        public virtual int Id_Fattura { get; set; }
        public virtual Int16 Id_Tipo_Fattura { get; set; }
        public virtual Int16 Cancellato { get; set; }
    }
}

Delete event Listener
    public class MyDeleteEventListener : DefaultDeleteEventListener
    {
        protected override void DeleteEntity(IEventSource session, object entity,
            EntityEntry entityEntry, bool isCascadeDeleteEnabled,
            IEntityPersister persister, ISet transientEntities)
        {
            if (entity is ISoftDeletable)
            {
                var e = (ISoftDeletable)entity;
                e.Cancellato = 1;

                CascadeBeforeDelete(session, persister, entity, entityEntry, transientEntities);
                CascadeAfterDelete(session, persister, entity, transientEntities);

            }
            else
            {
                base.DeleteEntity(session, entity, entityEntry, isCascadeDeleteEnabled,
                                  persister, transientEntities);
            }
        }
    }

Configured in that way
configuration.SetListener(ListenerType.Delete, new MyDeleteEventListener());

The event fire but it doesn't set Cancellato =1.
This is my DeleteCommand
   public void Delete(T entity)
    {
        NHibernateSession.Delete(entity);

        NHibernateSession.Flush();
    }

Comment: you need to make `transaction.Open` and `commit` without `commit` it's won't save anything to your db
Comment: Sure! Don't work...(p.s. Flush must be same effect..)
Comment: hard delete in nhibernate got build in transaction or as we speak out of the box
Comment: wrong because hard delete works fine...
Comment: Agreed. Have you tried with the transaction in your Delete method?
Answer: There is an error on my mapping file. Property
 <property name="Cancellato" column="Cancellato" />

haven't same name in my database column. In that way it's not updated. I just don't understand why at runtime there is no error...
I have set same name and work fine.

Soft Delete Nhibernate


Conditional Expectations & Law of Total Expectation

Question: I have a question about conditional expectations. In my problem, $X$ and $Y$ and independent standard normal random variables, and $Z = X^3 + Y^2$. I am asked to find $E(Z|X)$.
I was thinking of using the tower property of conditional expectations (or the law of total expectation) to find that:
$$
E(Z) = E[E(Z|X)]
$$
where
$$
E(Z) = E(X^3+Y^2) = E(X^3) + E(Y^2) = 1
$$
using the fact that for standard normal variables, $E(X^3) = 0$ and $E(Y^2) = 1$. This then implies that:
$$
E(Z) = E[E(Z|X)] = 1
$$
which then implies that $E(Z|X) = 1$.
I'm not entirely sure if my approach is "too easy" or if there's an appropriate way of calculating this conditional expectation. Any hints or suggestions would be greatly appreciated.
Comment: you cannot do this…always remember expectation is a number, but conditional expectation is a function of random variable x, so also random variable. if the mean weight of men is 200, you cannot say everyone weights 200
Answer: just write it out:
$$
E[Z\mid X]=E[X^3+Y^2\mid X]=E[X^3\mid X]+E[Y^2\mid X]
$$
the first term is simply $X^3$ for $E[h(X)\mid X]=h(X)$, the second one is $E(Y^2)=1$ by independence.
Comment: That's using the linearity property of expectations, right? I'm sorry if this is a very trivial question. It's been a few years since I've dealt with mathematical statistics. Thanks for your help.
Comment: you mean the first step? yeah, linearity of conditional expectations. and its ok,  maybe reviewing wikipedia is a good way to catch up


Hands of the clock, Revisited.

Question: It has already been answered (here) that it is impossible for the (continuously moving) hands of a clock to trisect the face of said clock. Even ideally the hour, minute, and second hand can never pairwise form 120 degree angles with each other. Can we find the time of day where they most nearly do?
To put a metric on this one could take the area of the smallest sector divided by the largest sector. (One being the unatainable optimum.) Or add up the amount by which the three angles are off.
$$\left|\frac{2\pi}{3} - \measuredangle H M\right| + \left|\frac{2\pi}{3} - \measuredangle H S\right| + \left|\frac{2\pi}{3} - \measuredangle M S\right|$$
where zero is now the unatainable optimum.
Comment: You could could aim to maximize the minimum angle.  These... might all be the same thing.  idunno.
Comment: Yeah, I don't know either.
Comment: Related (but not at all the same!) question: http://math.stackexchange.com/questions/397906/is-it-possible-that-a-clocks-three-hands-divide-the-clock-face-into-3-equal-par/397926#397926
Comment: @DavidK I put that link in my post!
Comment: Oops, sorry! My comment is redundant, then. I'll happily delete it if so desired.
Answer: Lemma.
$$
\left| e^{i\,\alpha} - e^{i\,\beta} \right|^2 = \\
\left| \; \left[ \cos(\alpha) + i\,\sin(\alpha)\right] - \left[\cos(\beta) + i\,\sin(\beta) \right] \; \right|^2 = \\
\left[ \cos(\alpha) - \cos(\beta) \right]^2 + \left[ \sin(\alpha) - \sin(\beta) \right]^2 = \\
\cos^2(\alpha) - 2\cos(\alpha)\cos(\beta) + \cos^2(\beta) +
\sin^2(\alpha) - 2\sin(\alpha)\sin(\beta) + \sin^2(\beta) = \\
2 - 2\cos(\alpha-\beta)
$$
Least Squares Method. Two possibilities:
$$
\left| e^{i(\theta+2\pi/3)} - e^{i12\theta} \right|^2 +
\left| e^{i(\theta-2\pi/3)} - e^{i720\theta} \right|^2 = \mbox{minimum}(\theta) \\
\left| e^{i(\theta-2\pi/3)} - e^{i12\theta} \right|^2 +
\left| e^{i(\theta+2\pi/3)} - e^{i720\theta} \right|^2 = \mbox{minimum}(\theta)
$$
Whatever minimum is the smallest. With the lemma:
$$
2 - 2\cos(\theta+2\pi/3-12\theta) + 2 - 2\cos(\theta-2\pi/3-720\theta) = \mbox{minimum}(\theta) \\
2 - 2\cos(\theta-2\pi/3-12\theta) + 2 - 2\cos(\theta+2\pi/3-720\theta) = \mbox{minimum}(\theta)
$$
Brute force algorithm with sampling $\Delta\theta = 2\pi/(60\times 720)$ :

program klok;
function minimum(theta : double; teken : integer) : double;
begin
  minimum := 2 - 2*cos(theta+teken*2*pi/3-12*theta)
           + 2 - 2*cos(theta-teken*2*pi/3-720*theta);
end;
function normal(theta : double) : double;
var
  OK : boolean;
  v : double;
begin
  v := theta;
  OK := false;
  while not OK do
  begin
    OK := (0 <= v) and (v < 2*pi);
    v := v - 2*pi;
  end;
  normal := v + 2*pi;
end;
procedure brute_force(teken : integer);
var
  k : integer;
  M,min,p,w : double;
begin
  min := 8; w := 0;
  for k := 0 to 43200-1 do
  begin
    p := k*2*pi/43200;
    M := minimum(p,teken);
    if min > M then
    begin
      w := p;
      min := M;
    end;
  end;
  Writeln('Minimum =',min);
  Writeln('H =',w,' <',w+2*pi/43200);
  Writeln('M =',normal(12*w),' =',w+teken*2*pi/3);
  Writeln('S =',normal(720*w),' =',w-teken*2*pi/3);
  Writeln;
end;
begin
  brute_force(+1);
  brute_force(-1);
end.

Output (in radians):

Minimum = 1.32888417836478E-0004
H = 3.04690854166910E+0000 < 3.04705398577343E+0000
M = 5.14697596413128E+0000 = 5.14130364406230E+0000
S = 9.42477796078314E-0001 = 9.52513439275905E-0001
Minimum = 1.32888417839328E-0004
H = 3.23627676551049E+0000 < 3.23642220961482E+0000
M = 1.13620934304831E+0000 = 1.14188166311729E+0000
S = 5.34070751109978E+0000 = 5.33067186790368E+0000
Differentiation of the minimum functions, resulting in two equations:
$$
11\sin(11\theta-2\pi/3) + 719\sin(719\theta+2\pi/3) = 0 \\
11\sin(11\theta+2\pi/3) + 719\sin(719\theta-2\pi/3) = 0
$$
Solving these with MAPLE, resulting in numerical refinement of the above:

Digits := 50;
fsolve(11*sin(11*theta-2*Pi/3)+719*sin(719*theta+2*Pi/3)=0,theta=3.0469..3.0470);
fsolve(11*sin(11*theta+2*Pi/3)+719*sin(719*theta-2*Pi/3)=0,theta=3.2362..3.2364);

Giving (Hours hand in radians):

3.0469223755142191756046177765785671073381009266460
3.2362629316653673013206689899804386610562378721043

Don't know which one of the two is the best. It looks like the minima are exactly the same, namely:

0.0000339325557414428099817512048046050292710528311
Comment: Looks like some good analysis. Let me just look it over first.
Comment: Looking at all possible minima, I think I found a lower value. See my answer below.
Comment: Note that your two solutions, $\theta_1=3.0469223...$ and $\theta_2=3.236262...$, sum to $2\pi=6.283185...$.  This is as it should be, and explains why the minima are exactly the same:  If you run the clock to a minimum and then look at it in a mirror, you'll see an identical minimum, but as if the clock were running backwards.  So if a minimum occurs at time $\theta_1$, it also occurs at time $-\theta_1$, hence also at (positive) time $\theta_2=2\pi-\theta_1$.
Comment: @BarryCipra: Thanks for a crystal clear explanation of the phenomenon that $F_+ = F_-$ (adopting amcalde's notation).
Comment: @amcalde: For such small differences: segment length $\approx$ arc length. Thus the square root of the minimum divided by $2\pi$ (and multiplied by $100$) is an estimate for the relative error in the $120$ degrees configuration. $\approx 0.1 \%$ is found with my own answer; this is even better, of course, with the improved answer by yourself, namely $\approx 0.05 \%$ .
Comment: The (excellent) analysis by amcalde shows that it's easy to miss quite a few minima with the above brute force sampling method. In retrospect, I could have been aware of that :-(
Comment: Indeed. In the above program, using instead a $60 \times$ refined sampling ($43200 \to 2592000$) and invoking MAPLE for finding the root in the neighbourhood gives the better approximation as published by amcalde.
Answer: Expanding on Han de Bruijn's work I think we can do better. (To be gentlemanly, I'll award the bounty.)
the two functions he sets out to minimize are:
$F_\pm(\theta) =  4 - 2\cos(\frac{2 \pi}{3} \mp 11 \theta)-2\cos(\frac{2 \pi}{3} \pm 719 \theta)$
Differentiating and simplifying, we find that the extrema of these functions are found at the roots of
$G_\pm(\theta) = 719 \cos(\frac{\pi}{6} \pm 719\theta)-11\sin(\frac{\pi}{3}+11\theta)$
There are 1437 such roots for each of $G_\pm$ over the interval $[0\le \theta<2\pi]$. Using Mathematica I found each one to high accuracy and substituted back into $F_\pm$. I found a unique minimum for $F_\pm$. I also found that the minima were the same but got a lower value:
{$\min F_\pm$, $\theta$} = { $8.483 \times 10^{-6}$, $1.52346119$ (radians of hour hand)}
So this happens at about $1.52346119*720/(2\pi) = 174.575$ minutes
Or at 02:54:34.54
Comment: (+1) for this nice improvement. Thanks for the bounty anyway.
Comment: Why are we using trig functions so much?  The angle variation, sector size, etc, they're all piecewise linear...
Comment: What exactly is being minimized here, the sum of the absolute values of the errors, or the sum of their squares?
Comment: @BarryCipra The absolute difference (complex norm) between the vector positions of the hands of the clock and where they should be if they were to trisect the face. This is just one of many possible measures.
Comment: @amcalde, it looks to me like you're minimizing the sum of the squares of the errors here.  In general, different minimizations give (slightly) different answers.
Comment: @DanUznanski: Trig functions pop up spontaneously because the clock is represented mathematically by that beautiful complex function $e^{i\theta}$ . But if you can come up with something simpler and "piecewise linear", please be my hero and feel free to formulate another answer.
Comment: @BarryCipra: The Least Squares norm has the enormous advantage that you can do calculus (differentiate) to find the minima (many more of these than I expected, though).
Comment: @HandeBruijn, agreed, Least Square, aka the $L^2$ norm, has its advantages.  It's just that the metric cited in the OP's displayed equation is an $L^1$ norm.  I think that's what Dan Uznanski was referring to.  Roughly speaking, the clock will pass through a possibly global minimum twice per hour (when the minute hand is more or less $20$ minutes past or before the hour hand and when the second hand is more or less $20$ seconds past or before the minute hand).  If you locate the time intervals where these occur, the linearity Dan notes may prove easier to work with.
Comment: @BarryCipra: I would say: let someone just do it. I'm truly curious to see how that may prove easier to work with.

Hands of the clock, Revisited.


Is it possible to make a confidence envelope for a two sample Q-Q plot in R (or Python)? If so, what is the simplest method?

Question: I want to show the confidence envelope for a two sample Q-Q plot in R (or Python). The aim is to use the Q-Q plot to give an indication of whether my two samples are drawn from the same population
The method of qqPlot() from the car package cannot do this, because this method does not support two sample tests, I think. The method of qqplot() from the base R installation cannot do this by itself, since it does not support confidence intervals, but it does support two samples.
Is it possible to use Kolmogorov-Smirnov or bootstrapping for this?
Thank you for your time!
Edit: I am hijacking my own question.
Part of the background for why I am asking is because I feel the Kolmogorov-Smirnov test (KS) and Anderson-Darling test (AD) give me misleading results for my comparisons. In my research I am interested in answering whether a factor might influence the size distributions of particles in my material. An example of KDEs, CDFs, Q-Q plot, and KS and AD test results for graphite particles are provided below (everything is calculated and plotted in Python):
KS p-value: 6.58*10^-6.
AD p-value: 0.001.
Number of particles counted: approximately 1000. (I know that for huge sample sizes [~5000] goodness of fit tests such as KS tend to give very low p values even if the deviation from normality is quite low (for one sample tests)).
KDE of sample 1:

KDE of sample 2:

CDF of samples:

Q-Q plot of samples, made in SciPy by first obtaining the quantiles with the statsmodels.graphics.gofplots.ProbPlot function, then plotting them using statsmodels.graphics.gofplots.qqplot_2samples:

I guess my second question is: Is my Q-Q plot and KS test result in conflict with each other? Should I focus on the Q-Q plot and disregard KS?
Edit: The culprit for the apparent conflicting results was a mistake when plotting the Q-Q plot. The Q-Q plot matches the KS results now. Thank you for the nice suggestion regarding permutations!
Comment: Re the second question: the qq plot is the visual depiction of the KS test, so they will never conflict with each other.  If they seem to, then most likely you are not employing the KS test correctly.
Answer: Bootstrapping will do nicely.  So will a permutation test, which is a little more attractive because it maintains more qualitative features of the distributions (such as an absence of ties for continuous data generation processes).
Given two datasets $x$ and $y$ as vectors of lengths $m$ and $n,$ respectively, the permutation test samples $m$ values without replacement from the concatenation of $x$ and $y$ and draws a QQ plot of that sample against its complement.  Here is what it might look like when the null hypothesis is true (both datasets are random samples of the same distribution).  The sizes in this example are $30$ and $20.$

This shows the original data in red along with 49 additional permutation samples.  The data lie well within the envelope.
The same plot in the alternative case shows what it might look like when the data are samples of different distributions:

The data are at the lower envelope, indicating $y$ tends to be dominated by $x.$
The calculations are fast and easy to do in R, especially if you exploit the built-in qqplot function to compute the plotting coordinates.
#
# Create some data.
#
set.seed(17)
a <- 1
b <- 2
x <- runif(30)
y <- rbeta(20, a, b)
#
# Sample from the permutation distribution of QQ plots.
# The first time through gives the QQ plot for the data themselves.
#
xy <- c(x, y)
plot.lst <- lapply(1:50, function(j) {
  if (j==1) i <- 1:length(x) else i <- sample.int(length(xy), length(x))
  X <- as.data.frame(qqplot(xy[i], xy[-i], plot.it = FALSE))
  X$Iteration <- j
  X
})
X <- do.call(rbind, plot.lst) # QQ plot coordinates are "x" and "y".
#
# Plot this sample.
#
library(ggplot2)
ggplot(X, aes(x,y, group = Iteration)) +
  geom_path(color = gray(0.65)) +
  geom_path(color = "Red", size = 1.25, data = subset(X, Iteration == 1)) +
  ggtitle(if(a == 1 && b == 1) "Null is True" else "Null is False")


Rodapé não acompanha layout do site e não fica responsivo

Question: Galera, boa tarde.
Estou com dificuldades de deixar o rodapé responsivo, ele não acompanha a página. Conseguem auxiliar através desse código que fiz?
(Tentei implementar algumas dicas que achei por aqui mas não deu certo)
Grato.

/*Edição do rodape (Footer)*/
footer {
 background-image: url(../img/fundo-rodape.png);
 clear: both;
 padding: 20px 0;
}

footer .container {
 position: relative;
 width: 900px;
}

.social{
 position: absolute;
 top: 12px;
 right: 0;
}

.social li{
 float: left;
 margin-left: 25px;
}

.social a{
 /*tamanho da imagem*/
 height: 32px;
 width: 32px;

 /*image replacement*/
 display: block;
 text-indent: -9999px;
}

.social a[href*="facebook.com"]{
 background-image: url(../img/facebook.png);
}

.social a[href*="twitter.com"]{
 background-image: url(../img/twitter.png);
}

.social a[href*="plus.google.com"]{
 background-image: url(../img/googleplus.png);
}
/*Edição do rodape (Footer)*/

Facebook

Twitter

Google+

Answer: É porque você definiu uma largura fixa na
, assim ela nunca irá acompanhar a largura da tela, porque terá sempre a largura de 900px.
Em vez de colocar width: 900px;, altere para width: 100%; e coloque max-width: 900px; e width: 100%; no :
footer {
    background-image: url(../img/fundo-rodape.png);
    clear: both;
    padding: 20px 0;
   background-color: red;
    max-width: 900px;
   width: 100%;
}

footer .container {
    position: relative;
   width: 100%;
}

Exemplo:

/*Edição do rodape (Footer)*/
footer {
 background-image: url(../img/fundo-rodape.png);
 clear: both;
 padding: 20px 0;
   background-color: red;
 max-width: 900px;
   width: 100%;
}

footer .container {
 position: relative;
   width: 100%;
}

.social{
 position: absolute;
 top: 12px;
 right: 0;
}

.social li{
 float: left;
 margin-left: 25px;
}

.social a{
 /*tamanho da imagem*/
 height: 32px;
 width: 32px;

 /*image replacement*/
 display: block;
 text-indent: -9999px;
}

.social a[href*="facebook.com"]{
 background-image: url(../img/facebook.png);
}

.social a[href*="twitter.com"]{
 background-image: url(../img/twitter.png);
}

.social a[href*="plus.google.com"]{
 background-image: url(../img/googleplus.png);
}
/*Edição do rodape (Footer)*/

Facebook

Twitter

Google+

Comment: Certo, eu tirei largura fixa e ficou ok quando executa no mobile, mas desktop continua, a margem ultrapassa o layout, entende?
Comment: Editei a resposta. Veja se era isso.
Comment: Agora o rodapé ficou no tamanho correto, porém ainda ele não alinha. Não consigo postar foto mas vou tentar explicar. Meu site tem um padrão de 940px (esse padrão foi solicitado), então ele não ocupa 100% da tela, o rodapé agora está correto fixo em baixo e o tamanho correto, porém ele distorce a página devido não centralizar junto ao resto do layout. Deu pra entender? Agradeço desde já sua ajuda.
Comment: Coloca `margin: 0 auto;` no footer.
Comment: Cara, ia comentar isso agora, um amigo sugeriu isso também e deu certo. Agradeço muito a ajuda. Valeu !!!
Comment: Show de bola!!!


How to detect user activity/inactivity in Android service/background application?

Question: How to detect user activity/inactivity in android service/background application - from same or other applications?
I want to know that user stop touching screen and not busy with another activity to show notifiation popup. Not sure is there no other method than use AccessibilityService.
Comment: did you find any solution?
Answer: One of the ways is to know that from reading the Android logs
Answer: Define a interface in activity and service.
in oncreate of activity set reference to activity in service method.
Now you have activity reference for entire service lifetime.
another way to do this is, application class, same application class is shared by activity and service, so you can both of these from each other using application class.
If you want see from applications then use broadcastreceivers that is the only wau
Comment: I need to see touches from other applications (not other activities of same application)?
Answer: Use a boot receiver to setup the AlarmManager (and of course also check to start the polling from your main Activity too, for the case when your app is installed and the system is not booted) and have the AlarmManager send an Intent for another receiver.
I think they answered it in this question to do what you are looking for:
android: running a background task using AlarmManager
Comment: It will not work if user touch activity in other application - I think so - what do you think?


O que tem de errado com esse meu código Python?

Question: Estou tentando criar um projeto simples para fins de estudos mas estou sentindo dificuldades em conclui-lo. Estou usando a biblioteca Pytube para baixar videos, consigo baixar os vídeos mas não consigo mostra a barra de progresso do download e o tamanho do arquivo (vídeo) a ser baixado.
Meu código é esse:
from pytube import YouTube
from pprint import pprint
from pytube import utils
from pytube.models import *

#Pede url do vídeo
url_video = str(input("URL do vídeo: "))

#Instancia a Classe.
yt = YouTube(url_video)

#Mostra o título do video + formato e resolução
print("\nTitulo do vídeo: ", yt.filename)
print("Formatos e Resoluções")

#Vai percorrer a Classe yt.get_videos formatos do vídeo e o tipo.
for i in yt.get_videos():
        print(i)

formato = str(input("\nQuais dos formatos a cima você deseja baixar seu vídeo(ex: .mp4 .3gp ): "))
resolucao = str(input("Digite qual das resoluções existente no video que você deseja baixar(ex: 720p): "))

print("\nVocê deseja renomear o nome do vídeo? Caso você diga queira o vídeo sera salvo com o nome padrão.\n")

perg = int(input("(1-Sim / 2-Não): "))

if perg == 1:
    nome = str(input("Digite o nome: "))
    yt.filename = nome

video = yt.get(formato, resolucao)
caminho = str(input("Caminho(ex: /usuario/Downloads/)>> "))
video.download(caminho)

Dei uma lida na documentação da biblioteca e fui até os módulos utils e models.
no módulo Utils diz o seguinte:
Def  print_status ( progress , file_size ):
    -benzóico.
    Esta função - quando transmitida como `on_progress` para` Video.download` - imprime
    O progresso de download atual.
    Argumentos:
    Progress - O comprimento dos bytes atualmente baixados.
    File_size - O tamanho total do vídeo.
    -benzóico.
    % = Progresso *  100 . / File_size
    Status =  r " % 10d   [ % 3.2f %% ] "  % ( progresso, porcentagem )
    Status = status +  chr ( 8 ) * ( len (status) +  1 )
    print status,

e no Models:
def download(self, path=None, chunk_size=8*1024,
             on_progress=None, on_finish=None):
    """
    Downloads the file of the URL defined within the class
    instance.
    Keyword arguments:
    path -- Destination directory
    chunk_size -- File size (in bytes) to write to buffer at a time
                  (default: 8 bytes).
    on_progress -- A function to be called every time the buffer was
                   written out. Arguments passed are the current and
                   the full size.
    on_finish -- To be called when the download is finished. The full
                 path to the file is passed as an argument.
    """

    path = (normpath(path) + '/' if path else '')
    fullpath = '%s%s.%s' % (path, self.filename, self.extension)
    response = urlopen(self.url)
    with open(fullpath, 'wb') as dst_file:
        meta_data = dict(response.info().items())
        file_size = int(meta_data.get("Content-Length") or
                        meta_data.get("content-length"))
        self._bytes_received = 0
        while True:
            self._buffer = response.read(chunk_size)
            if not self._buffer:
                if on_finish:
                    on_finish(fullpath)
                break

            self._bytes_received += len(self._buffer)
            dst_file.write(self._buffer)
            if on_progress:
                on_progress(self._bytes_received, file_size)

mas mesmo assim lendo não compreendi muita coisa. Alguém pra me ajudar?
Comment: No primeiro `for` existe um erro de indentação, mas acredito que isso ocorreu apenas quando postou o código aqui, senão nem baixaria o vídeo.
Comment: Tente fazer `video.download(caminho, on_progress=utils.print_status)`
Comment: Me retorno esse erro:  Traceback (most recent call last):
  File "C:\Users\RaquelCristina\Desktop\Estudos - Python\Projetinhos\BaixarVideosYoutube.py", line 34, in
    video.download(caminho, on_progress=utils.print_status)
  File "C:\dev\kivy\venv\lib\site-packages\pytube\models.py", line 77, in download
    raise OSError("Conflicting filename:'{0}'".format(self.filename))
OSError: Conflicting filename:'teste'
Comment: `Conflicting filename:'teste'`. Parece que está tentando salvar um arquivo que já existe (com mesmo nome).
Comment: Agora foi, eu coloquei assim: video.download(caminho, on_progress=print_status)
Comment: Obrigado pelo força!
Answer: Veja a documentação da função download:
"""
Downloads the file of the URL defined within the class
instance.
Keyword arguments:
path -- Destination directory
chunk_size -- File size (in bytes) to write to buffer at a time
              (default: 8 bytes).
on_progress -- A function to be called every time the buffer was
               written out. Arguments passed are the current and
               the full size.
on_finish -- To be called when the download is finished. The full
             path to the file is passed as an argument.
"""

Existe um parâmetro chamado on_progress, que é uma função que será chamada sempre que o buffer for gravado em disco. Esta função receberá dois parâmetros: o primeiro refere-se ao status atual da transferência e o segundo o tamanho total do arquivo.
A própria biblioteca já dispoem de uma função para isso, que é justamente a print_status, então basta você fazer o seguinte:
video.download(caminho, on_progress=print_status)

Se precisar de algo mais específico, você pode criar essa função manualmente:
def my_print_status (current, fullsize):
    print("Transferido {} de {}".format(current, fullsize))

Talvez será necessário a formatação dos parâmetros conforme a necessidade, mas a ideia é essa. A chamada de download seria:
video.download(caminho, on_progress=my_print_status)


How to find Chebyshev nodes

Question: I want to use Chebyshev interpolation. But I am a little confused for finding Chebyshev nodes. I use the following figure to illustrate my problem. Consider I have a vector of numbers I depicted as a line In "A". In "B", the red points are the chebyshev nodes. How can i choose these points? (I have used the picture to say that I know that Chebyshev try to choose more points at the ends)
Comment: In my problem, i should have nodes at the ends of my vectors too. is it possible in chebyshev nodes?!
Answer: For $n\;$ nodes $x_k\;$ in an interval $[a,b]\;$ other than $[-1,1]\;$ use the formula from http://en.wikipedia.org/wiki/Chebyshev_nodes:
$$x_k = \frac{1}{2}(a+b) +\frac{1}{2}(b-a) \cos\left(\frac{2k-1}{2n}\pi\right), \quad k=1\dots n$$
Comment: Sorry, I still do not know what you want. I cannot see any vector, not to say a
_beginning_ or _end_? Do you mean, that the end points of the interval should
be interpolation nodes? Then it can be no Chebyshev interpolation, or you have to specifically taylor a somewhat larger interval for a fixed value of n.
Comment: I have used it, but as it seems, choosing nodes starts from the center of the vector, but in my problem i should have nodes at the ends of my vector. is it possible in chebyshev nodes?
Comment: I am not sure what you mean, the nodes are in descending order. If you need them in ascending order use
$$x_k = \frac{1}{2}(a+b) +\frac{1}{2}(b-a) \cos\left(\frac{2(n-k)-1}{2n}\pi\right), \quad k=1\dots n$$
The interpolation should be independent of the labeling or sequence of the nodes (apart from small rounding errors).
Comment: As i depicted in the figure, can i have the nodes as illustrated? i mean at the beginning and the end of the vector of numbers
Comment: @gammatester should tha be 2(n-k)+1 / 2n * pi ?  (plus 1, not minus one) in your comment above?
Comment: @lamont: Yes, you are right. Thank you. Unfortunately I can not edit the comment.
Comment: You can also use a change of variables to translate the interval $[a,b]$ to $[-1,1]$ and then you can just use the more compact formula to find the Chebyshev nodes
Answer: Chebyshev nodes are the roots of chebyshev polynomials. Use the definition $$T_n(x) = \cos(n\arccos(x))$$ for the Chebyshev polynomials.
Comment: I want to find the nodes. by some equations, i can find the nodes between [-1,1], but how can i transfer these to the range of numbers i have?

How to find Chebyshev nodes


what is this colon in jquery after src?

Question: I was wondering anyone can explain this one line in the code for me.
        $(this).bind('click', {src: $(this).attr('href')}, function (e){
           //commented out since it's irrelevant
        });

Here is how I understand it:
Bind a function to $(this)'s click event. Also the e parameter is passed to the function too. However it's the middle part that doesn't make sense to me
                     {src: $(this).attr('href')}

What does this do? I tried google'ing what colon does in jquery. However I get the li:first results.
Answer: It's javascript object notation:
var object = {propertyName : propertyValue};

It describes an object literal.
Comment: to specify what {src: $(this).attr('href')} is...
i believe it is returning to you the value of the clicked links href. this would give you a url most likely.
You would access this by using dot notation obj.src would give you the link
Comment: @axrwkr this creates a new object with a new set of properties.. If you mean as far as the event data, it gets stored on the `eventObject.data` property to avoid those kinds of issues.
Answer: {src: $(this).attr('href')} is an object literal, you are passing an object into the .bind function as the second parameter. Doing so passes said data into the event handler where you can access it when the event happens using event.data.src or, e.data.src in your case since use used function (e)...
Answer: That's the way you define an object ( called an object literal ). For example if:
obj = { src: "Source" };

then obj.src would contain "Source".
Answer: You're supplying an inline object with a property src. src has a value of $(this).attr('href'); The reason you're not able to find jquery specific documentation is that this is a convention that exists in javascript itself.
Answer: From the docs for bind, second argument:
eventData
    Type: Object
    An object containing data that will be passed to the event handler.

So, you're passing an object.  The src: syntax means a member of that object whose name is src.
Answer: This is just aditional information to be passed to the callback, in the event object, and can be accessed using:
// callback function
function(e){
    alert(e.data.src);
}

what is this colon in jquery after src?


Threads communicating without locking

Question: If I can guarantee myself that only one method in my entire app will ever write to a certain variable, then may I allow other methods in my app to safely read that value ?
If so, can I get away that stunt without locking the variable ?
In this context, what I'm doing (or, trying to do, or want to do) is for one method in one thread to put a value into the variable, and then other methods in other threads will read that value and make decisions.
A very nice option would be to lock against writes, while allowing reads.
Looked here MSDN page on lock and didn't see a way to do that.
Comment: Read while writing is not the best idea. You can read mess, when object is already updating.
Comment: Have a look at [`ReaderWriterLock`](http://msdn.microsoft.com/en-us/library/system.threading.readerwriterlock.aspx)
Comment: @EricPetroelje: See Marc Gravell's answer. The problem is when an update in not atomic. For example, if the updated value is an int, then no problem. If the updated value is a struct made up of several ints (for example, a point class), then it could be a real problem. This is the "tearing" Marc refers to.
Comment: @Eric **changing a reference** is indeed guaranteed to be atomic and safe; the language guarantees that. If, however, the code updated **properties** of an existing object: bad things can happen without synchronisation. If you have a **reference** to an **immutable** class, then you are pretty safe to change the reference at any time... **BUT** all callers must **PROMISE** to take a snapshot of the reference **ONCE** and then use the snapshot. No `int x = Foo.X; int y = Foo.Y` - it has to be `var snap = Foo; int x = snap.X; int y = snap.Y;`
Comment: @Kamil Yes, correct. I have a second check on a different item after I read this one.
Comment: @Kamil - out of curiosity, do you have a source for that?  I would think a reader could get either the new value or the old value but not some kind of "in-between" value.
Comment: or indeed [Joe Duffy's Weblog](http://www.bluebytesoftware.com/blog/2006/02/08/ThreadsafetyTornReadsAndTheLike.aspx) /cc @Matt
Comment: @Matt - ok, that makes perfect sense.  Seems that most "simple" types or references are safe though (as per the C# spec)
Comment: @EricPetroelje its possible, that some bigger object, like array or struct may be read as "something between old and new". I don't have source for that information, its just logical for me.
Comment: @MarcGravell - thanks, that pretty much matches my understanding of things
Comment: @MarcGravell: Interesting link. I don't think it would have occurred to me that updating a long might not be atomic.
Comment: @EricPetroelje, this part "..._For example, if the updated value is an int, then no problem_...." was important, and I hope correct, because my stuff over here just worked 9,327 times before another bug elsewhere stopped the entire app. This was two separate threads in two separate classes writing and reading an int in a third separate class. Current state of things here: Hooray.  `Thread_A_Method_A_Class_A` does this: `Third_Class.TheCounter++;` while `Thread_B_Method_B_Class_B` does this: `CurrentCount = Third_Class.TheCounter;` and then uses `CurrentCount` to inform the user.
Comment: @MattBurland, ooops, my previous comment was supposed to be at you; sorry, still learning to navigate these waters.
Answer: As always, it depends a lot on the context.

a variable read in a tight loop may be stored in a register or local cache, so no change will be noticed unless you have a "fence"; volatile will fix this, but as a side-effect rather than by explicit intention; most people (including me) can't properly define what volatile means - so be very careful of using it as a "fix".
an oversize type (large struct) will not be atomic (for either read or write) - and cannot be handled safely without risk of tearing
an object or value might involve multiple sub-values; if they aren't changed atomically, it could cause problems

You might, however, find that Interlocked solves most of your problems without needing a lock. At the same time, an uncontested lock is insanely fast, and even a contested lock is still alarmingly fast. Frankly, I'm not sure that it is worth the thought you are giving it: a flat lock is almost certainly fast-enough, as long as you do the thinking first outside the lock, and only lock it when you know the changes you want to make.
There is also ReaderWriterLockSlim, but the number of cases where that actually improves performance is slim - in my experience, the simplest approach possible is usually the fastest, meaning either lock or Interlocked. ReaderWriterLockSlim is a more complex beast, designed for more complex scenarios, and has a little overhead because of it. Not massive amounts, but enough to make it worth looking carefully.
Comment: @User.1 `lock`===`Monitor`, not `ReaderWriterLockSlim`, but that's easy: to get it: `lock(someLock) { return theField; }`; to set it: `lock(someLock) { theField = value; }`
Comment: @User.1 yes, about 2 lines above, or on MSDN ;p
Comment: @EricPetroelje Yes, it applies, because he added the qualification "unless you add a fence", which is exactly what `volatile` does.
Comment: +1 for ... "an uncontested lock is insanely fast, and even a contested lock is still alarmingly fast."
Comment: I just spent awhile on [The MSDN lock page](http://msdn.microsoft.com/en-us/library/c5kehkcz(v=VS.100).aspx) and [The MSDN ReaderWriterLockSlim page](http://msdn.microsoft.com/en-us/library/system.threading.readerwriterlockslim.aspx) where lot of my time was mainly just _DUH_ ! Would you (or anyone reading) be willing to demonstrate the proper syntax and invocations of the `lock` mechanism for a variable  and three methods in three classes that can read it and write it ? Call them whatever you like;  `Comm.BackGroundRx, SomeButtonClick, xClass.Thread3,  MarcsClass.anyOldVariable` whatever.

Threads communicating without locking


Data is showing in console but not in screen - reactjs

Question: why am i not seeing the data in my screen when I map over "this.state.viciados" if i can see the data in the console? I have seen this problem in stack overflow but the only answer in that qustion doesn't work.
class App extends Component {
  constructor(props) {
super(props);
this.state = {
  viciados: []
}

}

componentDidMount() {
this.getViciados();
}

 getViciados = _ => {
fetch("http://localhost:3000/dependentes")
  .then(response => response.json())
  .then(response => {
    this.setState({ viciados: response.data })
    //console.log(response.data);
  })
  .catch(err => console.log(err));
}

render() {

 const viciados = this.state.viciados.map(viciado => {
  console.log(viciado['id_viciados'], viciado["nome_viciado"]);

{viciado["nome_viciado"]}

});
return (

    {viciados}

);
}
}

export default App;

Answer: You need to return data from map:
const viciados = this.state.viciados.map(viciado => {
  return
{viciado["nome_viciado"]}

});


Android: Check if a fragment is visible

Question: I have a FragmentActivity, some fragments and a service that is running in background. It is possible to check from the service if a specific fragment is visible?
Comment: It is not really a duplicated because the other question is asked in the context of a ViewPager, and the answers reflect that context. This is a more general question.
Comment: this is not a duplicate.
Comment: The solution that worked best for me was http://stackoverflow.com/a/6751537/1816603
Answer: check if(YourFragment.this.isVisible())
Comment: How to in a Tabbed Action Bar? Navigation is messed up in this fragment scheme.
Comment: How navigation is messed up? What you want to achieve?
Comment: I have an Action Bar tabbed activity, Fragments consist of tabs, I have a button on the Action Bar which once pressed navigates to a different fragment say "Info about the app". As soon as user navigates to info, I disable it so that it does not get called redundantly, then I re-enable it once back is pressed but say if user navigates to info ( info gets disabled until back is pressed) but I want to enable it again if back is not pressed but navigation between tabs occurs.
Comment: So far I tried a number of things like referencing the Menu item in the on start and on resume of tab fragments but gives me a null pointer, I am now writing a detailed script and posting as a Question.
Comment: yeah, good, you must post question with attempted code.
Comment: http://stackoverflow.com/questions/20393016/enable-an-actionbar-button-each-time-navigation-within-tabs-occurs
Answer: public boolean isFragmentUIActive() {
    return isAdded() && !isDetached() && !isRemoving();
}

Does the trick.
Comment: This returns false if i call it from the service, but maybe i'm doing something wrong.
Answer: Check the code below:
public class MyFragment extends Fragment
            @Override
            public void setMenuVisibility(final boolean visible) {
                super.setMenuVisibility(visible);
                if (visible) {
                    ...
                }
            }

It's from this thread. Or you can also check this one.

Android: Check if a fragment is visible


Can Drupal Commerce and Ubercart be installed on the same site?

Question: Can Drupal Commerce and Ubercart be installed on the same site or will they conflict with eachother?
Answer: These two modules define the same menu entries - so you can use only one of them.
Answer: Nope, they can be installed together. In admin menu there will be two tabs Store etc. Maybe there are some conflicts but I have not noticed yet.
Comment: Hello and welcome. Is your answer meant to show that what Eugene wrote is no longer valid?Have you checked it?
Comment: I am checking it right now. It works but maybe there are some issues. In several days I will post my experience.
Comment: After checking some functionalities, I must say that there are some issues with compatibilites of order and cart for users (for an administer it is OK). So, it is not recommended to install these two systems at the same time.


template filter to trim any leading or trailing whitespace

Question: Is there a template filter in django that will trim any leading or trailing whitespace from the input text.
Something like: {{ var.example|trim }}
Answer: Django templates allow you to access methods and properties by using the '.' syntax:
{{ var.example.strip }}

You can extend this by chaining other filters when you're dealing with HTML, e.g.:
{{ var.example.strip|safe|removetags:"p img" }}

Here we first remove any
 and  tags, then tell Django it can safely render the rest of the content, which we have stripped of any whitespace.
Comment: It's not Django function, but Python's. It is documented here: https://docs.python.org/2/library/stdtypes.html#str.strip . Documentation of Django template variables: https://docs.djangoproject.com/en/dev/ref/templates/language/#variables
Comment: one comment - `removetags` filter is being removed as of django 1.10, so be carefull
Answer: You can do it yourself
from django import template
from django.template.defaultfilters import stringfilter

register = template.Library()

@register.filter
@stringfilter
def trim(value):
    return value.strip()

Documentation
Comment: Using `{{ var.example.strip }}` is indeed simpler, however this solution here also has its use. For example it allows you to do `{% filter trim %}{% someothertag %}{% endfilter %}`, which is not otherwise possible.


Standard errors and p-values for interaction effects with GLM

Question: I'm wondering how to calculate the standard errors and p-values of interaction effects in logistic models.
For illustration, consider this toy example, where we think some treatment effect will differ between men and women (but not by education):
set.seed(101)
n <- 2000
dv <- sample(0:1,n,rep=TRUE)
condition <- sample(c("control","treat"),n,rep=TRUE)
gender <- sample(c("male","female"),n,rep=TRUE)
education <- sample(c("less_than_undergrad","undergrad","post_grad"),n,rep=TRUE)

m <- glm(dv ~ condition +
             gender +
             education +
             condition:gender,
           family = "binomial")

Which gives us this:
Coefficients:
                          Estimate Std. Error z value Pr(>|z|)
(Intercept)               -0.07849    0.11065  -0.709   0.4781
conditiontreat             0.22031    0.12658   1.740   0.0818 .
gendermale                -0.04788    0.12558  -0.381   0.7030
educationpost_grad         0.02815    0.10965   0.257   0.7974
educationundergrad         0.04922    0.10981   0.448   0.6540
conditiontreat:gendermale -0.13205    0.17921  -0.737   0.4612
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Here, the treatment effect for female (the reference category) is estimated at 0.22031, with an SE of 0.12658 and a p-value of 0.0818.
The treatment effect for male, by contrast, would be
beta <- 0.22031 - 0.13205 # 0.08826

But what is the SE and p-value for that estimate? I believe it's not the ones given on the final row of the output, since they speak to the difference in effect between male and female.
1. Calculating the SE
From this and this, I gather that the SE should instead be calculated as follows:
$\sqrt{var(\hat{\beta_1}) + var(\hat{\beta_5}) +  2*cov(\hat{\beta_1}\hat{\beta_5})}$
Applied to the case above, that would give us the following
beta_se <- sqrt(vcov(m)[2,2] + vcov(m)[6,6] + 2*vcov(m)[2,6]) # 0.1268722

2. Calculating the p-value
Once I have the SE, I gather from this that I would then simply calculate the p-value like this:
pnorm(-abs(beta)/beta_se) * 2 # 0.4866415

Have I understood all of this correctly?

EDIT: Full code in one block:
set.seed(101)
n <- 2000
dv <- sample(0:1,n,rep=TRUE)
condition <- sample(c("control","treat"),n,rep=TRUE)
gender <- sample(c("male","female"),n,rep=TRUE)
education <- sample(c("less_than_undergrad","undergrad","post_grad"),n,rep=TRUE)

m <- glm(dv ~ condition +
           gender +
           education +
           condition:gender,
         family = "binomial")
summary(m)

beta <- 0.22031 - 0.13205 # 0.08826
beta_se <- sqrt(vcov(m)[2,2] + vcov(m)[6,6] + 2*vcov(m)[2,6]) # 0.1268722
pnorm(-abs(beta)/beta_se) * 2 # 0.4866415

library(multcomp)
summary(glht(m, linfct = c("conditiontreat + conditiontreat:gendermale = 0")))

Which gives:
     Simultaneous Tests for General Linear Hypotheses

Fit: glm(formula = dv ~ condition + gender + education + condition:gender,
    family = "binomial")

Linear Hypotheses:
                                                Estimate Std. Error z value Pr(>|z|)
conditiontreat + conditiontreat:gendermale == 0  0.08825    0.12687   0.696    0.487
(Adjusted p values reported -- single-step method)

Answer: I cannot replicate your estimates with the R code provided, but you can verify your manual calculations by testing the null that linear combination of two coefficients is zero:
set.seed(101)
n <- 2000
dv <- sample(0:1,n,rep=TRUE)
condition <- sample(c("control","treat"),n,rep=TRUE)
gender <- sample(c("male","female"),n,rep=TRUE)
education <- sample(c("less_than_undergrad","undergrad","post_grad"),n,rep=TRUE)

m <- glm(dv ~ condition +
           gender +
           education +
           condition:gender,
           family = "binomial")

write.dta(data, "~/Desktop/data.dta")
summary(m)

names(coef(m))
library(multcomp)
summary(glht(m, linfct = c("conditiontreat + conditiontreat:gendermale = 0")))

The last part yields:
Simultaneous Tests for General Linear Hypotheses

Fit: glm(formula = dv ~ condition + gender + education + condition:gender,
    family = "binomial")

Linear Hypotheses:
                                                Estimate Std. Error z value Pr(>|z|)
conditiontreat + conditiontreat:gendermale == 0  -0.1838     0.1290  -1.425    0.154
(Adjusted p values reported -- single-step method)

This shows that while the effect of treatment on the log odds is negative for males, it is not statistically distinguishable from zero.

Here's the same calculation from first principles:
> (te <- m$coefficients["conditiontreat"] + m$coefficients["conditiontreat:gendermale"])
conditiontreat
    -0.1838191
> (se_te <- sqrt(vcov(m)["conditiontreat","conditiontreat"]
+               + vcov(m)["conditiontreat:gendermale","conditiontreat:gendermale"]
+               + 2*vcov(m)["conditiontreat","conditiontreat:gendermale"]))
[1] 0.1290362
> (pval <- pnorm(te/se_te)*2)
conditiontreat
      0.154286

Comment: I think you are right about the plus. I made a silly algebra mistake and have edited that out. I still can't get your numbers, but the calculations also work out for me. See the edit above. Not sure what's going wrong with R, but I do get the same estimates in Stata as in R on my end.
Comment: Thanks, Dimitriy. Very strange that we're not getting the same numbers. The only difference is that I'm not running `write.dta(data, "~/Desktop/data.dta")`. But at least we're both getting the `glht` and manual method to give the same results on our respective ends, which seems to confirm that the way I was calculating the SE and p-value in the original post seems to have been the correct way of doing it.
Comment: Thank you, Dimitriy! Three things:

(1) Apologies! There was a typo in the p-value calculation. The code should now replicate.

(2) When I run `summary(glht(m, linfct = c("conditiontreat + conditiontreat:gendermale = 0")))` I don't get an estimate of `-0.1838`. Instead I get the numbers I have in my manual calculations: an estimate of `0.08825` with an SE of `0.12687` and p-value of `0.487`.

(3) Are you sure it should be -2*Cov()? From the Wikipedia page it looks like it should be +2 for Var(aX + bY). And the +2 calculation seems to yield the 'correct' SE, as given by (2) above.
Comment: I don't see any edits to the data simulation part, so I still can't replicate your estimates/coefficients. The change to the manual p-value calculation does not alter that fact and our numbers won't line up without that fix. In your case, $b<1$, so the $Var(aX-bY)$ formula applies. Could certainly be that we are missing something.
Comment: Can't figure out why it's not replicating. Have added full code block in original post, which should replicate on copy+paste, and yield a positive estimate.
Answer: I know this question is over 2 years old, but I wanted to comment for anyone else who finds this post useful (as I did). The calculation of the treatment effect for male in this case should include not only the reference level for treatment, but also the effect of sex. In other words:
$$\beta_{male + treatment} = \beta_{male} + \beta_{treatment} + \beta_{male:treatment}$$ (in OP's original calculation, $\beta_{male}$ was missing).
Or, in terms of odds ratios:
$$OR_{male+treatment} = OR_{male}\times OR_{treatment}\times OR_{male:treatment}$$
This also affects the calculation of the SE, as you should include the variance and covariance of the interaction term in the equation, as below (for generic variables $X_1$, $X_2$, and their interaction term $X_{1:2}$):
$$SE(β_{1+2}) = \sqrt{var(β_1+β_2+β_{1:2})} = \sqrt{var(β_1) + var(β_2) + var(β_{1:2}) + 2[cov(β_1,β_2) + cov(β_1,β_{1:2}) + cov(β_2,β_{1:2})]}$$
In R:
beta_se <- sqrt(vcov(m)[1,1] + vcov(m)[2,2] + vcov(m)[3,3] + 2*(vcov(m)[1,2] + vcov(m)[1,3] + vcov(m)[2,3]))

Construction of 95% confidence intervals follows as:
$$\beta_{1+2} \pm 1.96\times SE $$
which can be converted to confidence intervals on the odds ratio by exponentiating them.


convergence of series $\sum_{n=1}^{\infty} a_n$ with $|a_n| \le \frac{n+1}{n^2}$

Question: Let us consider series:
$$S=\sum_{n=1}^{\infty} a_n$$
$$a_n=\frac{n\cos(n)+\sin(n^2+2n-5)}{n^2}$$
The bounds for $a_n$ is shown in this MSE question to be:
$$|a_n| \le \frac{n+1}{n^2}$$
Is it possible to prove that this series is convergent?
Numerical results showed that
$$\sum_{n=1}^{10^5} a_n=-0.931506$$
$$\sum_{n=1}^{10^6} a_n=-0.931501$$
Here is a plot of Christian Blatter's bounds $\pm (n+1)/n^2$ and $a_n$:

From the plot the series, we can see that the terms are oscillating, but not like $(-1)^n |a_n| $. There seems to be some randomness in the number of terms to be positive and number of terms to be negative.
Comment: So it is an alternating series? This should be specified. Otherwise it would diverge
Comment: alternating property is what needs to be proved.
Comment: "rom the plot the series, we can see that the terms are alternating" Actually they are not. Recall that "alternating" means that the signs are $+-+-+\ldots$.
Comment: @Did. Thanks. I changed "alternating" to "oscillating".
Answer:
$$|\frac{\sin(n^2+2n-5)}{n^2}|\leq\frac{1}{n^2}$$
$\sum\frac{\cos n}{n}$ is convergent by using dirichlet test.
Comment: gaoxinge nice answer!
Comment: Maybe I'm thick, but is it easy to see that the sum $\sum^N_{k=0}\cos(k)$ is bounded uniformly in $N$? I assume by 'Dirichlet test' you're reffering to [this](https://en.wikipedia.org/wiki/Dirichlet%27s_test)
Comment: @Teri This definitely can be done but the real question is whether the OP (mike) can do it.
Comment: @Did. You are right.  I can not do it. I took some math classes in physics dept when I was a freshman long time ago and can not remember if we were taught about "Dirichlet test" for convergence. ;)
Comment: Ahha! I was indeed thick; write $\cos (k) =1/2 \ (e^{ik}+e^{-ik})$ and use the formula for the partial sums of the geometric series. If mike is aqcuainted with these, he (at least now) can do it.
Comment: Mathematica showed that $\sum_{n=1}^{\infty} (1/n)\cos n=-(1/2)\log(2(1-\cos 1))$
Comment: "I can not do it." But you *accepted* the answer? I will never get it.


Setting the mime-type for files stored in the private directory

Question: How do I set the mime-type of files being served from the drupal private directory. I have a webm file in the private directory, and the I've added the <video> tag to the page using a tpl file.
I've tried adding
AddType video/webm .webm

to .htaccess. I've also tried installing the module "File Mime" to try to set the right mime type. I've tried adding
function mymodule_file_download($uri){
    return array('Content-type: '.'video/webm');
}

to my .module file. Nothing seems to work. Any help would be appreciated.
Answer: Editing includes/files.mimetypes.inc is a core hack, which you should strive to avoid.
Instead, use hook_file_mimetype_mapping_alter():
function mymodule_file_mimetype_mapping_alter(&$mapping) {
  // Add new MIME type 'video/webm'.
  $mapping['mimetypes']['webm'] = 'video/webm';
  // Add new extension '.webm' and map it to the 'video/webm' MIME type.
  $mapping['extensions']['webm'] = 'webm';
}

Answer: Figured this out myself: I had to add the webm mimetype in the appropriate places in the includes/files.mimetypes.inc file to get drupal to serve private files with the correct mimetype.


Question about Eigenspaces

Question:
The last part is something I don't understand.
The equation should be $x_1-x_3=0$ right? Therefore, $x_1=x_3$
I know how to write the eigenspace and find the basis if there is only one free variable in a 3x3 matrix. I don't understand what happened here though. Some help required :)
Comment: Everything is worked out pretty nicely in the solution you posted.  Do you have a specific question?
Comment: Yea, where do they get the linear combination s and t from?
Answer: Finding a basis for the eigenspace is the same as finding a basis for a null space of
$\left[ \matrix{ -1&0&1\cr 3&0&-3\cr 1&0&1 }\right] $.
So, we find  the solution set of the equation
$$\tag{1}\left[ \matrix{ 1&0&-1\cr 0&0&0\cr 0&0&0 }\right] {\bf x}={\bf 0}  $$
There are two free variables (corresponding to the columns that don't have a "leading row entry"):

$x_3$ is free, so set $x_3$ equal to an arbitrary value, say $s$.
$x_2$ is free, so set $x_2$ equal to an arbitrary value, say $t$.
The first row of the matrix in (1) will tell you $x_1=x_3=s$.
So the general solution to (1) is ${\bf x}=\left[ \matrix {s\cr t\cr s }\right]$, where $s$ and $t$ are arbitrary scalars.
Two free variables tells you the dimension of the eigenspace is 2. To find a basis for the eigenspace, we need to find two independent eigenvectors. We may do so by "spliting up the
vector $\bf x$ into parts" (as the sum of the "$s$ part" and the "t part"):
$$
{\bf x}= \left[ \matrix {s\cr t\cr s }\right]=\left[ \matrix {s\cr 0\cr s }\right] +\left[ \matrix {0\cr t\cr 0}\right]=
s\left[ \matrix {1\cr 0\cr 1 }\right] +t\left[ \matrix {0\cr 1\cr 0}\right].
$$
The vectors $\left[ \matrix {1\cr 0\cr 1 }\right]$ and $\left[ \matrix {0\cr 1\cr 0}\right]$ form a basis for the eigenspace.
Comment: I liked the first "version" better. But I got it :) I didn't understand the part of splitting the vector x into parts. Now I got it.

Question about Eigenspaces


How to convert voice to text?

Question: I am trying to converting my wav file to text file using  sphinx4.  Is it possible to recognize the word which is not include in grammar file?
Comment: no, in the same way that is not possible to find information that is not anywhere to be found :).
Comment: Thank you, is there any other open source tool to recognize our voice and convert into text file.
Comment: The only one I've used is Autonomy Softsound (which is pretty good), but like with all these tools, you need to train the engine with the grammar you want to use. You can use the default grammar, but expect it only to understand correct language only (even on pronunciation). For example, the default en-uk language module (as it's known in Softsound), will produce a substandard result with an USA speaker. Softsound has all kind of clever features to detect languages and localizations, so it can do a better job.
Comment: Thanks again, is softsound open source. Do you have any example which recognize the speech and convert into text file without using any grammar file. I tried many things in sphinx4 but without entering words to the  grammar file it wont accept the vocabulary.
Comment: Softsound it's not open source and it's quite expensive as far as I know :S.
Answer: This is a common misconception that you should exactly work without any grammar to be able to recognize speech efficiently and solve the application task. Instead, it's practical to use some solution to the problem which will work and decode you files.
If you are not sure about the domain or the language you can always use a very generic language model assisted with syllable-based grammar to decode unknown words. It's often common to use web queries after that to turn syllable-based variants into words thus allowing system to aquire vocabulary. That will allow you to get a good result for very generic types of speech.
Sphinx-4 supports that.
Comment: Thank you, i have tried Hello world and HelloNGram examples from demos files. Both the source using grammar file to recognize the speech. If i work without any grammar file what can i do. do you have any example for that. pls guide me. I am new to this concept.
Comment: You can try LatticeDemo for an example of the large vocabulary continuous speech recognition. You can learn more about CMUSphinx and the concepts behind by reading a tutorial http://cmusphinx.sourceforge.net/wiki/tutorial
Comment: Thank you, i tried lattice demo to recognize continues vocabulary, but whenever i tried to upload my voice to that application i can't get single word correctly. In which English slang sphinx recognize the voice.  I used to upload UK slang.
Comment: Most likely the issue is not the slang but mismatch in the acoustic format and the mismatch of the language model. You can get more definite answer if you will post the audio example you are trying to recognize and your expected results. See also the FAQ entry: http://cmusphinx.sourceforge.net/wiki/faq#qwhy_my_accuracy_is_poor

How to convert voice to text?


How to code an If Else Statement that has a condition if the 5th option in flutter is selected?

Question: May I ask on how can I code a statement and condition to let flutter know that the fifth option on list.generate is selected?
List<Answerlist> usingCollection = [
Answerlist('excellent', 'Excellent'),
Answerlist('verygood', 'Very Good'),
Answerlist('good', 'Good'),
Answerlist('fair', 'Fair'),
Answerlist('poor', 'Poor'),
];

list.generate(usingcollection.length,(int index)
{
  return GestureDetector
  (onTapUp: (detail)
  {
     if(Answerlist == //5th index is selected)
     {
       statustype = 'Poor'
     }
  }
}

Then on the Hero Code on flutter which is on the same dart file,, not two dart file
if (widget.statusType == 'Poor') {
      imgPath = 'images/sad.gif';
      msg = 'We are so sorry. What we did wrong?';
      feedbackList = [
        Feedback('Not helpful and not thoughful', false),
        Feedback('Slow and not responsive', false),
        Feedback('Support line is always busy', false),
        Feedback('Confusing', false),
        Feedback('Other', false),
      ];
    }

I was hoping it should show a screen like a hero style on flutter that will show this other option to indicate whats wrong why they choose this option?
Please let me know if i did something wrong thank you for all the help.
Answer: Assumption on Answerlist
class Answerlist {
   final String status;
   final String statusType;
}

You can do
list.generate(usingcollection.length,(int index)
{
  return GestureDetector
  (onTapUp: (detail)
  {
    statustype = usingcollection[index].statusType;
  }
}

Comment: am i not gonna put on assumption something like Answerlist{this. status, this.statustype}
Comment: and how will flutter understood that im looking for the 5th answer on the other code?,, as you may the answers are 1,excellent, 2.good,3 etc and the 5th is poor, and how does flutter will do it on your code if i may? :)
Comment: You want to generate 5 GestureDetectors with static function or what? If that's what you want, you can easily compare objects with the help of https://pub.dev/packages/equatable
Comment: something like this,,i have 5 options, (A,B,C,D,E),,, the first 4 options are okay to proceed, but when they press the E, it should show additional page, like E (F,G,H,D)


Topological proof of infinity of prime numbers and topological properties of the used space

Question: Solving a general topology test from a few semesters ago I found a very interesting exercise, the objective is to conclude that there are infinitely many prime numbers (among other things
). For this we define:
For $a,b\in\mathbb{Z}$ we define the arithmetic sequence
$$
    \text{PA}_{a,b}:=a\mathbb{Z}+b:=\{an+b:n\in\mathbb{Z}\}
$$
and we define
$$
    \text{PA}_{b}:=\bigcup_{a\in\mathbb{Z}} \text{PA}_{a,b}
$$
On the set $\mathbb{Z}$ we place the topology $\mathcal{T}_{\mathbb{Z}}$ given by the vacuum and by the arbitrary union of progressions $\text{PA}_{a,b}$ for $a\neq0$.
Is it possible to see that
$$
    (PA_{a,b})^{c} = \bigcup_{j=1}^{a-1}\text{PA}_{a,b+j}
$$
and with this we can verify that the sets $PA_{a,b}$ are open and closed.
From the above we can prove that
$$
    \mathbb{Z}\setminus\{-1,1\} = \bigcup_{p\text{ prime}}\text{PA}_{p,0}
$$
and conclude that there are infinitely many prime numbers (note that $-1,1$ they are not prime numbers), because the right hand union cannot be closed or $\{-1,1\}$ would be a finite open set, which cannot be.
Some properties that I would be interested in testing in this topological space are:

$(\mathbb{Z},\mathcal{T}_{\mathbb{Z}})$ satisfies the second axiom of countability?
$(\mathbb{Z},\mathcal{T}_{\mathbb{Z}})$ is separable?
$(\mathbb{Z},\mathcal{T}_{\mathbb{Z}})$ is $T_{2}$?
$(\mathbb{Z},\mathcal{T}_{\mathbb{Z}})$ is $T_{4}$?
In the space $(\mathbb{Z},\mathcal{T}_{\mathbb{Z}})$, $x_{n}\to0$ when $n\to\infty$ if and only if for all $m>0$ exist $n_{0}>0$ such that, if $n\geq n_{0}$, $x_{n}$ is divisible by $m$

Also using (5) we can prove that $\displaystyle\lim_{n\to\infty}n!=0$ (I have my doubts about this, but in the test they ask for it).
I have this: To prove (1), considering the countable base
(verifiable):
$$
    \mathcal{B} = \{\text{PA}_{a,b}:a,b\in\mathbb{Z}\}
$$
it is possible to conclude that the topological space satisfies the second axiom of countability, then it is also separable (2).
For (3): Let $x,y\in\mathbb{Z}$. Since the prime numbers are infinite, we know that there exists a prime number such that $\max\{x,y\}<p$. We also have to $x\in\text{PA}_{p,x}$ and $y\in\text{PA}_{p,y}$ for $n=0$. Since $p$ is prime and greater than $x,y$, then
$$
     \text{PA}_{p,x}\cap\text{PA}_{p,y}=\emptyset
$$
With the above we can conclude that the topological space is $T_2$.
I don't see how to approach (4) and (5). An idea I had was to see $T_{4}$ with the urysohn lemma, but this topology seems to me not to be metrizable, since it is different from the usual distance topology in $\mathbb{Z}$
Pd1: Hope to upload detailed solution after solving this.
Pd2: Do you know if it is possible to obtain other results of number theory from the general topology?
Comment: https://topology.jdabbs.com/spaces/S000053
Comment: No, there are no infinite prime numbers.
Comment: @KennyLau no, it's [this one](https://topology.jdabbs.com/spaces/S000050) instead.
Answer: See the Wikipedia page for some of its properties, or its $\pi$-base page for even more of them.
Obviously separable as $\Bbb Z$ is itself countable and dense; the given defiing base is also countable.
All basic sets are clopen as remarked so together with $T_2$ this implies $T_3$ and that with second countable implies $T_4$ and $T_5, T_6$ too here. The space is even metrisable by Urysohn.
The fact about $\lim n!$ is quite easy: a basic neighbourhood of $0$ is of the form $U=\text{PA}_{m,0}$ and so for all $n \ge m$, $n! \in U$ etc.
No point of this space is isolated, so a classical theorem says it's homeomorphic to $\Bbb Q$. It's a "gimmick space" to get a topological proof of the infinitude of primes, and not important beyond that I think.


Upright piano: High pitched chime-like ringing noise

Question: On a new upright piano that is freshly tuned, I hear a high pitched chime-like noise specially when I press and release two mid-range notes simultaneously, like B3-flat and D4. The noise is like someone hitting on crystals inside a cave far away, and it rings for short a moment even after I release the keys.
I can imagine this should have something to do with harmonics of other strings being excited. But is there a way to get rid of it? Or is this something that one has to accept as "quality" and has to live with it?
UPDATE:
I opened the case and tried to check the source, it seems that it is not the small strings that are without dampers. I also tried to damp the struck string by hands, but the chime persist.
I made recording which you can listed to the sound here. (You have to bring the volume up you clearly hear the ringing.) I created a time resolved frequency spectrum which is shown here:

From the spectrum I can see the principal component of the two mentioned notes (D4 = 293.665 Hz and B-falt 3 = 233.082 Hz). But there are the other harmonics that are present all the time, perhaps due to other strings in spite of having dampers.
Can it be that the dampers of other strings are not pressing hard enough?
Or am I just being too sensitive on this?
Comment: The uppermost strings don't (usually) have dampers, so perhaps it's a sympathetic vibration there. But when I've run into this kind of problem, it's usually something else in the room that is vibrating -- it just sounded like it was coming from the piano. The one other scenario I've encountered was on a grand piano where a misaligned string was vibrating against an agraffe.
Comment: Might be the strike-points (could well be called something else) where hammer hits on string is at 7th harmonic i.e. it cancels out the 7th harmonic. The hammer could be slightly misalligned. With hammer misaligned you will quite clearly hear the 7th harmonic.
Comment: Regarding your question about the dampers: yes, it's possible that dampers are misaligned or not shaped correctly.
Comment: Your description of the sound sounds to me like the sound my guitar strings make when the ends are not cutt off. I have littel to no experience with real pianos, but maybe some of the strings are not cut and they catch up the vibrations and start to make that sound? (Just a guess)
Comment: Thanks. I have actually opened the case and looked inside. Everything is really fastened tight in there, no loose parts nowhere. It sounds as if high pitched notes start to vibrate by themselves somehow.
Answer: To find out what's resonating you can try this:

Get one of those small cheap computer microphones on a stick. (Better but not indispensable if it's a directional microphone)

Connect it to a computer or smartphone so you can hear the sound from it

Wear headphones

Play or have someone play those keys while you move the microphone inside the piano, to find out where it's louder, and therefore where it's coming from.

When you get close to it, try damping anything in that area, until you find something that works. Note that you may not be able to see anything moving, but pressing a finger against a particular point might stop or reduce the resonance.

Having said all that, since you mentioned it's a new piano, it's also possible that this resonance will go away by itself after some time, as the whole thing shakes down over time, just by being played.
Update
After hearing the recording, where you play a Bb and the resonance is an F, an octave and a fifth above it, and noticing the resonance has quite a bit of sustain, I'd venture the following guess: the resonance is probably caused by the piano strings themselves, not being completely stopped by the felt dampers.
The resonating string might be the very same string you are playing (Bb), or the Bb above it, or one of the Fs. (Actually, you probably hear a lingering harmonic of the main frequency of the string, rather than the main pitch of the string)
You can easily test it by damping the strings with your finger, or some suitable material. To find out if it's one of the same Bb strings, play that Bb and keep the key down, and stop the sound by manually damping the vibrating string. If the sound stops without resonance, it was that. If the resonance continues, try manually damping other strings, the most likely culprits being octaves or fifths from the main note. But also try octaves and fourths below.
You can also try to gently press down further the dampers against the various strings. If by pressing further down the damper of a string prevents the resonance, that tells you that this particular damper is not stopping that string enough from vibrating sympathetically.
If this is found to be the problem, the solution would be to try and adjust the felt of the damper, so that is presses well enough against the strings, to stop their vibration completely.
Comment: That sounds to me like a harmonic from an insufficiently damped string. See my update above for more details
Comment: In addition, as it's brand new, record the noise as you're searching for it & send it to the manufacturer for comment.
Comment: Thanks. I tried a recording and put the results updates in the main post.
Comment: Yes, that seems to be it. It is now clear to me that it is the dampers that are not pushing hard enough. I recorded a new sound sample, this is G2-G3 octave and can be heard [here](https://raw.githubusercontent.com/xaratustrah/teaching/master/pkey/pkey2.mp3) generally it seems to happen in the range G2-C5. I contacted the dealer they told me it is nothing serious and can happen due to transport that the mechanics might move. They suggested to leave it for a couple of weeks until they come for the service tuning, which is already paid in advance with the brand new instrument.


Find matching equal orders using arrays and PHP

Question: I am cracking my brain and can't find a good solution for my problem. I am trying to design a system that I can use for batch picking in our order system.
The point is that from a set of orders I want to pick 6 orders that are most equal to each other. In our warehouse most orders are them so we can safe a lot of time by picking some orders at the same time.
Assume I have the following array:
<?php

$data = [
    156 => [
        1,
        2,
        7,
        9,
    ],
    332 => [
        3,
        10,
        6
    ],
    456 => [
        1,
    ],
    765 => [
        7,
        2,
        10,
    ],
    234 => [
        1,
        9,
        3,
        6,
    ],
    191 => [
        7,
    ],
    189 => [
        7,
        6,
        3,
    ],
    430 => [
        10,
        9,
        1,
    ],
    482 => [
        1,
        2,
        7,
    ],
    765 => [
        1,
        5,
        9,
    ]
];
?>

The array key is the order id, and the values are the product ID's it contains. If I want to pick the top 3 orders which look at much like each other, where do I start?
Any help would be much appreciated!
Comment: Define method to measure equallness.
Answer: Here is what i would do if I had the problem :
$topOrders = [];
foreach($data as $value):
    foreach($value as $order):
        if(isset($$order)):
            $$order++;
        else:
            $$order = 1;
        endif;
        $topOrders[$order] = $$order;
    endforeach;
endforeach;

print_r($topOrders);

In $topOrders, you have an array that contains as key the ID, and as value you got the number of orders. All you have to do is to sort your array to get your top 3.
Answer: 1. Step
Sort productId inside order (ASC)
2. Step
In loop check difference (array_diff) in each order to  each other.
Create array with defference. For example:
$diff = [
    '156' => [ //order id
        '234' => 4, // with order 234 has 4 differences
        '332' => 7, // with order 332 has 7 differences
        // and so on...
    ],
]

3. Step
Order $diff by ASC and receive order with less differences.
Improvement
Also you could add total size of products in order for compare with difference. For example, If you have an order with 100 products and 10 diffs - it's better than order with 10 products and 9 diffs.


when to prefer pessimistic model of transaction isolation over optimistic one?

Question: Do I understand correctly that table/row lock hints are being used for pessimistic transaction (TX) isolation models of concurrency ONLY?
In other words, when can table/row lock hints be used during engagement of optimistic TX isolation provided by SQL Server (2005 and higher)?
When one would need pessimistic TX isolation levels/hints in SQL Server2005+ if the later provides built-in optimistic (aka snapshot aka versioning) concurrency isolation?
I did read that pessimistic options are legacy and are not needed anymore, though I am in doubt.
Also, having optimistic (aka snapshot aka versioning) TX isolation levels  built-in SQL Server2005+,
when one would need to manually code for optimistic concurrency  features?
The last question is inspired by having read:

"Optimistic Concurrency in SQL Server" (September 28, 2007)

describing custom coding to provide versioning in SQL Server.
Answer: Optimistic concurrency requires more resources and is more expensive when the conflict occurs.
Two sessions can read and modify the values and the conflict only occurs when they try to apply their changes simultaneously. This means that in case of the concurrent update both values should be stored somewhere (which of course requires resources).
Also, when a conflict occurs, usually the whole transaction should be rolled back or the cursor refetched, which is expensive too.
Pessimistic concurrency model uses locking, thus downgrading concurrency but improving performance.
In case of two concurrent tasks, it may be cheaper for the second task to wait for a lock to release than spending CPU time and disk I/O on two simultaneous works and then yet more on rolling back the less fortunate work and redoing it.
Say, you have a query like this:
UPDATE  mytable
SET     myvalue = very_complex_function(@range)
WHERE   rangeid = @range

, with very_complex_function reading some data from mytable itself. In other words, this query transforms a subset of mytable sharing the value of range.
Now, when two functions work on the same range, there may be two scenarios:

Pessimistic: the first query locks, the second query waits for it. The first query completes in 10 seconds, the second one does too. Total: 20 seconds.
Optimistic: both queries work independently (on the same input). This shares CPU time between them plus some overhead on switching. They should keep their intermediate data somewhere, so the data is stored twice (which implies twice I/O or memory). Let's say both complete almost at the same time, in 15seconds.
But when it's time to commit the work, the second query will conflict and will have to rollback its changes (say, it takes the same 15 seconds). Then it needs to reread the data again and do the work again, with the new set of data (10 seconds).
As a result, both queries complete later than with a pessimistic locking: 15 and 40 seconds vs. 10 and 20.

When one would need pessimistic TX isolation levels/hints in SQL Server2005+ if the later provides built-in optimistic (aka snapshot aka versioning) concurrency isolation?

Optimistic isolation levels are, well, optimistic. You should not use them when you expect high contention on your data.
BTW, optimistic isolation (for the read queries) was available in SQL Server 2000 too.
Comment: @Quassnoi: I like your answer, but in the very common case where you *don't* have high write contention, optimistic locking typically performs better than pessimistic locking. Any chance you could mention that?
Comment: Great answer.  Just to put times on (2) for comparison, using your figures for (1) as indicative: both xacts take 20 secs; the first completes; the second is rolled back, another 20 secs; it retries and completes, another 20 secs. Total for second xact 60 secs.
Comment: can you further explain what did you mean under "optimistic isolation (for the read queries) was available in SQL Server 2000 too"?
Comment: @antinome: what exactly do you mean by "performs better"?
Comment: @Quassnoi: I mean, with optimistic locking, no actual lock is ever acquired. While no conflicts are occurring (hopefully most of the time), there is essentially no performance penalty compared to not locking at all. Whereas, doesn't pessimistic locking incur some penalty for acquiring locks even when there is no conflict?
Comment: @antinome: something which tells "this record is affected" is acquired. If it's a lock, it prevents another transaction from reading (or making changes). If it's just a marker, it makes another transaction roll back on conflict. But the thing itself is still there.
Answer: I have a detailed answer here: Developing Modifications that Survive Concurrency
Answer: I think there's a bit confusion over terminology here.
The technique of optimistic locking/optimistic concurrency/... is a programming technique used to avoid the following scenario :

start transaction
read data, setting a "read" lock on it to prevent any deletes/modifications to our data
display data on user's screen
await user input, lock remains active
keep awaiting user input, lock still preventing any writes/modifications
user input never comes (for whatever reason)
transaction times out (and this is usually not very rapidly, as the user must be given reasonable time to enter his input).

Optimistic locking replaces this with the following:

start transaction READ
read data, setting a "read" lock on it to prevent any deletes/modifications to our data
end transaction READ, releasing the read lock just set
display data on user's screen
await user input, but data can be modified/deleted meanwhile by other transactions
user input arrives
start transaction WRITE
verify that the data has remained unaltered, raising an exception if it has
apply user updates
end transaction WRITE

So the single "user transaction" to go fetch some data, and change and update them, consists of two distinct "database transactions".  What is usually called "isolation levels" applies to those database transactions.  The "optimistic locking" that you refer to applies to the "user transaction".
The matter is further complicated in that, broadly speaking, two completely distinct strategies are possible for the "isolating the database transactions part" :

MVCC
2-phase locking

I think the "snapshot versioning isolation level" means that the MVCC technique (well, one of its various possible variations) is being used for the database transaction.  The other commonly known isolation levels apply more to transaction isolation using 2PL as the serialization(/isolation) technique.  (And mixing them up can get messy ...)
Comment: -1 Commendable attempt at clearing up a confused subject, but I disagree with the terms you are using (they are standard terms which have standard meaning, not what you have described).  The result is, while there might be less confusion re MS usage of terms, there is more confusion about the generic issues and standard terms.
Comment: @PerformanceDBA, thanks. I wanted to save a few minutes on concrete specific doubts and ended up spending hours to revise my knowledge of  basic terms definitions I knew for years. Unfortunately, your comments did not appear in my notifications (upon pressing envelope button). Now, I am in doubt should I add my own questions (and answers) to favorites in order to be notified on comments to replies to my own questions?
Comment: @vgv8: 1-The standards terms existed before MS bubbled up from Hell.  2-MS misused and abused to terms, & created their own terms that referenced std terms, fraudulently implying that they provided the function of the std terms; the usual dishonest marketing spin. 3-Now MS have (a) done it again with MVCC/2Pl and (b) invented terms based on their previous misleading terms, which are now *two* levels removed from the std terms, with no relation to the std function.  Eg. there is nothing whatsoever "optimistic" about MVCC, or "pessimistic" about 2PL; in fact, it is the opposite.  Will post.