Nonparametric tolerance intervals for discrete variables

Question: Suppose you have a bunch of people rate how much they liked a movie on a discrete scale from 1 to 10, and you would like an interval [l, u] such that with (at least) 95% confidence, (at least) 90% of all people who see the movie will rate it no lower than l and no higher than u. [l, u] is then a (two-sided) tolerance interval with 95% confidence and 90% coverage. (To be clear, 95% confidence implies that if you repeated this procedure many times, 95% of the produced intervals would get at least 90% population coverage.) Of course, we generally want [l, u] to be as narrow as possible while still meeting our requirements.
I've seen various nonparametric methods for constructing tolerance intervals for continuous random variables. I've also seen methods for constructing tolerance intervals for binomial and Poisson variables. (The R package tolerance implements several of these methods; Young, 2010.) But what about discrete variables when the distribution is unknown? This is generally the case for rating scales like the one in my example, and assuming a binomial distribution doesn't seem safe because real rating-scale data often exhibits weirdness such as multimodality.
Would it make sense to fall back on the nonparametric methods for continuous variables? Alternatively, what about a Monte Carlo method such as generating 1,000 bootstrap replicates of the sample and finding an interval that captures at least 90% of the sample in at least 950 of the replicates?
Young, D. S. (2010). tolerance: An R package for estimating tolerance intervals. Journal of Statistical Software, 36(5), 1–39. Retrieved from http://www.jstatsoft.org/v36/i05
Comment: do you mean binomial or multinomial? multinomial would allow multimodal behaviour?
Comment: I do mean binomial. In the case of a rating scale, for example, you would set the number of Bernoulli trials to the number of scale points. Intervals across the categories of a multinomial distribution wouldn't make much sense, I think, since the categories are unordered.
Comment: @Kodiologist your outcome variable is a "discrete scale from 1 to 10" but that means it *is* an ordered multinomial response. (Or am I not getting something?)
Comment: @Jim "Ordered multinomial" is a bit of an oxymoron. In a multinomial distribution, the order of the categories is arbitrary.
Answer: The variable of interest is multinomially distributed with class (cell) probabilities: $p_1, p_2, ..., p_{10}$. Further, the classes are endowed with a natural order.
First attempt: smallest "predictive interval" containing $90\%$
p     = [p1, ..., p10] # empirical proportions summing to 1
l     = 1
u     = length(p)
cover = 0.9

pmass = sum(p)

while (pmass - p[l] >= cover) OR (pmass - p[u] >= cover)
    if p[l] <= p[u]
       pmass = pmass - p[l]
       l     = l + 1
    else # p[l] > p[u]
       pmass = pmass - p[u]
       u     = u - 1
    end
end

A non-parametric measure of the uncertainty (e.g., variance, confidence) in the $l,u$-quantile estimates could indeed be obtained by standard bootstrap methods.
Second approach: direct "bootstrap search"
Below I provide runable Matlab code that approaches the question directly from a bootstrap perspective (the code is not optimally vectorized).
%% set DGP parameters:
p = [0.35, 0.8, 3.5, 2.2, 0.3, 2.9, 4.3, 2.1, 0.4, 0.2];
p = p./sum(p); % true probabilities

ncat = numel(p);
cats = 1:ncat;

% draw a sample:
rng(1703) % set seed
nsamp = 10^3;
samp  = datasample(1:10, nsamp, 'Weights', p, 'Replace', true);

Check that this makes sense.
psamp = mean(bsxfun(@eq, samp', cats)); % sample probabilities
bar([p(:), psamp(:)])

Run the bootstrap simulation.
%% bootstrap simulation:
rng(240947)

nboots = 2*10^3;
cover  = 0.9;
conf   = 0.95;

tic
Pmat = nan(nboots, ncat, ncat);
for b = 1:nboots

    boot  = datasample(samp, nsamp, 'Replace', true); % draw bootstrap sample
    pboot = mean(bsxfun(@eq, boot', cats));

    for l = 1:ncat
        for u = l:ncat
            Pmat(b, l, u) = sum(pboot(l:u));
        end
    end

end
toc % Elapsed time is 0.442703 seconds.

Filter from each bootstrap replicate the intervals, $[l,u]$, that contain at least $90\%$ probability mass and calculate a (frequentist) confidence estimate of those intervals.
conf_mat = squeeze(mean(Pmat >= cover, 1))

     0         0         0         0         0         0         0    1.0000    1.0000    1.0000
     0         0         0         0         0         0         0    1.0000    1.0000    1.0000
     0         0         0         0         0         0         0    0.3360    0.9770    1.0000
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0

Select those that satisfy the confidence desideratum.
[L, U] = find(conf_mat >= conf);
[L, U]

 1     8
 2     8
 1     9
 2     9
 3     9
 1    10
 2    10
 3    10

Convincing yourself that the above bootstrap method is valid
Bootstrap samples are intended to be stand-ins for something we would like to have, but do not, i.e.: new, independent draws from the true underlying population (short: new data).
In the example that I gave, we know the data generating process (DGP), therefore we could "cheat" and replace the code lines pertaining to bootstrap re-samples by new, independent draws from the actual DGP.
newsamp = datasample(cats, nsamp, 'Weights', p, 'Replace', true);
pnew    = mean(bsxfun(@eq, newsamp', cats));

Then we can validate the bootstrap approach by comparing it to the ideal. Below are the results.
The confidence matrix from new, independent data draws:
     0         0         0         0         0         0         0    1.0000    1.0000    1.0000
     0         0         0         0         0         0         0    1.0000    1.0000    1.0000
     0         0         0         0         0         0         0    0.4075    0.9925    1.0000
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0
     0         0         0         0         0         0         0         0         0         0

The corresponding $95\%$-confidence lower and upper bounds:
 1     8
 2     8
 1     9
 2     9
 3     9
 1    10
 2    10
 3    10

We find that the confidence matrices closely agree and that the bounds are identical... Thus validating the bootstrap approach.
Comment: Tolerance intervals and confidence intervals are different things. Actually, what you've described is not a confidence interval but a prediction interval, which is yet another distinct kind of interval.
Comment: Your edit seems to be an implementation of what I meant when I wrote "a Monte Carlo method such as generating 1,000 bootstrap replicates of the sample and finding an interval that captures at least 90% of the sample in at least 950 of the replicates". While intuitive, I'm not sure this actually works or makes sense, which is why I created this question.
Comment: Showing that the bootstrap method is correct for this problem in its full generality means showing that it has the right confidence and coverage regardless of the prior distribution of parameters (it is, after all, a frequentist method). For that, I think, simulation wouldn't suffice; you would need mathematical proof. But you've been remarkably persistent and shown that the bootstrap works at least sometimes, so you deserve an A for effort.
Comment: @Kodiologist The answer now contains a section validating the bootstrap approach. Of course, this could be taken further, e.g. nested in loops over sample size and class-probabilities.


iptables DROP policies not working with allowed incoming http connection

Question: I have a task to secure the nginx server, which is working with docker-container as a reverse proxy on port 8732. I have to disable all incoming connections except port 80 for issuing SSL certificates and, of course, SSH connections. I've set some rules, but they aren't working with INPUT and OUTPUT default policies set to DROP, which is required in the task. The point is, all rules working like a charm with policies set to ACCEPT.
Here's my iptables rules:
root@bw:/home/user# iptables -S
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-A INPUT -s <**external address**/24> -p tcp -m tcp --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -p tcp -m tcp --sport 22 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -d <mail server/32> -p tcp -m tcp --dport 587 -m state --state ESTABLISHED -j ACCEPT
-A DOCKER -d 172.19.0.2/32 ! -i br-9c6ddba356b8 -o br-9c6ddba356b8 -p tcp -m tcp --dport 80 -j ACCEPT

here's an output from nmap:
    root@bw:/home/user# netstat -tlpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      886/nginx: master p
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN      766/systemd-resolve
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      842/sshd: /usr/sbin
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      886/nginx: master p
tcp        0      0 0.0.0.0:8732            0.0.0.0:*               LISTEN      49697/docker-proxy
tcp6       0      0 :::443                  :::*                    LISTEN      886/nginx: master p
tcp6       0      0 :::22                   :::*                    LISTEN      842/sshd: /usr/sbin
tcp6       0      0 :::80                   :::*                    LISTEN      886/nginx: master p
tcp6       0      0 :::8732                 :::*                    LISTEN      49702/docker-proxy

and output from iptables:
user@bw:~$ sudo iptables -L -v -n
Chain INPUT (policy DROP 110 packets, 30439 bytes)
 pkts bytes target     prot opt in     out     source               destination
11398  751K ACCEPT     tcp  --  *      *       <my subnet address.0/24>       0.0.0.0/0            tcp dpt:22 state NEW,ESTABLISHED
33540   15M ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
 2129  230K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 state NEW,ESTABLISHED

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy DROP 838 packets, 54784 bytes)
 pkts bytes target     prot opt in     out     source               destination
 8303 1951K ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp spt:22 state ESTABLISHED
33551   15M ACCEPT     all  --  *      lo      0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            <mail server ip>         tcp dpt:587 state ESTABLISHED
  176 19390 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp spt:80 state ESTABLISHED

Chain DOCKER (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     tcp  --  !br-9c6ddba356b8 br-9c6ddba356b8  0.0.0.0/0            172.19.0.2           tcp dpt:80

Comment: I don't grok what your problem is?
Comment: @vidarlo the point is i have to restrict an internet access to the virtual machine to fully secure it and only allow HTTP. This iptables configuration is working, but only when default policies INPUT and OUTPUT are set to ACCEPT, but I have a task to make it DROP by default, and then add a rule to allow SSH from subnet, and HTTP from the world.
Comment: How do you verify if they work or not?
Comment: @vidarlo, by checking in a web browser the address of the resource. also by CURLing the address, I can't get a 200 response. Probably I mixed up with open ports on network interfaces.
Answer: Your OUTPUT chain is missing the rule to allow established connections from port 80.
Use
iptables -A OUTPUT -p tcp --sport 80 -m state --state ESTABLISHED -j ACCEPT

Your nmap output is from netstat, which is a completely different tool.
The netstat output shows which processes have active listening TCP sockets. Whenever you run a daemon that opens a listening socket, it always appears in the list.
Firewall rules operate inside the network stack, and act as a separate isolation layer to access services. When you configure firewall entries, they will restrict traffic to the server. Those rules will have no effect on what netstat shows.
You need to check if the rules work by running iptables -nvL, which will show counters on how many packets each rule provided. Another check is to make requests to the server from outside and see if the rules work.
Comment: sorry for confusing you, i meant netstat
Comment: i've updated the question with attached output from iptavles
Comment: I updated the answer with a missing rule.
Comment: thanks for your answer, unfortunately, it didn't fixed the problem
Comment: @Taras Paslavskyi post iptables -nvL output again so we can see what you actually done
Comment: @gapsf sure thing, buddy, I have updated my post now
Comment: @Taras Paslavskyi port 80 is now ok it should work. How exectly you check it? Commandline or url you try in browser. Also why you dont check it without any firewall rules just to check it works at all as you expected? I guess firewall is not source of problem
Comment: I'm trying in my browser, which is giving me 504 connection timed out, and in command line with curl, wget. My UFW firewall is disabled, I only using iptables in this case
Comment: @gapsf, it works with iptables input and output policies set to accept. my task is to secure the virtual machine and restrict the internet connection by only allowing HTTP and SSH.
It could be a problem with my reverse-proxy to docker container, which is running on localhost:8732
Comment: To what port you trying to connect from a browser: 80 or 443 or 8732 ? Is it http or https? I need to know exact connection url to figure out this.
Comment: @gapsf i'm trying to connect to port 80, which is giving me 301 and redirects me to https version and after that I'm receiving 504. If I going straight to 443, I'm getting 504 instantly
Comment: If you receive any response, it means that the iptables rules allow the traffic. You need to look at the destination server why it returns HTTP status code 504.
Comment: @TeroKilkanen my nginx reverse proxy looks like this
location / {
                proxy_pass http://localhost:8732/;
        }
all the matter, with iptables default policies INPUT, OUTPUT to ACCEPT it is working with the rules, and with policies to DROP it is not :)
Comment: @Taras Paslavskyi
Obviously you should open 443/tcp port as well
Comment: @gapsf, thanks for getting back to me. this port is opened https://imgur.com/OQ6dYiT
Comment: @Taras Paslavskyi No, 443 port is closed by firewall - it's not allowed in INPUT and dropped according the DROP policy. Your screenshot is about listening sockets, not about firewalling. I'v post the answer for current situation
Answer:
I have to disable all incoming connections except port 80 for issuing
SSL certificates and, of course, SSH connections

i'm trying to connect to port 80, which is giving me 301 and
redirects me to https version and after that I'm receiving 504. If I
going straight to 443, I'm getting 504 instantly

By default - HTTP is 80/tcp, HTTPS is 443/tcp. You have 80/tcp opened only.
-A INPUT -p tcp -m tcp -m multiport --dports 80,443 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
-A OUTPUT -p tcp -m tcp -m multiport --sports 80,443 -m state --state ESTABLISHED,RELATED -j ACCEPT

Also notice, in OUTPUT you do not allow any NEW connection (so effectively block all NEW)   from the host. So for example DNS queries blocked or maybe something else impotent for your host functioning.
Also you don't need repeat rules in OUTPUT for each corresponding INPUT rule.
Better organize rules in INPUT and OUTPUT in cleaner, predictable and secure way like this pattern
-P MAIN_CHAIN DROP
-A MAIN_CHAIN -o lo -j ACCEPT
-A MAIN_CHAIN -m state --state ESTABLISHED,RELATED -j ACCEPT
# next rules ACCEPT only NEW for all your needs
-A MAIN_CHAIN -m state --state NEW match-something1 -j ACCEPT
-A MAIN_CHAIN -m state --state NEW match-something2 -j ACCEPT
#... and so on

So your rules would be:
-P INPUT DROP
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -m state --state NEW -j ACCEPT
-A INPUT -p tcp -s <**external address**/24> --dport 22 -m state --state NEW -j ACCEPT
-A INPUT -p tcp -m multiport --dports 80,443 -m state --state NEW -j ACCEPT

-P OUTPUT DROP
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

Comment: hello there, i've provided more info In chat as you've requested, cheers
Comment: thanks for your answer, i've noticed there should be A INPUT -i lo -j ACCEPT instead of -o in that line and dash before the "m" here -A INPUT m state --state ESTABLISHED,RELATED -j ACCEPT and here -A OUTPUT m state --state ESTABLISHED,RELATED -j and in the end should be added ACCEPT after -j
Comment: besides that, unfortunately it does not help me, with allowed 80 and 443 and INPUT DROP it still not opening and I can't ping on that port
root@bw:/home/user# ping localhost:8732
 Name or service not known
Comment: Update answer. You can't ping tcp/udp port only IP, ping use icmp protocol and should be accepted
Comment: thanks for pointing this, i'm absolute noob in networking. Is there any ways to troubleshoot why nginx cant serve docker-container ? it's working as reverse proxy
Comment: What commandline you use tu run your docker container? Read "Public Networking" here https://earthly.dev/blog/docker-networking/ Until that i guess from your `netstat -tlpn` output you map docker's 80 port to 8732 so you should open tcp 8732 in INPUT and connect from browser on 8732 or if you want standard 80 you should map port 80 of container to port 80 of host in `docker run` command. Also check what iptables rules looks like after docker run - docker may change it for you
Comment: im running it with "docker-compose up -d", the docker-compose file has the following port mapping:
ports:
      - "127.0.0.1:8732:80"
and In nginx:
location / {
                proxy_pass http://127.0.0.1:8732/;
Comment: root@bw:/opt/vault# docker ps
PORTS 3012/tcp, 127.0.0.1:8732->80/tcp
Comment: Ok, nginx listen on host's 80 and proxy it to container on 127.0.0.1:8732. Should work. What nginx logs says about connections? Can you connect to 127.0.0.1:8732 from a browser from you host?
Comment: Also you use SSL when try to connecting from outside? What is you full configuration of nginx server{} block? Are you opened 443 in INPUT? nginx listen on 443? Provide actual information and configuration for iptables and nginx and full urls you connect in a browser
Comment: Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/138803/discussion-between-taras-paslavskyi-and-gapsf).


Comparar registro atual com registro anterior em laço/loop

Question: Cenário
Vamos supor que tenho 100 registros em um BD qualquer.
ID | PESSOA | CARRETO | ENDERECO | DISTANCIA
1  |   A    |    C10  |    XX    |    20
2  |   B    |    C20  |    XY    |    25
3  |   D    |    C50  |    XZ    |    19
4  |   D    |    C50  |    XZ    |    19
5  |   F    |    C10  |    XW    |    27
...
...

A partir de uma query, vou gerar um Array com todos estes registros, e enquanto faço o fetch, eu preciso verificar algumas condições, comparando o registro anterior do registro corrente:

ID do reg. anterior = ID do reg. corrente -1
PESSOA do reg. anterior = PESSOA do reg. corrente
CARRETO do reg. anterior = CARRETO do reg. corrente
ENDERECO do reg. anterior = ENDERECO do reg. corrente
DISTANCIA do reg. anterior = DISTANCIA do reg. corrente

Exemplo:
ID | PESSOA | CARRETO | ENDERECO | DISTANCIA
3  |   D    |    C50  |    XZ    |    19
4  |   D    |    C50  |    XZ    |    19

(Se caso todas as condições forem verdadeiras, eu gravo o registro corrente com a DISTANCIA = 0.)

Como fiz
No laço que faço o fetch, eu usei variáveis auxiliares para trazer e comparar o registro anterior verificar as condições, e gravar o atual no array:
// variáveis auxiliares:
$id_ant = null; // seto como null para a 1a comparação
$pes_ant; $car_ant; $end_ant; $dis_ant;

while ($row = $result->fetch_assoc()) {

   // verifico as condições
   if ($id_ant == $row['ID']-1 && $pes_ant == $row['PESSOA'] && $car_ant == $row['CARRETO'] && $end_ant == $row['ENDERECO'] && $dis_ant == $row['DISTANCIA'])
       $row['DISTANCIA'] == 0;

   // gravo o registro no array
   $array[] = $row;

   // seto cada variável para depois utilizar na comparação posterior
   $id_ant = $row['ID'];
   $pes_ant = $row['PESSOA'];
   $car_ant = $row['CARRETO'];
   $end_ant = $row['ENDERECO'];
   $dis_ant = $row['DISTANCIA'];
}

Exemplo como ficaria o registro 4 no array:
(
    [ID] => 4
    [PESSOA] => D
    [CARRETO] => C50
    [ENDERECO] => XZ
    [DISTANCIA] => 0
)

Dúvidas

E se eu tivesse uma tabela com 100 colunas a serem comparadas? Teria que fazer tudo isso uma a uma ou tem uma forma "automática" de fazer isso?
É vantagem eu fazer isso já no fetch ou seria melhor jogar tudo para um Array  e depois trabalhar nele? (pensando em desempenho)
Comment: Isso não seria coisa de um `select distinct` ? O que está a tentar fazer é ficar apenas com os que não são duplicados em todos os campos ?
Comment: @Isac, não, preciso de todos. A única diferença é tratar se a condição for verdadeira, a distância fica 0, mas o registro tem que ir também!
Comment: No `if` você está usando `=` no lugar de `==`.  E você precisa comparar todos os campos? Só o `id` e o nome da `pessoa` não seria suficiente?
Comment: @ThiagoMagalhães opa cara, essa do `=` foi falha de digitação! Preciso comparar vários, por isso já estou pensando lá na frente, se caso tiver que comparar 100 colunas, entende!? rs
Comment: Você poderia salvar tudo em uma array e depois percorrer usando foreach.. dessa forma você poderia usar uma lógica para percorrer os atributos e fazer a comparação.
Comment: @ThiagoMagalhães então cara, acho que você está dizendo da forma que o gabriel.pelegrina postou como resposta!?
Comment: É uma forma. E sobre passar tudo para `array`, pensando em desempenho, o ganhou ou perda nesse caso seria minimo.
Comment: @ThiagoMagalhães Então. Mas eu fazer comparação direto no `fetch` é 1 passo. Gerar o Array e depois fazer `foreach`, seriam 2 passos. Se pegar um volume grande, acredito que seja considerável sim.
Comment: O passo de gerar uma `array` computacionalmente terá um tempo de `linear = f(n)`. Então irá consumir o tempo do seu `foreach` + `n`, onde `n` é a quantidade de resultados. Aumentar `1 n` considero pouco, se comparar com funções de `(n^2)` e `(2^n)`, por exemplo.
Comment: Não entendi. Se eu fizer o `fetch`para gerar o array, eu percorro por exemplo 10.000 registros. Depois meu `foreach` terá que percorrer novamente os 10.000 registros. Não é assim!?
Answer: Tente o exemplo abaixo:
//$array são todos os resultados da query ($result->fetch_all() )

if(count($array) > 1) { //verifica se existem mais de 1 item no array

  foreach ($array as $key => $atual) { //percorre o resultado da query

    if ($key > 0) { //compara se não é o primeiro item, pois este não terá item anterior

        $keyAnterior = $key - 1; //armazena o indice do registro anterior

        $anterior = $array[$keyAnterior]; //pega o item anterior do array

        if ($anterior['id'] == ($atual['id'] - 1)) { //comparar os ids (esta comparação ocorre devido a regra solicitada)

            unset($atual['id'], $anterior['id']); //remove a "coluna" id dos dois valores pois este ja foi comparado acima

            $saoIguais = $anterior === $atual; //compara se as colunas restantes do item anterior e do item atual são identicas

            if($saoIguais){
                //as demais colunas são iguais entre os itens
            }else{
                //as demais colunas são diferentes
            }

        } else {
            //a validação do id anterior == (id atual -1) não é verdadeira
        }
    }
  }
}

EDIT
Utilizando o método fetch_assoc você pode tentar assim:
$anterior = null;
while ($row < $result->fetch_assoc()) { //percorre pelo resultado da query
    $atual = $row; //utilizada variavel auxiliar pois o indice id será excluido do array, e precisaremos dele na proxima iteração

    if(!is_null($anterior)) { //verifica se o anterior não é null | primeira iteração
        if ($anterior['id'] == ($atual['id']) - 1) { //verificação do id

            unset($anterior['id'], $atual['id']); //remove o indice do item anterior e do atual

            if ($atual == $anterior) { //compara os indices restantes (podem ter N indices)
                echo "São iguais";
            } else {
                echo "São diferentes";
            }
        }
    }

    $anterior = $row; //atribui a variavel auxiliar ao item anterior
}

**Resposta Dúvida 1: ** Sim, é possível comparar N indíces com o operador de igualdade ( == ou ===);
**Resposda Dúvida 2: ** De acordo com php.net utilizar o método fetch_all() irá consumir mais memória pois retorna todas as linhas de uma só vez. Neste caso em específico você ainda precisa iterar cada linha com um loop, o que pode sim impactar na performance. No entanto, para resultados de poucas linhas é imperceptível.
Espero ter ajudado.
Comment: Seria bom explicar o que você fez...
Comment: Opa, vou editar
Comment: Olá Gabriel! Primeiramente agradeço sua resposta, entendi ela perfeitamente, gostei. Como você está usando o `array`, não teria como então já fazer isso direto no `fetch`? E gostaria de saber sobre a 2a questão, se é melhor fazer no `fetch` ou tratar no array como fez. ✌
Comment: @rbz dá uma olhada no edit. Espero que ajude.
Answer: Faça assim
<?PHP
while ($row = $result->fetch_assoc()) {

   // verifico as condições
   if ($id_ant == $row && $row['ID']==$id_ant['ID']-1){
    $array[] = $row;
   }
   $id_ant = $row;
}
?>


SQL PDO Wrong syntax near "@P1"

Question: I try to use LIMIT with PDO SQL.
My code is this:
$sql = "SELECT * FROM TblWerte LIMIT :limit1";
$result = $db->prepare($sql);
$result->bindParam(':limit1', $limit1);
$limit1 = 15;
$result->execute();

But when I write $result->execute(); I keep getting this error:

PHP Fatal error:  Uncaught PDOException: SQLSTATE[42000]:
  [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Falsche Syntax
  in der Nähe von "@P1". in C:...\werte.php:96
Stack trace:
0 C:\...\werte.php(96): PDOStatement->execute()
1 {main}
thrown in C:\...\werte.php on line 96

The line 96 is "$result->execute();"
I already tried different this like using bindValue() instead of bindParam(), I tried to directly parse the value "limit1" into an integer in bindParam() ...
The solution
I had to use TOP instead of LIMIT. Also I had to remove the bindParams() part.
$sql = "SELECT TOP 15 * FROM TblWerte ORDER BY ID DESC";
$result = $db->prepare($sql);
$result->execute();

Comment: the error message doesn't seem to be related the the code posted. Let alone SQL server doesn't support LIMIT. Something is very wrong here
Comment: Actually a part of the code was wrong (see the solution in my edited post). I also hat to change LIMIT (see the first answer of Gabriel Durac)
Answer: The LIMIT clause is not part of standard SQL. It's supported as a vendor extension to SQL by MySQL, PostgreSQL, and SQLite.
SQL Server has something similar: TOP
$sql = "SELECT top :limit1 * FROM TblWerte";
$result = $db->prepare($sql);
$result->bindParam(':limit1 ', $limit1);
$limit1 = 15;
$result->execute();

Comment: Thanks for your answer. I did it like you wrote it, but same error :/
Comment: Hello, I finally figured out the problem. Your answer is working well. The problem was result->bindParam. I posted the solution in the original post.

SQL PDO Wrong syntax near "@P1"


T-SQL select rows between duplicated rows

Question: I have table Customer with these columns:
sales_ID varchar(50)
product_ID int
ProcessNumber int
CreateDate datetime

and this sample data:
sales_Id       product_id    ProcessNumber
---------------------------------------------
00000020041       1733           15
00000020041       1733           15
00000020041       1733           15
1522              1111           11
1522              1111           11
76002             1333           45
76002             1333           45

Question:
If I use this query
SELECT  *
FROM
   (SELECT
       *,
       CountOfRowsInGroup = COUNT(*) OVER (PARTITION BY sales_Id, product_ID, ProcessNumber)
    FROM Customer) AS tbl
WHERE
    CountOfRowsInGroup > 1
ORDER BY
    Sales_ID;

Query result:
    sales_Id       product_id    ProcessNumber
    ---------------------------------------------
    00000020041       1733           15
    00000020041       1733           15
    00000020041       1733           15
    1522              1111           11
    1522              1111           11
    76002             1333           45
    76002             1333           45

However I want result display this result with duplicated rows, as below
sales_Id       product_id    ProcessNumber
---------------------------------------------
DUPLICATED ROW
00000020041       1733           15
00000020041       1733           15
00000020041       1733           15
DUPLICATED ROW
1522              1111           11
1522              1111           11
DUPLICATED ROW
76002             1333           45
76002             1333           45

Thanks
Answer: From the query result you got,you can append the below query to get your desired output. Here I have taken the ranking as DENSE_RANK to get unique id's for each sales_id. For DUPLICATE ROW values, the ranking will be 1,2,3 etc and for the existing values, the ranking will be in the order 1.1,2.1,3.1 etc.
Then I use UNION ALL to combine the result of existing values and DUPLICATE ROW and the ORDER BY will be in the format 1,1.2,2,2.1 .... etc.
;WITH CTE AS
(
    SELECT DENSE_RANK() OVER(ORDER BY sales_Id)+0.1 RNO,
    DENSE_RANK() OVER(ORDER BY sales_Id) RNO2,'DUPLICATE ROW' DUP,*
    FROM #TEMP
)
,CTE2 AS
(
    SELECT DISTINCT RNO2 AS ROWNUM,CAST(DUP AS VARCHAR(30)) sales_Id ,NULL product_id,NULL ProcessNumber
    FROM CTE
    UNION ALL
    SELECT RNO,CAST(sales_Id AS VARCHAR(30)),product_id,ProcessNumber
    FROM CTE
)
SELECT sales_Id,product_id,ProcessNumber
FROM CTE2
ORDER BY ROWNUM

SQL FIDDLE

Note that the table I created is the result from your sample query in your question. You need to combine them.


How do people recognise the frequency of a played note?

Question: How can people be sure that the A4 note of a Piano is 440Hz? I'm asking in a more practical/physical way. As far as I know, any notes on the keyboard are made of different frequencies, so A4 isn't made by 440Hz alone (if it does make up the sound).
This is how you can have different instruments and say them all play A4 and yet they sound different, they're made of different frequencies with different combination of amplitudes (which makes up the timbre).
Is there some sort of device or way that receives a sound wave and returns a single frequency?
Comment: Are you asking about how a piano tuner might tune a piano, for example (ie: what tools or technology do they use to ensure perfect tuning), or are you asking about humans with perfect pitch (ie: how does the ear and brain identify pitches?).  It's not clear from your question which you mean.  I think either way this probably is a better fit on another stack, either related to technology or human biology, depending on what you're after.
Comment: Yeah in short a practical way would be to represent the signal by a fourier series. This gives you the amplitudes of component frequencies. The highest amplitudes could be indicative of the note moat times. Now obviously you can get into harmony, where you have multiple notes which take front stage. So. There are things called the fundamental. That is the note, then there can be secondary notes as well, which are harmonic. That mean multiple of the period of the fundamental. There could be overtones etc. Lots of stuff.
Answer: You are right that each note contains both a main frequency that defines its pitch and additional frequencies that create the color or timbre of the note. The main pitch is called the fundamental, and it is the strongest frequency that sounds. So no matter what instrument is making the sound, the fundamental frequency will correspond to the note name (A = 440Hz, for instance). An electric tuner can hear a complex instrument, and it will identify the fundamental.
The additional frequencies are known as harmonics. These are multiples of the fundamental frequency, and the amplitude of these various frequencies creates the sound of unique sound of each instrument. Real life sound waves are incredibly complex and also contain frequencies that are not multiples of the fundamental, and there are other factors that determine an instruments character besides the frequency content - most notably attack, but I digress.
There is a tool called a spectrum analyzer that allows you to see the amplitudes of these various frequencies. They were originally pieces physical equipment, but today they are readily available in digital form. They are usually included in DAW packages and even as visualizer plugins for playback software.
Comment: The fundamental is not necessarily the strongest frequency. Indeed, for some instruments it is effectively absent.
Comment: @BobsaysreinstateMonica This is true, but it does not really affect the correctness of my answer. Notice that I said "strongest sounding." While it may or not be the the frequency with the highest amplitude, it is the pitch that our ear recognizes.
Comment: No, that's not what you wrote in your answer.  And fwiw an electronic tuner will not always identify the same fundamental as the human ear, it will only generally do so in the case of harmonic sounds. Many tuned percussion instruments for example have non-harmonic overtone series, electronic tuners can be very bad at determining the correct pitch as they assume a harmonic overtone series.
Comment: The question specifically references pitch (A = 440Hz), so I'm addressing pitched instruments. I believe it is counter-productive to include too much additional info in a basic. Feel free to edit my answer if you want to add something specific.
Comment: Basic spectrum analysers are available as free smartphone apps (e.g. FFT spectrum analyser on Android). For example if I whistle normally I get a very clean spectrum: the fundamental (with side lobes I'll ignore) and a first harmonic at twice the frequency (plus a bit of broadband low frequency from the rushing air).  If I whistle the same note through my teeth I get more harmonics.  If I hum I get more, including one at half the dominant frequency, and if I sing I get the same as humming but much stronger at higher frequencies.  They're not exactly quantitative but the graphs help
Comment: (Of course the dominant frequency to the ear isn't necessarily the dominant frequency to the software)
Answer: The best tool for analysing what pitch will be perceived from a particular complex waveform is... the human ear and brain.   Because it IS about what is perceived, not always about what's actually there!
Mix a set of sine waves corresponding to the upper harmonics of a note, the fundamental will be perceived, even though that frequency doesn't actually exist!
Apparently the sound of timpani lacks the fundamental.  This link has far more than you want to know on the subject.
https://wtt.pauken.org/
Bells are peculiar too.  Their overtones are non-harmonic.  This link is very tecchie, but note that the fundamental isn't always the strongest frequency, or the perceived one.  In fact, perception of a bell's pitch, without the clues of a harmonic set of overtones, can be ambiguous.
http://www.hibberts.co.uk/phd/phdart3.htm
But yes, more generally, the fundamental is there, and is heard as the pitch.  It's not always the strongest component though.  Which is why electronic tuners generally get the name of the note right but are easy to fool about what octave it's in.
Comment: Glad you mentioned bells. I read Aaron's answer and questioned the last statement. Most instruments - not all.
Comment: I too am glad to see something outside the harmonic sequence pointed out.
Comment: Also, people seem to think it's all about the instrument but the attack matters too.  The physics of the instrument may determine what is possible but the attack determines what from those possibilities is actually produced.
Comment: Your right, but the OP asked specifically about frequency, not pitch.
Comment: How would a person recognise a frequency other than by its pitch?
Answer: A long time ago, an instrument called the tuning fork was invented. It was 1711, by a trumpeter and lutenist John Shore, according to Wikipedia. Its tines are a specific length, so that when hit against something hard, it would vibrate (for an A=440Hz) making a pretty pure A note. Held against a wooden surface, it was easily audible.
By using this, and playing the suspect note, it is simple to hear any differences. If the suspect note is within a semitone, beats - fluctuations in volume - will be heard between the two notes.
Something I do with students on guitar is fold a tiny piece of paper, half a stamp, say, and rest it on the A string. Hit the tuning fork and place it on the guitar body.When the string is exactly right, the paper will fall off. Sympathetic vibration causes the string to move. Although here, the string is actually two octaves lower - we can tell just by listening - but it still works. For a more acurate one, capo top string on 5th fret.
Otherwise - electronic tuners tell exactly what note, in what octave it is too.
Answer: A device called a tuner can detect pitch. Although any pitch contains several frequencies, they vary in amplitude (volume). The strongest of these is called the "fundamental", and that is what the tuner detects.
Comment: This is simply false, the strongest is NOT always the fundamental and it's pretty easy to excite wave forms where the fundamental is not the strongest.  I'd recommend editing this or deleting it.
Comment: The fundamental isn't necessarily the strongest.
Comment: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/116937/discussion-on-answer-by-aaron-how-do-people-recognise-the-frequency-of-a-played).
Answer: You seem to be asking several different question.
"How can people be sure that the A4 note of a Piano is 440Hz?"
Based on the title I thought you were referring to person identifying the Hz of a tone by hearing it.  In general, from what I've been taught, most people CANNOT identify absolute frequencies.  We are very good at identifying relative frequencies, i.e. intervals.  But it takes a special gift to ID the frequency by hearing.  I read an article about 20 years ago that claimed people who speak languages that require tone for definition (specifically Chinese) are highly likely to have perfect pitch in adulthood even if they do not show musical talent or interest.
"I'm asking in a more practical/physical way. As far as I know, any notes on the keyboard are made of different frequencies, so A4 isn't made by 440Hz alone (if it does make up the sound)."
This statement hints at two things.  The first being, is there a device that is capable of measuring frequency (or many frequencies). The second being that an instrument produces many frequencies to produce a tone. To the first point, yes there are devices that are capable of registering a frequency and they are very easy to make.  Even before electronic tuners people could dissect the multiple harmonics in a complex tone using resonators.  The idea is based on simple physics.  For a complex system that produces waves we usually have many (possibly and infinite number) of harmonics all mixed up in the wave field.  If you create a mechanical system that has one and only one resonant frequency, say a pendulum, or mass on a spring, and figure out a way to excite that with the acoustic field then it will only move with a large amplitude when its resonant frequency is present in the field.  This type of experimental set up was used by Helmholtz to analyze acoustics.  The method probably goes back thousands of years.
Now we have sophisticated electronic devices that can capture the full signal and dissect it using a mathematical signal processing algorithm called a Fast Fourier Transform.
"This is how you can have different instruments and say them all play A4 and yet they sound different, they're made of different frequencies with different combination of amplitudes (which makes up the timbre)."
This comment speaks to the fact that in complex systems a single excitation will generally produce hundreds or thousands of frequencies related to the vibrational mechanics of the system.  And yes, the tone or timbre of the instrument, i.e. that characteristic sound, comes from two things (1) the mixture of harmonics that is allowed by the instrument and (2) the attack of the musician.
"Is there some sort of device or way that receives a sound wave and returns a single frequency?"
This may NOT be the right thing to do!  It depends, the resonator I described will respond noticeably at one and only one frequency (Newton's second law says it will move in the presence of any force, but resonance means that even a small input at the right frequency produces a very large motion so there is really no chance of confusing the response).  Since it is hard to produce a pure tone without harmonics this type of device is really the only thing that will output a single pure tone in response to an acoustic vibration.  Modern electronic tuners probably respond to either the highest amplitude frequency in the complex waveform (which may NOT be the fundamental) or if there is some sophisticated circuitry in there it may be doing some signal processing to guess at the fundamental.  Vibrating strings and resonant air columns in a tube come pretty close to obeying the "harmonic" relationship fn = n * f1.  And we make instruments to follow this relation.  If a {microphone + data acquisition card + computer} system picked up a wave and found 440Hz and 660Hz you would NOT have a harmonic relation, you would have a perfect 5th and that may mean that you truly have 2 fundamental tones.  The only way to know for sure is some other data, like you are a piano tuner (the person not the device) and you know for a fact that you only hit one key while the others were dampened.  Then perhaps these are 2 harmonics from the sequence (220Hz, 440Hz, 660Hz) and the real note being played was 220Hz.  This can actually happen!  There is some possibility for ambiguity in what is really being produced.  The system could have been excited in such a way that the fundamental was very weak and the first two harmonics were strong, and the noise floor was high enough that the fundamental could not be determined in the spectrum.  There is really no way to know.  If you truly had two notes being played, 440 and 660 then you may have the harmonics of each intertwined.  This produces a signal fingerprint and we can separate the sequences out, inferring that 440 and 660 were really each separately present.  There are a couple things you should know about humans: (1) our ears are non-linear and produce aural harmonics when excited by a single pure tone.  Thus is is not possible for a human to have ever experienced a single frequency, and (2) the brain has the ability to discover the harmonic sequence and determine what the fundamental is even if it's missing.  This is called fundamental tracking.  Because of this you can trick a human into hearing something that is not there, not present in the acoustic field and NOT created by any source.  This makes human's unreliable in an uncontrolled environment.
Lastly, some vibrating systems do NOT follow the harmonic sequence, they have harmonics but they follow some other sequence.  I have personally done research with the acoustics of such objects and can tell you that when you excite multiple harmonics you hear them as different tones rather than a complex wave.  This is probably due to the fact that we have evolved to respond to the harmonic sequence and anything off that our brains handle differently.
When you FFT a complex wave form and try to infer the fundamental using some type of logic there are ways to get the wrong answer.
To answer your question there are in fact devices that will take a complex wave as the input and spit out a single result but they may either be too limited (like the mechanical resonator) or error prone.
Answer: Depends on who you mean by "people".
Professional piano tuners often use a strobe tuner to measure whether the A4 key on a piano produces a sound (within 1 cent or less) of 440 Hz in pitch.  A stobe tuner doesn't display the frequency of the sound.  Instead it shows how close the periodicity of the sound waveform matches a repeat rate of 440 per second, by the drift against the strobe.  Lots of waveforms can repeat at 440 time per seconds, even though they may not look at all like a sine wave with a frequency of 440 Hz.
The highest peak in a sound frequency plot is very often not the pitch of the piano key (or string bass or guitar string or male voice, etc.)  Thus, using just the highest peak in a sound spectrum will often produce a wrong answer for a pitch estimate.
Regular humans (not using mechanical or electronic tuners, etc.) detect pitch by completely different (subconscious/ear-brain) psycho-acoustic mechanisms, for which there are many hypothesis.  One of the better hypothesis, IMO, for sounds with a rich timbre (not pure sinusoids) is that the ear-brain combination looks at the spacing between overtones, and uses that information to guess the pitch, as rich sounds have an overtone series that corresponds well to the mechanical mechanism producing pitched sounds.
Comment: Very interesting. Incidentally many blind people learn to be piano tuners and presumably can't access a strobe.  Presumably they have to use a tuning fork.  P.S. I must buy one of these strobes for tuning my guitar, it sounds like a great idea.


Mostrar o maior e o menor número

Question: Por que está errado?
var n1 = parseFloat(prompt("Digite um número:"));
var n2 = parseFloat(prompt("Digite um número:"));
var n3 = parseFloat(prompt("Digite um número:"));
function maiorDosTres(n1, n2, n3) {
  if ( n1 > n2 > n3) {
    alert( "O maior número é: " + n1 + " e o menor é: " + n3);
  } else if ( n1 > n3 > n2) {
    alert( "O maior número é: " + n1 + " e o menor é: " + n2);
  } else if ( n2 > n1 > n3) {
    alert( "O maior número é: " + n2 + " e o menor é: " + n3);
  } else if ( n2 > n3 > n1) {
    alert( "O maior número é: " + n2 + " e o menor é: " + n1);
  } else if ( n3 > n1 > n2) {
    alert( "O maior número é: " + n3 + " e o menor é: " + n2);
  } else if ( n3 > n2 > n1) {
    alert( "O maior número é: " + n3 + " e o menor é: " + n1);
  }
}
maiorDosTres(n1, n2, n3);

Comment: Já implementaram o Math.max(args) e o Math.min(args)
Comment: Eu não posso usar funções. E o problema é que quando eu executo, em vez de mostrar a string com o maior e menor número, mostra undefined.
Comment: Tem os operadores `&&` e `||` que podem ajudar vc a resolver isso. Falta algo na suas comparações :P
Comment: Obrigado. Ajudou muito.
Comment: Dê uma olhada no [tour]. Você pode aceitar uma resposta se ela resolveu seu problema. Você pode votar em todos os posts do site também. Alguma lhe ajudou mais? Precisa que algo seja melhorado?
Answer: Está errado porque você não pode comparar mais que dois operandos ao mesmo tempo. Isto não existe:
if ( n1 > n2 > n3) {

O correto seria:
if ( n1 > n2 && n2 > n3) {

Coloquei no GitHub para referência futura.
Desta forma você tem três expressões com dois operandos cada: n1 > n2, n2 > n3 e a operação relacional && que tem como operandos os resultados da expressões anteriores.
O mesmo vale para os demais. Acho que já pegou o jeito.
Tem maneiras melhores de fazer isto. Usar Math.max() e Math.min() seria uma delas. Não vou colocar porque me parece que você está fazendo isto para fins de estudo e não quer a melhor solução.
Answer: Você pode utilizar as funções Math.max() e Math.min(). As duas funções recebem todos os valores para comparação como parâmetros. Apesar da a pergunta ser o motivo de estar errado, essa forma deixará o seu código mais enxuto.

var n1 = 10;
var n2 = 5;
var n3 = 7;

var max = Math.max(n1, n2, n3);
var min = Math.min(n1, n2, n3);

var pre = document.body.appendChild(document.createElement('pre'));
pre.textContent = 'dos valores ' + n1 + ', ' + n2 + ' e ' + n3;
pre.textContent += ' o máximo é ' + max + ' e o mínimo é ' + min;

Se os dados estiverem em um array utilize apply():
var numeros = [4,6,2,7,9,5];
Math.max.apply(Math, numeros);

Comment: Math.max() e Math.min() é a maneira semanticamente correta/nativa para comparar valores. `+1`
Answer: Grave os valores em um array e use o "for" para este caso, a função fica mais flexível e serve para qualquer número de argumentos:
var n = [1, 2, 3];

numero=n(i)

for (i = 0; i < n.length; i++) {
    if (n(i) >= numero) { maior = n(i); }
    if (n(i) <= numero) { menor = n(i); }
}
alert( "O maior número é: " + maior + " e o menor é: " + menor);

Comment: Poderia explicar seu código? Fiquei curioso sobre o funcionamento.
Answer: Por que não ordenar os números?
function maiorDosTres() {
    var a = Array.prototype.sort.call(arguments);
    alert( "O maior número é: " + a[a.length - 1] + " e o menor é: " + a[0]);
}

Fiddle
Answer: function maxEMim(n1, n2, n3){
var max = Math.max(n1,n2,n3);
var mim = Math.min(n1,n2,n3);

console.log(`Maior: ${max} e o menor: ${mim}`);

}
maxEMim(10,5,0);
Answer: let n1 = 1;
let n2 = 5;
let n3 = 7;
let max = Math.max(n1,n2,n3);
let mim = Math.min(n1,n2,n3);
console.log(Maior: ${max} e o menor: ${mim});

Mostrar o maior e o menor número


What do I need in order to draw conclusions from this data?

Question: I have three techniques, called A, B and C. Each can be used independently when trying to perform four related tasks (Tasks 1, 2, 3 and 4). I have run lots of tests, trying all combinations of each technique being on or off. My results look something like this. Each number represents how many times the task was completed successfully when attempted, using the given combination of techniques, 100000 times. So, the higher the number, the better.
$$
\begin{array}{l|r|r|r|r|r|r|r|r|r|}
\mbox{Technique $A$} & - & - & - & - & X & X & X & X \\
\mbox{Technique $B$} & - & - & X & X & - & - & X & X \\
\mbox{Technique $C$} & - & X & - & X & - & X & - & X \\ \hline
\mbox{Task $1$} & 433 & 277 & 911 & 492 & 686 & 4211 & 3775 & {\bf 9732}\\
\mbox{Task $2$} & 149 & 1063 & 5562 & {\bf 6035} & 3 & 58 & 1391 & 1708\\
\mbox{Task $3$} & 220 & 1278 & 7014 & {\bf 7018} & 10 & 97 & 2083 & 4452\\
\mbox{Task $4$} & 218 & 1255 & 6142 & {\bf 8656} & 1 & 73 & 1087 & 2056\\
\end{array}
$$
I've highlighted the largest number in each row. Looking at the numbers, it seems that $B+C$ is good for Tasks 2, 3 and 4, and that adding $A$ as well is best for Task 1. But I want to say a bit more. I'd like to be quantitative if I can. My question is: can I deduce anything quantitative from this data? Or do I really need some measure of the variance of the observations? That is, I suspect the numbers might be different if I ran all the tests again.

Edit. I originally presented my question pretending there were 3 techniques, just in order to save space. There are in fact 4. I'm mentioning this now because it might affect how the results are analysed. In particular: note now that the `intercept' is zero, i.e. there is no success at all when none of the techniques are used.
$$
\begin{array}{l|r|r|r|r|r|r|r|r|r|}
\mbox{Technique $Z$} & - & - & - & - & - & - & - & - & X & X & X & X & X & X & X & X \\
\mbox{Technique $A$} & - & - & - & - & X & X & X & X & - & - & - & - & X & X & X & X \\
\mbox{Technique $B$} & - & - & X & X & - & - & X & X & - & - & X & X & - & - & X & X \\
\mbox{Technique $C$} & - & X & - & X & - & X & - & X & - & X & - & X & - & X & - & X \\ \hline
\mbox{Task $1$} & 0 & 0 & 0 & 0 & 0 & 2700 & 1 & 10113 & 433 & 277 & 911 & 492 & 686 & 4211 & 3775 & {\bf 9732}\\
\mbox{Task $2$} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 149 & 1063 & 5562 & {\bf 6035} & 3 & 58 & 1391 & 1708\\
\mbox{Task $3$} & 0 & 0 & 0 & 0 & 0 & 664 & 0 & 3043 & 220 & 1278 & 7014 & {\bf 7018} & 10 & 97 & 2083 & 4452\\
\mbox{Task $4$} & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 218 & 1255 & 6142 & {\bf 8656} & 1 & 73 & 1087 & 2056\\
\end{array}
$$
Here are those numbers in by-column form (for pasting into R):
0,    0,    0,    0,
0,    0,    0,    0,
0,    0,    0,    0,
0,    0,    0,    0,
0,    0,    0,    0,
2700, 0,    664,  0,
1,    0,    0,    0,
10113,0,    3043, 0,
433,  149,  220,  218,
277,  1063, 1278, 1255,
911,  5562, 7014, 6142,
492,  6034, 7018, 8656,
686,  3,    10,   1,
4211, 48,   97,   73,
3775, 1391, 2083, 1087,
9732, 1708, 4452, 2056

Comment: What are the numbers in the cells of the table? Are they from one person? Average of many people? Or what?
Comment: @PeterFlom Thanks for your question. I've edited my question to include the answer. The tasks are performed by computer, not by people. Each number represents the number of successfully-completed tasks out of 100000 attempts.
Answer: You can use logistic regression. The dependent variable would be "success" and the independent variables will be technique and task. Task could be coded as three 2-level dummy codes plus interactions. Or, you could code it as one categorical variable with 8 levels. These are equivalent. The former is easier to code but the latter may be easier to understand.
Answer: It is a good question, reflecting the understanding that data are only an uncertain reflection of reality.  Any solution should therefore attempt to quantify this uncertainty.
The numbers would indeed be different if the tests were run with different inputs.  Assuming these runs are considered representative of future runs and they were independent of each other, we can assess how much random variation there will likely be and use that to inform our estimates of how much each technique influences the result.
Because the proportions of successes were so small, they can be modeled with a Poisson Generalized Linear Model.  (Alternatively, use a Binomial GLM.  In this case it will give almost identical results.)  This model can be used to estimate the contributions of each technique to the successes in each task.  To evaluate the variability of these estimates, bootstrap the data: repeatedly sample with replacement from these 3.2 million results and re-estimate the coefficients.  Use the variability of the repeated estimates to assess the uncertainty in the model coefficients.  This can be done separately for each task.
Creating a new sample of 3.2 million values is much simpler than it sounds, because each of the counts is modeled as a binomial outcome.  Thus only 8 random numbers have to be regenerated for each bootstrap trial of each task.
Here is an example of (some of) the information that can be produced with this approach.  It is based on 99 bootstrap trials.
======================
        Task 4
======================
            Actual    Mean     SD       Z     p     Min    Max
(Intercept)  5.384   5.383  0.069  78.568 0.000   5.198  5.557
A           -5.384 -13.013 10.915  -1.192 0.233 -27.859 -3.766
B            3.338   3.340  0.070  47.952 0.000   3.181  3.526
C            1.750   1.754  0.075  23.441 0.000   1.568  1.964
A:B          3.653  11.282 10.917   1.033 0.301   2.039 26.085
A:C          2.540  10.153 10.895   0.932 0.351   0.767 25.137
B:C         -1.407  -1.410  0.076 -18.649 0.000  -1.632 -1.228
A:B:C       -2.246  -9.862 10.897  -0.905 0.365 -24.880 -0.485

This is a "saturated" model in which the results are teased apart into eight contributions, each of which is to be added and then exponentiated to predict the
number of successes per $10^5$ runs.  The Actual column presents the estimates based on the original data.  The other columns summarize the repeated estimates, as explained below and in the code.

The "intercept" is a base level reflecting overall success rates for all techniques.
"A", "B", and "C" are contributions attributed to those techniques (assuming "X" means the technique is used and "-" means it is not in the table in the question).
"A:B", "A:C", and "B:C" are contributions from combinations of pairs of techniques ("interactions").
"A:B:C" is a contribution from the simultaneous use of all three techniques (a "three-way interaction".

For instance, when both $A$ and $C$ are used for Task 4, the predicted rate of success is computed as
$$\exp(5.384 - 5.384 + 1.750 + 2.540)/10^5 = 73/10^5,$$
exactly as in the data.  However, each of these coefficient estimates is uncertain: the initial $5.384$ has a standard deviation of only $0.069$, but the $-5.384$ associated with $A$ has an SD of $10.915$, making it uncertain whether it even differs from $0$ (the p-value for this test is only $0.233$, which nobody would consider significant).  Indeed, it is doubtful whether technique $A$ contributes anything to the success of Task 4, because all the p-values associated with it ($A$, $A:B$, $A:C$, and $A:B:C$) are high.  Moreover, the large discrepancies between the estimates based on the actual data and the mean of the bootstrapped estimates for the "$A$" columns suggest the actual estimates might be biased--this deserves further investigation, perhaps with a more careful examination of the results of a larger number of bootstrap trials.
(Without the bootstrapping, all we could have obtained is the first column of this output: the estimates based on the actual data.  Because the model is saturated, it perfectly fits the data, leaving no degrees of freedom to assess uncertainty.)
The R code that produced this result produced similar information for the other three tasks.  It should be self-explanatory.
technique <- expand.grid(Task=1:4, C=0:1, B=0:1, A=0:1)
technique$Success <- c(433, 149, 220, 218,
                           277, 1063, 1278, 1255,
                           911, 5562, 7014, 6142,
                           492, 6034, 7018, 8656,
                           686, 3, 10, 1,
                           4211, 48, 97, 73,
                           3775, 1391, 2083, 1087,
                           9732, 1708, 4452, 2056)
    n.trials <- 1e5
    n.iter <- 99 # Set the number of bootstrap iterations
    set.seed(17) # Create a reproducible starting point
    for (task in 1:4) {
      #
      # Extract the data for this task.
      #
      d <- e <- subset(technique, subset=(Task==task))
      coefficients <- matrix(NA, n.iter, 8)
      for (i in 1:n.iter) {
        #
        # Fit these data.
        #
        fit <- glm(Success ~ A+B+C + A:B + A:C + B:C + A:B:C,
                   family=poisson(link="log"), data=d)
        coefficients[i, ] <- coef(fit)
        #
        # Create a new bootstrap sample.
        #
        d$Success <- rbinom(dim(d)[1], n.trials, e$Success/n.trials)
  }
  #
  # Summarize the results.
  #
  dimnames(coefficients) <- list(Iteration=1:n.iter, Coefficient=names(coef(fit)))
  results <- apply(coefficients, 2,
                   function(b) {
                     m <- mean(b)          # Mean estimate
                     s <- sd(b)            # Standard deviation of the estimates
                     z <- m/s              # A Z statistic
                     p=2*(1-pnorm(abs(z))) # The p-value
                     c(Mean=m, SD=s, Z=z, p=p, Min=min(b), Max=max(b))
                     })
  results <- rbind(Actual=coefficients[1, ], results) # Best (original) estimates
  cat("=======================\n")
  cat(paste("        Task", task))
  cat("\n======================\n")
  print(t(round(results, 3)))
}

Comment: Wow! Thank you so much for this incredibly comprehensive answer. I'm in the process of working through it now, so I may come back to you with requests for clarifications shortly. But in the meantime, *thank you*! By the way, your R code has a small bug: replace `n.tries` with `n.trials`.
Comment: Thank you for catching the typo--it was a vestige of an earlier name for the same variable, and so the code worked on my system. (Which is a relief, because it means the output was unaffected.)  The non-zero intercept reflects the overall average success rate: it has nothing to do with the presence or absence of other techniques.
Comment: I've edited my question to mention the fact that I had lied a bit: in fact there is a fourth technique, which in my original question was always switched on. That accounts for the non-zero intercept.
Answer: This isn't really an answer, but a report on 'what I did in the end'. I would very much appreciate it if those who kindly provided the answers above would glance over the following text and see if it's "ok".
User whuber suggested considering each combination of techniques in turn, but in what follows, I've opted just to consider each technique individually. It feels much simpler to me, and simplicity is what I'm aiming for, but I would be interested to know if it's still statistically sound.
Here is what I wrote...

To give a rough idea of the relative effectiveness of each
  technique, we perform a logistic regression analysis. The four
  techniques provide four binomial input variables, while the output
  variable is the success rate for each task. Since the success rates
  are quite low, a Poisson model is appropriate. In the following table,
  we gather the coefficients of this model. The standard deviations are
  around a couple of percent, so we give the figures to one decimal
  place.
$$
 \begin{array}{lrrrr}
 & \mbox{Task $1$} & \mbox{Task $2$}  & \mbox{Task $3$} & \mbox{Task $4$} \\
(intercept) & 3.6 & -17.1 &  4.3 & -17.1 \\
\mbox{Technique $Z$} & 0.5 &  23.2 &  1.8 &  23.3 \\
\mbox{Technique $A$} & 2.7 &  -1.4 & -0.4 &  -1.6 \\
\mbox{Technique $B$} & 1.1 &   2.5 &  2.3 &   2.5 \\
\mbox{Technique $C$} & 1.6 &   0.2 &  0.6 &   0.5 \\ \end{array}
 $$
The smaller the coefficient, the less the technique contributes to the
  success rate; indeed, when the coefficient is negative, the technique
  is actually a hindrance. The `intercept' coefficients represent the
  success rate averaged across all techniques.
In summary, we find that Technique $Z$ has by far the highest impact,
  and that Technique $A$ tend to decrease the success rate except when
  performing Task $1$. Techniques $B$ and $C$ are consistently helpful.

This is the R script I ran in order to justify the text above
It's just a slightly modified version of whuber's script.
technique <- expand.grid(Task=1:4, D=0:1, C=0:1, B=0:1, A=0:1)
technique$Success <- c(
    0,    0,    0,    0,
    0,    0,    0,    0,
    0,    0,    0,    0,
    0,    0,    0,    0,
    0,    0,    0,    0,
    2700, 0,    664,  0,
    1,    0,    0,    0,
    10113,0,    3043, 0,
    433,  149,  220,  218,
    277,  1063, 1278, 1255,
    911,  5562, 7014, 6142,
    492,  6034, 7018, 8656,
    686,  3,    10,   1,
    4211, 48,   97,   73,
    3775, 1391, 2083, 1087,
    9732, 1708, 4452, 2056
    )
        n.trials <- 1e5
        n.iter <- 99 # Set the number of bootstrap iterations
        set.seed(17) # Create a reproducible starting point
        for (task in 1:4) {
          #
          # Extract the data for this task.
          #
          d <- e <- subset(technique, subset=(Task==task))
          coefficients <- matrix(NA, n.iter, 5)
          for (i in 1:n.iter) {
            #
            # Fit these data.
            #
            fit <- glm(Success ~ A+B+C+D, family=poisson(link="log"), data=d)
            coefficients[i, ] <- coef(fit)
            #
            # Create a new bootstrap sample.
            #
            d$Success <- rbinom(dim(d)[1], n.trials, e$Success/n.trials)
  }
  #
  # Summarize the results.
  #
  dimnames(coefficients) <- list(Iteration=1:n.iter, Coefficient=names(coef(fit)))
  results <- apply(coefficients, 2,
                   function(b) {
                     m <- mean(b)          # Mean estimate
                     s <- sd(b)            # Standard deviation of the estimates
                     z <- m/s              # A Z statistic
                     p=2*(1-pnorm(abs(z))) # The p-value
                     c(Mean=m, SD=s, Z=z, p=p, Min=min(b), Max=max(b))
                     })
  results <- rbind(Actual=coefficients[1, ], results) # Best (original) estimates
  cat("========================\n")
  cat(paste("        Task", task))
  cat("\n======================\n")
  print(t(round(results, 1)))
}

And this is the data that running that script produces:
========================
        Task 1
======================
            Actual  Mean    SD       Z p   Min   Max
(Intercept)  3.563 3.565 0.032 111.734 0 3.480 3.641
A            0.471 0.470 0.009  50.772 0 0.449 0.493
B            2.693 2.692 0.025 106.485 0 2.633 2.761
C            1.103 1.102 0.012  89.164 0 1.062 1.133
D            1.556 1.556 0.015 107.043 0 1.512 1.598
========================
        Task 2
======================
             Actual    Mean    SD        Z p     Min     Max
(Intercept) -17.104 -17.105 0.022 -760.229 0 -17.174 -17.048
A            23.216  23.220 0.017 1402.716 0  23.187  23.268
B            -1.403  -1.400 0.021  -68.192 0  -1.454  -1.329
C             2.454   2.451 0.028   86.965 0   2.383   2.519
D             0.220   0.221 0.014   15.949 0   0.189   0.260
========================
        Task 3
======================
            Actual   Mean    SD       Z p    Min    Max
(Intercept)  4.253  4.255 0.023 181.558 0  4.198  4.308
A            1.789  1.786 0.018  98.931 0  1.742  1.832
B           -0.406 -0.404 0.013 -31.285 0 -0.444 -0.369
C            2.342  2.341 0.020 116.633 0  2.292  2.395
D            0.574  0.573 0.011  49.948 0  0.538  0.595
========================
        Task 4
======================
             Actual    Mean    SD        Z p     Min     Max
(Intercept) -17.066 -17.068 0.019 -906.664 0 -17.120 -17.020
A            23.268  23.269 0.014 1673.551 0  23.241  23.307
B            -1.621  -1.619 0.019  -84.484 0  -1.672  -1.571
C             2.451   2.452 0.024  102.533 0   2.403   2.504
D             0.480   0.479 0.015   32.899 0   0.447   0.515

Comment: In my answer I recommended models that include the interactions because they are *hugely* better than the simplified models here.  You can test this by comparing their AICs or by examining the p-values of the interaction coefficients. It is possible to remove some of those interaction variables for various tasks, but if you want a reasonably accurate and predictive model then you must include some of the interactions.  The predictive abilities of these models can be checked with cross-validation or by testing against a held-out portion of the dataset.
Comment: @whuber thanks for your comment. I can well imagine that models that include interactions can be a lot more faithful. That said, what I'm after is something really simple. Thing is, this analysis is really not central to my article, it's really just a remark. So would you say that even if my model is not very accurate, my text does at least describe a sound application of the principles?
Comment: I don't know what to say. Comparing the estimated coefficients of $A$ through $D$ for the models with and without interactions indicates the two approaches lead to two quite different conclusions. They cannot both be correct. If you value simplicity over predictability or insight, then you might still favor the simpler model--but it is not clear that your readers would share that opinion. To me, and perhaps to them, it is very interesting that the interactions are not only significant but *strong* in the sense of having an important bearing on understanding the relationships in the data.


nodejs mongodb object id to string fontend

Question:
I can't get data from mongo because ID is ObjectID.
I want to convert it to string.
I can't get data from mongo because ID is ObjectID , I want to convert it to string
enter code here

const SingleProduct = () => {
    const {id} = useParams();
    const [data, setData] = useState([]);
    // const [filter, setFilter] = useState(data);
    const [loading, setLoading] = useState(false);
    let componentMounted = false;

    useEffect(() => {
        const getProducts = async () => {
            setLoading(true);
            // Link database product
            const response = await fetch(`http://localhost:8000/api/products/${id}`);
            setData(await response.json());
            setLoading(false);
        }
        getProducts();
    }, []);

Comment: You probably have not defined the `id` parameter in your `Route` so `const {id} = useParams()` returns `undefined`. Can you post the code where you specify all the routes in your front end?
Comment: you can see code my GitHub: https://github.com/khuongviettai/Learn_nodejs.git
Answer: In your frontend/src/components/products/Products/Products.jsx file you should probably pass the _id of the item to the Item component:
return (
  <>
    {data.map((item, index) => {
      return (
        <Item
          id={item._id} // Here pass the ObjectId of the item, since `id` is not defined
          img={item.image}
          title={item.title}
          desc={item.desc}
          price={item.price}
          item={item}
          key={index}
        />
      );
    })}
  </>
);

Comment: Thank you. The solution is really effective


Select options from JSON file

Question: I have JSON file with some id and value, and this need to be populated in form as select list:
$form['select_item'] = [
      '#type' => 'select',
      '#options' => $list,
      '#title' => $this->t('Choose'),
      '#required' => TRUE,
    ];

What is the best way to approach this. Can I somehow convert this JSON in array or something and use it in options?
I would like to have this values form JSON in separate file and to use it from there.
Any idea?
Answer: Drupal doesn't have a lot to help with this, it's really a general PHP task.
If you're determined to use the API for it, just load the file data in any way you like (fopen, file_get_contents, etc), and use Json::decode() to turn it into an array.

Select options from JSON file


If $H<G$ is of finite index, and for some $x\in G$, $xHx^{-1}\subset H$, prove that $xHx^{-1}=H$

Question: The original question had 2 parts:

a. If $H<G$ is finite, and for some $x\in G$, $xHx^{-1}\subset H$, prove that $xHx^{-1}=H$
b. If $H<G$ is of finite index, and for some $x\in G$, $xHx^{-1}\subset H$, prove that $xHx^{-1}=H$

Now I was able to solve part (a) by finding an injection from $H$ to $xHx^{-1}$, but the same approach doesn't seem to work in (b). There also seems to be a lot of (duplicate) questions here about the status without the limitation on $H$, but none of the responses seem to help me in my case.
How is the finite index of $H$ supposed to affect my thinking?
Answer: Hint: If $xHx^{-1}\subseteq H$ then show $x^{k+1}Hx^{-k-1} \subseteq x^{k}Hx^{-k}$ for all natural numbers $k$. If $H$ has finite index in $G$, then for some $m$, $x^m\in H.$
Comment: Consider the cosets $x^mH$ where $m$ varies over the natural number. There must be finitely many distinct such cosets (by finite index,) so ... @Nescio
Comment: Why does the second statement, about $x^m$ being in $H$ for some $m$, follow from the finite index of $H$?
Comment: I see.. Thanks!


Will I improve through "sloppy" practice?

Question: I am taking piano lessons as an adult (at roughly an early intermediate level), an hour each week. For the most part I am having tremendous fun, and my teacher is extremely kind, but a little bit of frustration is starting to set in. Her advice aligns with other things I have read on the internet, and I'm pondering how strictly to stick to her advice.
I would like to try playing a lot of different music, some of it above my level, without worrying too much about perfecting it. For example this summer (shortly after I'd started) there was a month break in the lessons and I spent a lot of time hacking through pieces like Bach Inventions without worrying too much about my mistakes. It was a lot of fun, and I felt like I was definitely improving -- even if I knew it would be a very long time before I could play these pieces at performance level.
My teacher discourages this approach (and discourages me attempting Bach's Inventions at all). She is picky about phrasing, dynamics, and articulation -- for me, the musical equivalent to being asked to eat vegetables instead of pizza. (Yes, vegetables are good for you!) She asks me to keep practicing the same pieces week after week, even after I have gotten pretty good at them. Even though l have room to improve, I confess to getting a little bored of the music. I don't feel like I'm improving as fast (even though I suspect I actually am).
I am not naive -- I understand that all of this is very likely to my long-term benefit, at least if I aim to develop to an advanced level. Indeed, as a professional math teacher my own approach has a huge amount in common with hers!
I understand that I should spend a substantial proportion of my practice time taking my teacher's advice fully to heart. Nonetheless -- would making additional time for some "sloppy" and "fun" practice also contribute to my long-term improvement?
Comment: My piano teacher once said something along the lines of the following. It's better to slow down and play the piece accurately than to speed up and do it sloppy. Play accurately (and as slow as needed) when you care about the learning process. Play sloppy when you're concerned about the outcome here and now. http://www.bulletproofmusician.com/is-slow-practice-really-necessary/
Comment: @ChristiaanWesterbeek- that's exactly my experience as well.  I learned to play the recorder fast and sloppy and got pretty good, but I hit a wall.  I had to start over, taking it slow, relaxed, and rhythmic, before I could progress.
Comment: Thanks everyone for the answers. I didn't accept any of them (since there doesn't seem to quite be a clear consensus), but +1 all around. In any case, I'm soon leaving town for five months -- and perhaps when I find a new teacher there, or when I come back to town, will be a particularly good opportunity to revisit afresh what I hope to accomplish. Thanks to all!
Answer: As a child, I had the same sort of teachers. Everything was exam orientated, and perfection was the goal. It very nearly stopped me playing piano, but did pass grade VIII for me. However, that goal was probably all those teachers saw. Not one saw me learning music, only passing exams. "Give me a monkey, a piano, and enough time..."
Please continue with 'sloppy practice', as it's going to do you a lot of good later, whether it's Bach, showtunes, pop tunes, jazz, etc. All playing will improve your piano playing, and broaden your horizons, which I expect won't happen with the current teacher.
When teaching, I try to go somewhat in the direction the student wants, and on the way, get the techniques sorted through what they enjoy playing. That way, they probably practise more. Even with them playing stuff I can't stand!
You are paying, so you ought, as an adult, have some say in the deal.Yes, she's the expert, but you need to discuss. Maybe she'll say it's her way or no way, but at least talk frankly.
Answer: I am not a teacher. My son was given Bach two parts Invention 8 when he was turned 7 year old. The teacher was very strict on polish this. My son spent 2 months on this song. At first I am not totally agree with the teacher --- my son was so little, why not allow him a sloppy practice for some fune. This is a very hard piece anyway. Then the teacher explained, that this Bach Invention, without correct dynamics, two-hand-balancing and clear articulate, its level would down to grade2. Only by apply all these requirements, this song becomes a grade 7 or 8.
By practicing as the teacher requested, my son learned a lot --- he begins to listen to his both hands, pays attention to his fingers to get a better sound quality, and endured two months to perfect one song (not easy for a 7 year old).
After get used to heard my son's play, I have heard many other kids play this piece. Most of them are very Sloppy. I kind of feel sorry for these kids who had spent weeks of time to practice but never had once feel the beauty of the song. After finish this song, my son was in a fast track when learned new pieces, because he gained some self-conscious for his own playing sound. He passed his RCM8 exam at age 8.
If your purpose of learning piano is playing as many piece for fun as possible (sloppy or not), then you use your own way to practice. If you look each piece as ladder that lead you to a better pianist, you should listen to you teacher, he/she is far more better know you than yourself could be.
Answer: I would say yes, 'sloppy' practice as you describe is a good thing.
Firstly, a word of warning. I'm nowhere near good enough to be a classical piano teacher, and my abilities when it comes to reading Bach are far worse than they should be. I'm much more comfortable with a chord chart than with sheet music. So some grains (or tablespoons) of salt may be required.
I think the different kinds of practice teach you different things. Honing in on the fine detail of a piece refines your technique. More importantly, it makes you aware of all the musical details that you miss on a first read. As you say, this includes things like phrasing, articulation, dynamics, and the like.
If the only outcome of this focused attention is that you can play one specific piece very well, then I think it's a waste of time. I think the intention is to teach you how to apply those things to the next piece you encounter, and the next piece, and so forth. It should teach you how to play piano, not how to play one specific piece.
The 'sloppy' practice can teach you other things. At a minimum, it will hone your sight reading skills. It will also expose you to much more music, which is excellent. True, it could possibly result in suboptimal technique, but I doubt that is a significant risk, because your teacher will catch it. More importantly, it will help you enjoy playing the piano. Sod getting all the notes right, play some music! Too many people give up music because they get sick of playing the same piece ad infinitum. I nearly did.
So, whilst I'm not an expert here, I'd say that you should do both, and both have their benefits. In fact, I would hazard a guess that you'll be a better piano player for it. If it's causing conflict, I might just forget to mention it to your teacher. My reading between the lines says that she's worried you'll get distracted, and stop working on the technical stuff. If you can show that isn't an issue, I can hardly see why she would object to a student playing too much piano.
Comment: this... "should teach you how to play piano, not how to play one specific piece" ...should be the goal, in my opition.
Answer: It's rather hard not to improve by excessive practice, even sloppy and without oversight.  The problem is that you'll also improve with masking your problems and getting used to them.
Ultimately, however, the progress on any amount of practice will be plateauing out.  And your workarounds may end you on a plateau that is higher than what you started with but quite off from your peak capacity, and you might be in for quite a lot of painful backtracking through interconnected faulted abilities if you want to get better than you are.
If your goofing around makes the difference between becoming hooked for life or getting frustrated enough to quit eventually, it may still be worth it.
For your teacher, it means a lack of control over your progress and your practice and lacklustre progress on things she considers in most need of improvement.  That's frustrating for her (and may affect how much she can inspire you) and makes her less effective (which is partly a problem of your own expectation of return from investment).
And she might have a point.  Try recording yourself occasionally and listen to the recording paying attention on what she is trying to tell you.  There are things easier to notice/realize from the position of a listener than a player, and of course those will affect how much use you can make of your musical abilities for more than your own amusement.
Answer: If you have any specific goals other than to enjoy yourself by making music, then you will want to do what is necessary to accomplish those other goals, starting with discussing those goals with your teacher.
If like me you are mostly focused on having fun and participating in the joy of music, then fun practice definitely serves that goal. That was is my approach to music and I've definitely continued to improve on all the instruments I play.
That said, I've never paid for a teacher, because I have tons of fun without one. I could see your teacher being frustrated with goals that seem frivolous. Either way you might want to make sure you're on the same page so you're getting the most bang for your buck.

Edit:
I've gotten more serious about music and I'm currently taking lessons from three different teachers (on three different instruments). Having a teacher is much more effective than teaching yourself, if you can afford it. And it can still be fun if you find a teacher who is a good fit.
Answer: You will improve, especially if you're trying new things in your sloppy practice.
The one thing I would be wary about is establishing bad habits for particular passages or musical patterns/motives.Continuing to see good piano teacher will help to mitigate that.
I think your plan of spending a substantial time practicing like your teacher advises, and and supplementing it with practicing whatever and however you like will benefit you in the long run.
Comment: This is never true.  You only improve sloppiness.
Answer: Adult piano students notoriously give up through frustration at how initial rapid progress has to give way to a lengthy period of consolidation.  And, unlike younger players, they can be 'entitled students' who want to control WHAT they learn and HOW they do it.
In a good teacher/student relationship there will be both "NO! Play it slower and play it RIGHT!" and "Here, have a lollipop!"   All playing is good.  Some is more useful than other in developing your skills - the skills that will enable you to play "fun" music more effectively.
Answer: I read a great quote recently: "practice makes permanent."
Are you practicing sloppy? Then you will develop permanent sloppy playing.
Are you occasionally playing sloppy when sight reading or just plowing through a piece to try it out, or perhaps to analyze it? Are you just letting it all hang out after an hour of concentrated practice. That kind of sloppiness shouldn't become permanent.
Be honest with yourself about how you want to develop and then practice accordingly.
I don't have a teacher, but I wouldn't want one that didn't understand this difference.

EDIT
I want to add a comment about something that happened to me recently. I was trying to read through part of K.333. (I was reviewing part of it for a post here on Stack Exchange.) While trying to read it I could feel two impulses: one was to just let my hands reflexively move, the other was to stop and check I was not going to play a wrong note. (Notice the attention to pitch, but neglect of rhythm.) I really wish I could shut down the later impulse. So what if I hit a wrong note, but got most of it right? My goal was to plod through and get the jist of the music. In this case 'sloppy' might also be an important stage in developing reflexive movements. Surely if I keep 'stopping to check' that will be a become a larger block to development.
I think if one digs deeper into this topic there will be a distinction between impromptu or improvised playing and recital playing and differing approaches to practice for developing skills in either area.
Answer: I am a music teacher and professional musician and I'd say...
(1) stop judging what your teacher tells you based on internet sources, this is offensive.  It's like telling your doctor "That's not what I read on WebMD".  If your teacher is a pro then do what they say, they know better.
(2) you will never get better with sloppy practice.  This is a scientific fact.  If you practice sloppy you reinforce slop.  You are rewarding yourself for making mistakes.  Your instructor is trying to give you advice that will result in good technique and easy, fast, development.
(3) if you don't care about getting better than just don't do it.  But please don't ask communities of professional musicians to ballast your position.  If it's a hobby and you are fine being sloppy then just be sloppy, but perhaps stop taking lessons and just use youtube resources for lessons.
Answer:
She is picky about phrasing, dynamics, and articulation -- for me, the musical equivalent to being asked to eat vegetables instead of pizza. (Yes, vegetables are good for you!)

Why not the other way around? Let me explain with a pizza as metaphor.
A pizza has two main parts, the pizza bread and the toppings.
Imagine you made a pizza only with the bread. It can be a great bread especially if it has a nice crunchy crust, but you would certainly like some toppings, typically cheese and tomato but also many other things and maybe some spices on the top.
With music it is similar. Playing the right notes at the right time is like the pizza bread. It can certainly be nice and great. But if you add toppings on the pizza you enter another level of excitement. Phrasing, dynamics, and articulation are the toppings and the spices which brings the music into another level of excitement.
It can be summarized with one word: "care". You care for dynamic expression, quality of tone, phrasing, articulation, you bring fantastic life to the music.
Well, that doesn't mean you can not dabble with different pieces of music. Of course you can. What you learn from your teacher you can apply in other pieces even when you dabble with something.
Actually I find it is a great sign when a student of mine tries out music which we didn't cover in the lessons. It shows that the student can utilize his/her skills.
How long time I let a student work with one piece totally depends on the student. Sometimes it can be a good idea to work with more than one piece, where one piece could be a long time commitment and shorter easier pieces can bring variation. Other times only one piece is relevant; for some students you do need to go on with something else even you would like to work more on that particular piece; with other students you can go into heavy work with details. Often you can take up a piece the student learned in the past and bring that piece into a new level of quality.
I work on a music school where there are concerts or recitals now and then. Preparing for an upcoming event give birth to great motivation.


How to check a string contain NSattributedstring or not

Question: I have add a image to a textview's text by adding NSattributedstring.
The textview can be edited by the user.
How can I check that is the textview's text still containing the NSattributedstring?
by
UIFont *font = [UIFont fontWithName:@"Helvetica-Bold" size:14.0];
NSDictionary *attrsDictionary = [NSDictionary dictionaryWithObject:font forKey:NSFontAttributeName];
NSString *temp1 = [NSString stringWithFormat:@"%@\r\n\r\n",contenttextview.text];
NSMutableAttributedString *attributedString = [[NSMutableAttributedString alloc] initWithString:temp1 attributes:attrsDictionary];
NSTextAttachment *textAttachment = [[NSTextAttachment alloc] init];
textAttachment.image = [self imageWithImage:chosenImage scaledToSize:CGSizeMake(200, 200)];
NSAttributedString *attrStringWithImage = [NSAttributedString attributedStringWithAttachment:textAttachment];
[attributedString replaceCharactersInRange:NSMakeRange(attributedString.length -1, 1) withAttributedString:attrStringWithImage];
contenttextview.attributedText = attributedString;

How can i check that?
Comment: If you add the `NSAttributedString` you know it's range. You can check if the range still has attributes.
Comment: textView.attributedText length will return all the number of string in it..
Comment: How to check has attributes?
Comment: Check its `attributedText` ?
Comment: You can use a for loop from `0` to `contentView.attributedText.length-originalAttributedText.length` and use `attributedSubstringFromRange:` and compare with `isEqualToAttributedString:`. Do not forget to do the necessary checks for the ranges (contentView.attributedText.length < originalAttributedText.length`, returns FALSE, etc. If you want only to check the image: http://stackoverflow.com/questions/29152660/extract-uiimage-from-nsattributed-string/29153172#29153172
Answer: Use this code,
Got the fine code from here..
//First convert your string to array
NSArray *array = [contenttextview componentsSeparatedByString:@" "];
 for(id object in array)
 {
     //Cycle thru array and use below method that is any part of string is image? If return yes even single time that means there is a image.
     BOOL containImage = [object canBeConvertedToEncoding:NSASCIIStringEncoding];
     if(containImage)
       {
           NSLog(@"containImage: NO");
       }
       else
       {
           NSLog(@"containImage: YES");
       }
 }

//This method will be used to check if string does contain image
- (BOOL)stringContainsEmoji:(NSString *)string {
    __block BOOL returnValue = NO;
    [string enumerateSubstringsInRange:NSMakeRange(0, [string length]) options:NSStringEnumerationByComposedCharacterSequences usingBlock:
     ^(NSString *substring, NSRange substringRange, NSRange enclosingRange, BOOL *stop) {

         const unichar hs = [substring characterAtIndex:0];
         // surrogate pair
         if (0xd800 <= hs && hs <= 0xdbff) {
             if (substring.length > 1) {
                 const unichar ls = [substring characterAtIndex:1];
                 const int uc = ((hs - 0xd800) * 0x400) + (ls - 0xdc00) + 0x10000;
                 if (0x1d000 <= uc && uc <= 0x1f77f) {
                     returnValue = YES;
                 }
             }
         } else if (substring.length > 1) {
             const unichar ls = [substring characterAtIndex:1];
             if (ls == 0x20e3) {
                 returnValue = YES;
             }

         } else {
             // non surrogate
             if (0x2100 <= hs && hs <= 0x27ff) {
                 returnValue = YES;
             } else if (0x2B05 <= hs && hs <= 0x2b07) {
                 returnValue = YES;
             } else if (0x2934 <= hs && hs <= 0x2935) {
                 returnValue = YES;
             } else if (0x3297 <= hs && hs <= 0x3299) {
                 returnValue = YES;
             } else if (hs == 0xa9 || hs == 0xae || hs == 0x303d || hs == 0x3030 || hs == 0x2b55 || hs == 0x2b1c || hs == 0x2b1b || hs == 0x2b50) {
                 returnValue = YES;
             }
         }
     }];

    return returnValue;
}

If containImage return YES that means it does have image, Remember that you must have space before and after of image.
This is for other user,<>
If you want to add image in text, than use below code..
NSMutableAttributedString *attributedString = [[NSMutableAttributedString alloc] initWithString:@"Good   Image"];
NSTextAttachment *textAttachment = [[NSTextAttachment alloc] init];
textAttachment.image = [UIImage imageNamed:@"image.png"];
CGFloat scaleFactor = 0;
textAttachment.image = [UIImage imageWithCGImage:textAttachment.image.CGImage scale:scaleFactor orientation:UIImageOrientationUp];
NSAttributedString *attrStringWithImage = [NSAttributedString attributedStringWithAttachment:textAttachment];
[attributedString replaceCharactersInRange:NSMakeRange(5, 1) withAttributedString:attrStringWithImage];
self.aTextView.attributedText = attributedString;

NSLog for Array,
array: (
    Good,
    "\Ufffc",
    Image
)

NSLog for Boolean Variable,
contianImage: NO
contianImage: YES
contianImage: NO

And this is the image i have added,

For your code its working as well,
Check my log, which generated using your code,
2016-06-16 12:52:15.030 Imagetest[2394:109009] array: (
    "
\n
\Ufffc"
)
2016-06-16 12:52:15.031 Imagetest[2394:109009] contianImage: YES

Comment: No, Have checked with image, and its working, check my way of adding image in Text


Panels block title translation

Question: I use blocks in a panel with Drupal 8. With view generated blocks there are no problems: The block titles are translated accordingly to the view settings. With custom blocks, it appears to be problematic to get the block title translated. I hoped that these titles would be translatable via the user interface translation but they are not; plus, in the panel settings, I can set the block title, but I cannot translate it.
How can I translate block titles (and maybe block content) of custom blocks in a panel, with Drupal 8?
Answer: You can override the block.html.twig template and handle the translation of the title directly on the template:
<code>    {{ label|t }}
</code>
You may need to confirm the title structure (if is #markup or plain text/string)
Answer: Page variants are translatable using config translation at /admin/config/regional/config-translation

Panels block title translation


Angular Controller not receiving values from ng-model

Question: var app = angular.module("colorIt",[]);
app.controller("ColorCtrl",function() {
  this.shape = '';
  this.style = {
    'border-color': '',
    'background-color': '',
    'border-width': '',
    'border-style': ''
  };
});

<!Doctype html>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="assets/css/app.css">
    <link href="https://fonts.googleapis.com/css?family=Niconne|Quicksand:400,700" rel="stylesheet">
    <title>Colorit</title>

Colorit

            Shape
            <select class="shapes" name="shapes" ng-model="shape">
            <option value="">Select Shape</option>
            <option value="square">Square</option>
            <option value="circle">Circle</option>
          </select>
            Background Color
            <input type="color" name="background-color" ng-model="style['background-color']">
            Border Style
            <input type="text" name="border-style" value="solid" ng-model="style['border-style']" placeholder="solid dashed dotted inset">
            Border Width
            <input type="text" name="border-width" value="2px" ng-model="style['border-width']" placeholder="2px 4px 2px 4px">
            Border Color
            <input type="color" name="border-color" ng-model="style['border-color']" ng-init="#000" value="#000000">
            Box Shadow
            <input type="text" name="shadow" ng-model="shadow" value="5px 5px 10px" placeholder="5px 5px 10px">
            Shadow Color
            <input type="color" ng-model=color name="shadow-color" value="#000">

            {{color.style['border-color']}},{{color.style['background-color']}},{{color.style['border-width']}}

The controller is not storing values from the ng-model. Initially I had built this app using only directives, which was working, but then I had to set default values of various inputs, so I am using a controller.
Also, I am getting this error in my console
Error: [$parse:lexerr] http://errors.angularjs.org/1.6.0/$parse/lexerr?p0=Unexpected%20nextharacter%20&p1=s%200-0%20%5B%23%5D&p2=%23000
    at https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:6:425
    at pc.throwError (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:221:149)
    at pc.lex (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:218:369)
    at r.ast (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:225:175)
    at Cd.compile (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:235:100)
    at qc.parse (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:260:332)
    at https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:131:115
    at m.$eval (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:147:65)
    at Object.pre (https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:284:350)
    at https://ajax.googleapis.com/ajax/libs/angularjs/1.6.0/angular.min.js:16:71

Comment: Parse error is from using ng-init improperly. It doesn't assign anything and you shouldn't use `ng-nit` anyway for this use case
Comment: your inputs aren't within the controller scope
Comment: The error message is unrelated to your ng-model issue. The error is happening because you are saying `ng-init="#000"`, which is invalid syntax.
Comment: @charlietfl AJ Richardson, thank you for providing that info. It's going to help a lot in the future.
Answer: Move ng-controller="ColorCtrl as color" to the body tag and use controllerAs reference color in your bindings:
ng-model="color.style['border-width']

etc. for all ngModels.