Anomaly detection based on clustering

Question: I understand that there a lot of different methods for anomaly detection, based on classification, clustering, nearest neighbors, statistical, etc.
I'm trying out clustering based approach. So, I'm clustering data and as a result getting some representatives (centroids, medoids) and each cluster has some kind of total (or if you would like, average) distance.
This cluster representatives form a model. My question is, what to do next when you have a model? What would be a good anomaly metric?
I have some ideas, such as comparing the distance of object being questioned to existing representatives and than comparing that distance with the average distance in the closest cluster. But, I could also use some multiple of that distance. If you consider K-means I basically have a variance for a cluster, so I could extend that to standard deviation and us 3*sigma, as a known value for finding objects that are "not usual". But is that a good approach?
I have mentioned my reasoning about K-means, but what happens when you have a medoid? You don't have variance anymore, but you do work with some similarity measure. What to use than?
So to concertize my question - how to detect anomalies after performing clustering which produces some representatives such as centroids or medoids?
Answer: ELKI includes a class called KMeansOutlierDetection (and many more).
But of all the methods that I have tried, this one worked worst:

Even on easy, artificial data it doesn't work too well, except for the trivial objects (that literally any method will detect).
The problems with cluster-based outlier detection is that you need a really really good clustering result for this to work. On this data set, k-means does not work too well (the colors are not k-means clusters).
Here, k-means did not work too well, and thus you have false outliers along the bad cuts that k-means did:

Even worse, k-means is sensitive to outliers. So when you have lots of outliers, it tends to produce really bad results. You will want to first remove outliers, then run k-means; not the other way round!
You will end up having lots of outliers at the borders between clusters. But if the clusters are not good, this may well be in the very middle of the data!
Comment: Does ELKI describe its algorithms of outlier detection (in detail sufficient for somebody to program it)?
Comment: It's open-source, and there usually is a reference attached to each algorithm in their [JavaDoc](http://elki.dbs.ifi.lmu.de/releases/release0.6.5~20141030/doc/de/lmu/ifi/dbs/elki/algorithm/outlier/clustering/KMeansOutlierDetection.html). But this particular one says "The scores are assigned by the objects distance to the nearest center. We don't have a clear reference for this approach"; so no reference here, only source code. :-(

Anomaly detection based on clustering


Limits in multivariable function

Question: $$\lim \limits_{(x, y) \to (0,0)} {x^3 + \sin(x^2+y^2)\over{y^4 + \sin(x^2+y^2)}}$$
I don't visualize a limited function anywhere to evaluate this limit (by the way, I have the information that this limit does exist).
Comment: Convert to polar coordinates
Comment: hint $\sin(z) = z + \mathcal O(|z|^3)$.
Answer: If this limit really exists, then so should the limit of each variable, one at a time. This is both a necessary and sufficient condition for the existence of the overall $\mathbb{R}^2$ limit.
Look at $\lim \limits_{y \to 0} \lim \limits_{x \to 0} \frac{x^3+\sin(x^2+y^2)}{y^4+\sin(x^2+y^2)} = \lim \limits_{y \to 0} \frac{\sin(y^2)}{y^4 + \sin(y^2)}$. Now using the variable transformation $p=y^2$, this becomes $\lim \limits_{p \to 0^+} \frac{\sin p}{p^2 + \sin p} = \lim \limits_{p \to 0^+} \frac{\frac{\sin p}{p}}{p + \frac{\sin p}{p}}$ and because we know $\lim \limits_{p \to 0^+} \frac{\sin p}{p} = 1$, the limit expression simplifies to $\frac{1}{0 + 1} = 1$.
On the other hand, $\lim \limits_{x \to 0} \lim \limits_{y \to 0} \frac{x^3+\sin(x^2+y^2)}{y^4+\sin(x^2+y^2)} = \lim \limits_{x \to 0} \frac{x^3 + \sin(x^2)}{\sin(x^2)} = \lim \limits_{x \to 0} x\frac{x^2}{\sin(x^2)} + 1$ and this is just $0 \times 1 + 1 = 1$.
Since both the permutations of limits exist and are equal, the limit in question exists too and is equal to the common value, i.e. $1$.
Comment: Why is the common value 0? Shouldn't it be 1?
Comment: You're right. My bad!

Limits in multivariable function


How do I get the Default Content module to work?

Question: I saw the Default Content module. I wanted to know the real purpose of this module, how do we exactly use it, and what the difference with the node_export module is.
Comment: Hi you need to rethink your question, you say you want to use it because you saw the description but then state you don't know the purpose of it. If you don't know the purpose of it then why do you want to use it?
Answer: You can use default content to give your nodes a machine name (as seen when editing a node).  Then you can use features module to export those nodes to a feature.
Comment: How is this module different from node export module?
Answer: You don't use it as such, rather it provides extra options to the Features module:

When the module is installed nodes get optional "machine name" field. Nodes with filled-out machine names automatically become exportable from the Features interface.
Comment: How is this module different from node export module?
Comment: @Hacker The difference is that with the Default Content module, you export the nodes as a feature.


Como funciona o metacaractere \t em uma regex?

Question: Tenho esta variável:
y <- c('stack overflow', 'user number  2', 'nova\nlinha', 'nova \n linha')

E estas funções com os respectivos resultados:
library(tidyverse)

Com \n:
str_detect(string = y, regex(pattern = '\\n'))

[1] FALSE FALSE  TRUE  TRUE

Com \s:
str_detect(string = y, regex(pattern = '\\s'))
[1] TRUE TRUE TRUE TRUE

Nas strings 'nova\nlinha', 'nova \n linha', enquanto que na primeira não existe espaços mas na segunda sim, o retorno da função é TRUE para ambos os casos.
Tentei usar o \t, como consta nesta pergunta:
str_detect(string = y, regex(pattern = '[ \\t]'))
[1]  TRUE  TRUE FALSE  TRUE

Funcionou corretamente.
Bom, aí fiquei com algumas dúvidas. Na documentação de regex, \t opera de outra forma. Ele procura por um tab na string. Tenho duas perguntas frente a isso:

qual a diferença de tab para espaçamento e em que situações tabs são mais comuns que espaços?
por que a última função que escrevi funcionou? Eu a usei, mas não entendi sua lógica (esta aqui: str_detect(string = y, regex(pattern = '[ \\t]'))).

NOTA: Uso R em Linux e é preciso usar a dupla barra (\\) para operar ao invés de uma (\). Assim, por exemplo, ao invés do convencional \s deve-se usar o \\s.
Comment: Eu respondi sobre a regex, mas quanto ao uso de TAB x espaço, acho que é um assunto amplo demais que foge ao escopo da regex: https://www.google.com/search?q=tab+vs+space
Comment: [Leitura recomendada sobre "tabs vs espaços".](https://softwareengineering.stackexchange.com/questions/57/tabs-versus-spaces-what-is-the-proper-indentation-character-for-everything-in-e)
Answer: Sim, \t procura por um TAB. O último caso funciona porque os colchetes formam uma classe de caracteres, e a regex encontra um match caso a string tenha qualquer caractere pertencente à classe.
No caso, [ \\t] é uma classe que contém um espaço e um TAB (\t) - repare que há um espaço depois do [ - então se a string tiver qualquer um desses caracteres, já serve (ela não precisa ter todos, basta um deles existir na string que a regex já encontra um match). Nenhuma das strings possuem um TAB, mas a primeira, segunda e quarta possuem um espaço, por isso o resultado é TRUE TRUE FALSE TRUE.
Tanto que se você retirar o espaço dos colchetes:
str_detect(string = y, regex(pattern = '[\\t]'))

Dará FALSE para todas as strings, pois agora a classe de caracteres só tem o TAB (não tem mais o espaço), e nenhuma das strings possui um TAB (mas se uma das strings fosse por exemplo 'com\tTAB', aí daria TRUE).
Se bem que nesse caso, a expressão poderia ser apenas '\\t', já que não há ganho nenhum em usar uma classe de caracteres que possui apenas um caractere.

Já o \s é um atalho que corresponde a espaço, TAB ou quebras de linha (podendo variar seu significado conforme a linguagem ou engine utilizada). Por isso ele detecta o \n na terceira string, mesmo ela não tendo nenhum espaço ou TAB.


Deformation Retract of Open Square With 3 Points Removed

Question: Consider the open unit square with three points removed (say $a, b, c$) $S=(0,1)\times(0,1)-\left\{a, b, c\right\}$. Is a bouquet of three circles (wedge sum of three circles) a deformation retract of $S$? If so, why?
Comment: Yes, it is. You can also generalise to $n$ points deleted.
Answer: Pick any of the points remaining in the square. Draw loops starting at that point, encircling a single one of the removed points, without self intersections, and without intersecting the other two loops.
Since the interior of the loops have a removed point each, then they can be retracted to the points of the loop. The exterior of the concatenation of all the loops can be retracted to the points of the loops as well.
Comment: @Nina Yes, except that you need to make a distinction. If some of the points removed belong to the boundary, then you can't loop around them. You would loop around the removed points in the interior only. Then the closed square with the points removed deforms to a a wedge of loops with one loop for each point removed **from the interior** of the square.
Comment: Does this logic work for the closed square with three points removed as well?
Comment: I see, so the total number of circles in our bouquet would directly correspond to the number of points in the interior specifically. Thank you very much!
Comment: @Nina That's right. That's probably why your original problem is a claim about the open square, such that there are no ambiguities or multiple cases in the solution.
Comment: Right, that makes sense, thanks again!
Comment: As I think about this again I don't see why we would get a wedge of 3 circles for a closed square with 2 removed interior points, I can only see this becoming a wedge of 2 circles.
Comment: @Nina If you remove two interior points from a closed square, you will be able to retract to a wedge of two circles, yes. If you removed three points from the interior, then three circles. Now, if you remove two points from the interior, and one from the boundary, then you will be able to retract to a wedge of *two* circles again. The removed point from the boundary would not add a third circle to the count.
Comment: That makes sense now, thank you! :)
Answer: Take the square with one point removed. You can certainly radiallyretract to the boundary  to just get $S^1$.
For $2$ points removed, put one at $(1/3,1/2)$ and $(2/3,1/2)$ (which I'll include just for concreteness.
Using the vertical line at $x=1/2$, you can radially retract each "half of the square" onto the boundary of the "half square."
You can do this for $n$ points removed, by spacing them evenly and considering "$1/3$ squares" and so forth.
Comment: Why does this becomes a wedge of three circles? I can only see this becoming a wedge of two circles.
Comment: It does, that was two points removed. Do the same thing for three
Comment: So a closed square with two points removed doesn't deformation rectract to a wedge of 3 circles?
Comment: No definitely not.
Comment: What about a torus with two points removed?
Comment: Different story. Can u do it for one point removed? There are many answers on this site discussing that
Comment: Yeah I understand that one, it's with two points I don't.
Comment: See #5 here https://www.google.com/amp/www.math3ma.com/mathema/2016/4/18/clever-homotopy-equivalences%3fformat=amp
Comment: I see now! Thank you :)


How to synchronize different debian server

Question: I'm developing a project that consists in a tomcat installation with a web app with many classes and libraries and a php side with many many folder and files.
I've got two server debian wheeze and every time i need to do a change, i necessary need to copy from ftp the edited files from local to each two servers.
If edits has been made on java classes, i need to copy .class of each edited files and restart the two tomcat instances of each server.
Is there a way to synchronize in a simple way two folders from a server to another?
Thanks!
Answer: There are many different ways to do this. What's best depend on your situation.

rsync etc.
Networked file system like NFS to store the data
Deployment scripts tied into your VCS (or standalone)
...

My first look would be into hooks for your VCS - if done right, a simple git push can make all this happen.
Answer: I've read about it, but didn't use it yet: Lsyncd Maybe this can help you.


Como abrir uma página em nova aba sem sair da página atual

Question: O que quero é algo que funcione como o target="_blank", mas não quero sair da página onde o link foi clicado.
Ou seja, ao clicar no link, aparentemente nada mudaria, mantendo a visualização da página atual, no entanto, uma nova aba seria aberta.
Qual código poderia utilizar para fazer isso?
Answer: Nova aba x Nova janela
Primeiro, só para esclarecer: sobre abrir numa "nova aba". Caso, você queira que abra somente numa "nova aba" e não numa "nova janela", isso não é atualmente possível. O CSS3 define a propriedade target-new para lidar com isso, mas ela não foi implementada em nenhum navegador até agora.
Ok, agora sobre o tema principal:
Abrir link em segundo plano
Você pode fazer um workaround emulando Ctrl+Click:
Link para esta questão
<button onclick="simulateClick(true, false, false);">Simulate Ctrl+Click</button>

var simulateClick = function (ctrl, shift, isMiddle) {
    var evt = document.createEvent('MouseEvents');
    evt.initMouseEvent('click', true, true, window, 0, 0, 0, 0, 0, ctrl, false, shift, false, isMiddle ? 1 : 0, null );
    document.getElementById('link').dispatchEvent(evt);
}

Ver um jsFiddle semelhante
Porém,
Esse é um comportamento do navegador, configurado pelo usuário. E deveria ser assim e somente assim. Com razão, não queremos abrir possibilidade abrir links em segundo plano sem notarmos enquanto navegamos pela web! Isso é uma opção do usuário!
Mais do que isso, até o target="blank" só encontra respaldo em certas ocasiões: quando o usuário tem algo em andamento que não deve ser perdido com um redirecionamento local de página. Um link aberto enquanto está tocando um vídeo, ou um rascunho não salvo de um e-mail num cliente de emails web.
A ideia de tudo isso é que não devemos alterar o comportamento natural do navegador. Se o usuário quiser abrir o link na própria janela, deixe. Se ele quiser abrir em outra, deixe. Se ele quiser abrir numa aba em segundo plano, deixe. Mas se ele quiser abrir em primeiro, também deixe.
O foco da aplicação web sobre isso deve ser em fornecer a melhor experiência para usuário através de um bom layout e um bom fluxo, sem limitar o usuário a escolhas que ele pode em outra aplicação. Essa abordagem pode até irritar o usuário.
Answer: Vai precisar de uma código como este:
function openNewBackgroundTab(){
    var a = document.createElement("a");
    a.href = window.location.pathname;
    var evt = document.createEvent("MouseEvents");
    //the tenth parameter of initMouseEvent sets ctrl key
    evt.initMouseEvent("click", true, true, window, 0, 0, 0, 0, 0,
                                true, false, false, false, 0, null);
    a.dispatchEvent(evt);
}

var is_chrome = navigator.userAgent.toLowerCase().indexOf('chrome') > -1;
if(!is_chrome)
{
    var url = window.location.pathname;
    var win = window.open(url, '_blank');
}
else
{
    openNewBackgroundTab();
}

Fonte: https://stackoverflow.com/a/22858261/194717


Why was the thinline stratocaster never really a thing?

Question: I know Eric Johnson has a signature thinline strat but why was the thinline strat never really a thing like the thinline teles were / are?
Why would Fender make a semi-hollow version of the teles for decades but not for their other flagship guitar model?
Answer: isnt the stratocaster rather more heavily chamfered* making a chambered version technically quite  difficult to make ?
A tele can easily be made semi-hollow by building the body from two pieces: the main part of the body and then a cap. That's because the sides are mostly squared off. The contoured edges of the strat body make that a lot harder.
*chamfered refers to the removal of some wood, mainly on the outside cormer where the right arm swings, and also on the back, making it a more snug fit against the chest, or for some, to leave room for an overlarge belly.


Calculating expected value of random walk with one stop value.

Question: I know that for a random walk with two stop values, the expected value of the number of steps needed is $ab$ where the stop values are $-a$ and $b$ and the initial position is at 0.
What about for random walks with probability $1$ of reaching the stop value? For example, I am  at position $1$ and I have probability $0.6$ of going left and $0.4$ of going right. I obviously have probability $1$ of reaching $0$, but what is the expected value of number of steps?
Simulations seem to say $E[V] = 5$ which is equal to $\sum_{i=0}^{\infty} C_i * 0.4^i * 0.6^{i+1}$ where $C_i$ is the $i$th Catalan number. However this is equal to  of $\sum_{i=0}^{\infty} 0.6^{i+1}*0.4^i*\binom{2i+1}{i}$ which doesn't make intuitive sense. I would have reasoned it to be $\sum_{i=0}^{\infty} 0.6^{i+1}*0.4^i*\binom{2i}{i} = 3$ since the last head is set (the last coin tossed has to be a head). The first equation seems to double count?
Thanks! (Also is there a general formula or approach for random walks with one stopping value?)
Answer: $E[V] = \displaystyle\sum_{i=0}^{\infty} (2i+1) C_i \, 0.4^i \, 0.6^{i+1}$ with an extra $(2i+1)$ term compared with your first expression.
Since $C_i= {2i \choose i}/(i+1) ={2i+1 \choose i}/(2i+1)$ you get your second correct expression when the two $(2i+1)$ terms cancel.
Your third expression is missing a $\frac{(2i+1)}{(i+1)}$ term so intuitively it is not a surprise that it is just over half the correct answer.
For a possible general approach, first check that you do in fact stop with probability $1$.  Then see how many ways there are of being one step away from stopping without having previously stopped and what the probability of each of them are multiplied by the probability of stopping on the next step.
Answer: Let $N$ denote the expected number of steps before hitting $0$ when starting at position $1$ and having probability $p\gt\frac12$ of going left and $1-p$ of going right.
The (simple) Markov property after one step indicates that $N=p\cdot1+(1-p)\cdot(1+M)$ where $M$ denotes the expected number of steps before hitting $0$ when starting at position $2$. Since this requires to hit $1$ starting from $2$ then to hit $0$ starting from $1$ and since both phases require the same mean number of steps $N$, one has $M=2N$.
Solving this for $N$ yields $N=1/(2p-1)$.
The full distribution of $N$ stems from the same decomposition applied to the generating function $E[s^N]$, for every $s$ in $[0,1]$, which yields the system $E[s^N]=p\cdot s+(1-p)\cdot s\cdot E[s^M]$ and $E[s^M]=E[s^N]^2$, implying that
$$
E[s^N]=\frac{1-\sqrt{1-4p(1-p)s^2}}{2(1-p)s},
$$
from which $P[N=2n+1]$ can be deduced for each $n\geqslant0$ by a series expansion of the square root in the numerator.


Policy on posting surveys to the site?

Question: Im looking to collect data for a research project. can I post a survey to the main site or meta?
Answer: I would say no. If allowed, survey questions would probably outnumber other types of questions.
Answer: As denesp said, in general no. But if you did want to enquire further, then send an email to email@example.com and ask the staff.

Policy on posting surveys to the site?


Simple Sphere Suspension Question

Question: I have heard it said that the suspension of an $n$-sphere is an $n+1$-sphere. This is stated without proof in chapter 0 of Hatcher's book on algebraic topology. More generally, it seems that the suspension of a simplex or cube is a simplex or cube (respectively) of one higher dimension. I can convince myself of this when the objects are of low enough dimension to visualize, but I can not see a way to prove this rigorously. How does one prove it?
Comment: Think carefully about the case of a circle and a sphere. It will become clear to you.
Comment: @ncmathsadist I think that maybe we can identify $S^{n}$ with the equator of $S^{n+1}$ and each end of the suspension with the north and south pole of $S^{n+1}$ and radially map to $S^{n+1}$, but I don't see how this works for the cube or simplex case.
Comment: @ncmathsadist Haha, they're all homeomorphic. Not on the ball today.
Answer: The following is the way that I've found most natural, and easiest to think about.
First prove the following lemma (which is a good reason to think about smash products, for example!)
Lemma. Let $V$ and $W$ be vector spaces. Then there is a natural homeomorphism $S^V \wedge S^W \cong S^{V\oplus W}$, where $S^V$ denotes the one-point compactification of the vector space.
The result follows trivially, then, as soon as you prove
Lemma. The suspension, $\Sigma X$, of a pointed space is the same as $S^1 \wedge X$.
Happy learning!
Comment: (Hint: To prove the first lemma, use the universal property of one-point compactifications!)
Comment: Thanks. This is very helpful.

Simple Sphere Suspension Question


gravar hora no banco Postgresql

Question: Na aplicação eu informo a hora de inicio e fim da atividade, e guardo essa informação no banco.
Segue Query:
$status = $_GET['status'];
$intervalo = $_GET['intervalo'];
$dataini = $_GET['horainicio']."
";
$horainicio = date('H:i:s',$dataini);
$datafim = $_GET['horafinal']."
";
$horafinal = date('H:i:s',$datafim);
$qtdtentativas = $_GET['qtdtentativas'];
$destino = $_GET['destino'];
$skill = $_GET['skill'];
$pacing = $_GET['pacing'];
$prefixo = $_GET['prefixo'];

$query = sprintf("update bettaivr.public.ivr_campanha set status = %d,horainicio = %d,horafinal = %d, intervalo = %d, pacing = %d, qtdtentativas = %d  where id = %s",
$status,$horainicio ,$horafinal ,$intervalo, $pacing, $qtdtentativas,$campanha);

Entretanto não consigo guardar os horarios no banco, ele retorna 0 sempre.
Como posso guardar esse horario no banco?
Erro sinalizado ao executar a query:
Problem with query update bettaivr.public.ivr_campanha set status = 0,horainicio = '0',horafinal = '0', intervalo = 20, pacing = 1, qtdtentativas = 5 where id = 4
Comment: esta como time no banco.
Comment: Qual o tipo de dado que esta no campo de horas do banco de dados?
Comment: os campos de hora estão entre apostrofos??
Comment: no banco não. na aplicação estão desse jeito como demonstrado, adicionei a resposta que vem do banco
Answer: Seu problema está aqui:
$horafinal = date('H:i:s',$datafim);
$horainicio = date('H:i:s',$dataini);

tenta assim:
$horafinal = date('H:i:s',strtotime($datafim));
$horainicio = date('H:i:s',strtotime($dataini));

E usa a query dessa forma:
$query = "
    UPDATE
        bettaivr.public.ivr_campanha
    SET
        status = '$status',
        horainicio = '$horainicio',
        horafinal = '$horafinal',
        intervalo = '$intervalo',
        pacing = '$pacing',
        qtdtentativas = '$qtdtentativas'
    WHERE id = '$campanha'
";

Algumas referências da documentação sobre date e strtotime:

strtotime — Interpreta qualquer descrição de data/hora em texto em inglês em timestamp Unix
date — Formata a data e a hora local. Retorna uma string de acordo com a string de formato informada usando o inteiro timestamp informado
Comment: o erro que ele deu, foi que o campo pacing não deve receber a hora, por algum  motivo quando eu faço o get ele pega os dados do dataini, mas ta dando certo.
Comment: Ainda não resolveu, o problema persistiu.
ele devolveu este erro
ERROR: column "horainicio" is of type time without time zone but expression is of type integer LINE 1: ...r.public.ivr_campanha set status = 0,horainicio = 0,horafina... ^ HINT: You will need to rewrite or cast the expression.
Comment: quando eu escrevo para o banco:
sprintf("update bettaivr.public.ivr_campanha set status = %d, horainicio = %i ,horafinal = %i, intervalo = %d, pacing = %d, qtdtentativas = %d  where id = %s",
  $status,$horainicio ,$horafinal ,$intervalo, $pacing, $qtdtentativas,$campanha);
acho que o formato da escrita pode estar errado, mas não acho o certo
Comment: Editei a resposta, tenta ai pra ver, usando o que eu falei tbm sobre as variaveis de hora, já que o erro acusa que o valor esta sendo passado em tipo int
Comment: Show cara, ja começou a dar outra cara para a query, agora o erro é de variavel que recebe..
ERROR: invalid input syntax for integer: "08:00:00" LINE 9: pacing = '08:00:00', ^

vou olhar como esta recebendo esses valores
Comment: Ele esta indicando erro de integer, falando que o tipo é int, verifica isso

gravar hora no banco Postgresql


Consultar últimos registros por hora

Question: Estou com dúvidas em extrair dados da última hora do banco mysql, o código abaixo faz a soma de valores na tabela pelo último dia, mais relacionar pela última hora não estou conseguindo fazer.
   $busca = mysql_connect("$local","$usuario","$senha") or die("ERRO AO CONECTAR AO MYSQL, VERIFIQUE COM O ADMINISTRADOR" . mysql_error());
            mysql_select_db("$banco") or die("BASE DE DADOS INVÁLIDO");
            $pesquisa = mysql_query("SELECT sum(ProdValor) FROM vendas WHERE data BETWEEN CURRENT_DATE()-30 AND CURRENT_DATE()");
            while($sum = mysql_fetch_array($pesquisa)){
            $soma5 = $sum['sum(ProdValor)'];
            }
            //Mostrando o Resultado

            //$resultado = number_format($soma,2,",",".");

            echo $soma5;

Comment: @Sergio atualizei para: `mysql_query("SELECT sum(ProdValor) FROM vendas WHERE data >= DATE_SUB(NOW(), INTERVAL 1 HOUR)");` e não é demostrado nenhum valor, mesmo havendo registros no banco de dados a 1 hora atrás.
Comment: Qual é o formato do campo `data`?
Comment: @Sergio está no formato date.
Comment: Experimenta com `WHERE data >= DATE_SUB(NOW(), INTERVAL 1 HOUR)`
Answer: Para mim o que Sergio falou deu certo:
WHERE data >= DATE_SUB(NOW(), INTERVAL 1 HOUR)

Talvez seja porque na estrutura do campo na sua tabela você tenha colocado somente DATE, coloque DATETIME igual tá aqui pois comigo funcionou.
Answer: Existe essa forma também:
$data_hora_intervalo =
    date('Y-m-d H:i:s', strtotime('-1 hour', strtotime(date('Y-m-d H:i:s'))));

where data>='$data_hora_intervalo'

Consultar últimos registros por hora


Mongoose Sorting nested array by field

Question: I have a GameStatistics collection and I would like to sort all the players DESC by their points.
All the players are listed in a nested activePlayers array with a game. Each player has a field points (I have put the code and the Schema down below).
It seems that I am missing something, I've already tried with aggregate() but no luck either.
sort = async (req: Request, res: Response, next: NextFunction) => {
  const id = "0021900805";
  const stats = await GameStats.findById(id)
    .sort({ "vTeam.activePlayers.points": -1 })
    .exec();
  return res.status(200).json(stats);
};

The collection looks like this:
[
  {
    "vTeam": {
      "activePlayers": [
        {
          "teamId": "1610612759",
          "firstName": "Lonnie",
          "lastName": "Walker IV",
          "points": "23",
          "assists": "0",
          "rebounds": "4",
          "fgp": "40.0",
          "to": "0",
          "stl": "3",
          "blk": "0"
        },
        {
          "teamId": "1610612759",
          "firstName": "LaMarcus",
          "lastName": "Aldridge",
          "points": "4",
          "assists": "3",
          "rebounds": "14",
          "fgp": "45.0",
          "to": "2",
          "stl": "1",
          "blk": "1"
        }
      ]
    },
    "_id": "0021900805",
    "createdAt": "2020-04-05T15:34:26.457Z",
    "vTeamScore": "114"
  }
]

Answer: You can use mongodb aggregation framework to solve this problem.
Playground
sort = async (req: Request, res: Response, next: NextFunction) => {
  const id = "0021900805";
  const stats = await GameStats.aggregate([
    {
      $match: {
        _id: id, // or mongoose.Types.ObjectId(id)
      },
    },
    {
      $unwind: "$vTeam.activePlayers",
    },
    {
      $sort: {
        "vTeam.activePlayers.points": -1,
      },
    },
    {
      $group: {
        _id: "$_id",
        activePlayers: {
          $push: "$vTeam.activePlayers",
        },
        doc: {
          $first: "$$ROOT",
        },
      },
    },
    {
      $replaceRoot: {
        newRoot: {
          $mergeObjects: [
            "$doc",
            {
              vTeamActivePlayers: "$activePlayers",
            },
          ],
        },
      },
    },
    {
      $addFields: {
        "vTeam.activePlayers": "$vTeamActivePlayers",
      },
    },
    {
      $project: {
        vTeamActivePlayers: 0,
      },
    },
  ]);

  return res.status(200).json(stats);
};

Answer: According to mongoose's docs your solution would be to replace 1 with -1
So: const stats = await GameStats.findById(id).sort({'vTeam.activePlayers.points': -1}).exec();
Comment: Yes, I know. The actual array is much bigger. I changed it up to see if would the sort() would sort anything at all (wich it doesn't). But I forgot to change it back

Mongoose Sorting nested array by field


I getting "invalid session id"

Question: My issue is for the first time I am implementing data driven framework and I am able to run the program and it is getting executed and I am able to take the screen shot as well.But the issue is I am getting below mentioned error in my console which will of course reflects in my report. Please suggest what should I do and in case of any other information, please ask the same.
Console Error : -
<code>org.openqa.selenium.NoSuchSessionException:  invalid session id
  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.17134 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
</code>
I am using Test NG also to get the results.
Regards,
Sunil
Comment: Update the question with your code trials and error trace log
Comment: dude please share script that you are try to run
Comment: Please find the below script-:public void initialization(){
      // To set the path of the Chrome driver.
  System.setProperty("webdriver.chrome.driver", "./ChromeDriver/chromedriver.exe");
  driver = new ChromeDriver();

  driver.manage().window().maximize();

      // To launch facebook
      driver.get("https://www.mahindrafirstchoice.com/sell");
Comment: Then I tried giving EXPLICIT WAIT so that it will open my website and will select the city -:: WebDriverWait wait = new WebDriverWait(driver,10);

      wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@class='modal show']/div/div/div/ul/li[3]/a")));

      //TO SELECT CITY

      driver.findElement(By.xpath("//div[@class='modal show']/div/div/div/ul/li[3]/a")).click();

      }
Comment: Hi Debanjan, please find my console error and session ID --:   invalid session id
  (Driver info: chromedriver=2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a),platform=Windows NT 10.0.17134 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: '3.13.0', revision: '2f0d292', time: '2018-06-25T15:32:14.902Z'  System info: host: 'DESKTOP-MO0FC5C', ip: '192.168.3.148', os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_191'
Comment: Session ID: 112ddd472d778af7469463947d16f153
Comment: When you add information, please edit the info into your question so that future readers don't have to read all comments to get all the relevant data.
Comment: ok, but is there anyone who can help me with the answer, coz m completely stuck and not able to proceed further. As I am new to this website, I am not aware which informations should i fill.
Answer: Check your chrome browser, it might got updated.
The chrome driver which you are using is not supporting the updated chrome browser
Check the version of the chrome browser and update your driver accordingly.
then execute the script.

I getting "invalid session id"


Why for a real number $x$ and an even integer $n, x^{1/n}$ is not real if $x<0$?

Question: Why for a real number $x$ and an even integer $n$, $\;x^{1/n}$ is not real if $x<0$?
What I'm really asking is how do you define a complex number without referring to imaginary numbers.
How do you define a complex number without circular logic that involves a complex number
($a+bi$ where $i$ is the imaginary unit, and $a, b$ are real).
Explain without assuming I'm defining functions.
Comment: Is $x\ge 0$? (Otherwise what happens if $x=-1$ and $n=2$ for example?)
Comment: $x^{\frac{1}{n}} = \sqrt[n]{x}$, if $x$ is negative and $n$ is even, you cannot have a real negative square root
Comment: $i$ is the imaginary unit; it's like saying to define $10$ without using any real numbers. However, there are some specific ways to define some complex numbers, e.g. the roots of unity, but almost all complex numbers must be defined using the imaginary unit.
Comment: "how do you define a complex number without referring to imaginary numbers?" the answer is that you need to refer to imaginary numbers to define complex numbers. Why is $x^{\frac{1}{n}}$ not real under those conditions? and without referring to imaginary numbers? suppose it is real, then you get a contradiction, this suggest that there are operation in the reals that don't give you back a real number, and you need a bigger family of numbers. It's like asking: what's $3\div 2$ in the integers?, we don't know, that's why we need rationals etc.
Comment: Are you familiar with the theorem saying every real polynomial can be factored as a product of linear polynomials and quadratic polynomials with negative discriminant?
Answer: Let $x<0$,$n=2k$
Assume $x^{\frac{1}{2k}}$ to be a real number
What this means that there exist a Real number $p=x^{\frac{1}{2k}}$ such that $p^{2k}=x$
But $p^{2k}=(p^k)^2$, and The square of any real number is always non-negative, Hence $(p^k)^2$ is also non-negative, which in fact means $p^{2k}$ is non-negative
But $p^{2k}=x$, Which follows x must be non-negative, Which contradicts the assumption that There exist a real number $x^{\frac{1}{2k}}$,Such that $x<0$
Hence our assumption is wrong, There does not exist a a real number $x^{\frac{1}{2k}}$,Such that $x<0$
Now $p$ is not real
That means there is a whole set of such cases where all such $p's$ are not real
Now Imaginary numbers were named as such numbers, Whose $2^{nd}$ or $4^{th}$ or even powers would lead to a negative real number, Historically, They were first
conceived by Italian Mathematician Gerolamo Cardano, While solving a general cubic equation
Roots such as $a+b\sqrt-1$ Were expressed, Now, These itself had no real meaning in those times, But when these roots were put into the polynomial, The result would give out "$0$",Meaning they had mathematical usefulness, Such complete expressions were later named complex numbers, But the imaginary parts were indeed already used before, For a short history lesson, This might be useful, Note that $i$,whose notation as $\sqrt-1$ was given by Rafael Bombelli, always was used historically for its mathematical usefulness as a number whose square is $-1$
It was later in time that geometrical interpretations were given to such expressions and later called complex numbers as $a+ib$, Note $i$ as complex number $0+1i$ was later seen as "complex", after these definitions were rigorously defined, Hence, historically complex numbers were consequence of Applying mathematical operations on imaginary and Real numbers
Answer: "How do you define a complex number without circular logic that involves a complex number is $a+bi$ where $i$ is a complex number, $a$ real $b$ real."
First, you write ${\bf R}[x]$ for the ring of all polynomials with real coefficients.
Then you let $J$ be the ideal of all multiples of the polynomial $x^2+1$.
Given a polynomial $p(x)$ in ${\bf R}[x]$, you define the coset of $J$ containing $p(x)$ to be the set of all polynomials of the form $p(x)+j(x)$ where $j(x)$ runs through $J$, that is, it's the set of all polynomials that are $p(x)$ plus a multiple of $x^2+1$. We'll use the notation $\overline{p(x)}$ for this coset.
Notice that any polynomial $p(x)$, when you divide by $x^2+1$, gives you a quotient and a remainder, and the remainder is a polynomial of degree one, $ax+b$ for some real numbers $a,b$. You have
$$
p(x)=(x^2+1)q(x)+ax+b
$$
($q(x)$ is the quotient when you do the division). Notice that in this situation, $\overline{p(x)}=\overline{ax+b}$; since $p(x)$ and $ax+b$ differ by a multiple of $x^2+1$, they define the same coset of $J$.
So, the set $\bf C$ of all cosets of $J$ is the set of all $\overline{ax+b}$ with $a,b$ real.
Well, $\bf C$ is the complex numbers, defined here with no circularity. The element $\overline x$ satisfies $\overline{x}^2=\overline{-1}$ because $\overline{x^2+1}=\overline0$. So $\overline x$ plays the rôle of $i$.


Find a non-principal ideal in $ \Bbb Z [2i]$.

Question:
Find a non-principal ideal in $ \Bbb Z [2i]$.

I think it might be $(1+2i,1-2i)$, but have problems with proving this.
I know that $|1+2i|=|1-2i|=5$.
Moreover, there are only 6 elements with non-bigger norm than 5 (except these two),
I mean $1,-1,2,-2,2i,-2i$. None of these has norm equal to 5, so if this ring was principal, then $1+2i = a(1-2i)$, where $|a|=1$. Of course this is not possible, so our ideal is not principal.
Is this correct?
Comment: It might be, but it isn't: $(1+2i)(1-2i)-2[(1+2i)+(1-2i)]=?$
Answer: Hint. $\mathbb Z[2i]\simeq\mathbb Z[X]/(X^2+4)$. Now recall the usual example of a non-principal ideal in $\mathbb Z[X]$.

$(2,2i)$ is a non-principal ideal in $\mathbb Z[2i]$.


Logstash filtering unstructured Logs

Question: I'm pretty new to Logstash and try my best to get a grip on it. My Problem is the following:
I have a Log which is structured like this:
time - data - message
every 10 to 15 log entrys are linked to one Job which has a ID but only the first entry of the job is marked with the ID.
I now want to try to add a field to every Entry that follows in order in identify all entrys related to a specific ID.

Here an Example
29.09.15 16:17:00:191 - text text ID: 00000001 text text
29.09.15 16:17:00:206 - text text
29.09.15 16:17:00:253 - text text
...
29.09.15 16:17:26:539 - text text ID: 00000002 text text
29.09.15 16:17:30:233 - text text

I'm a bit lost. My current approach would be grok filtering to identify the ID but I'm clueless how to remember the ID and add it to the following entrys when I read them...
Answer: I am too late for a resuce but if am not wrong, you should use multiline filter plugin of Logstash.
There you can mark the logline with ID as a start of an event. So whenever logstash finds a line with ID, it takes all the following lines and stash it as one full event till it finds another log line with ID again.
Read this below doc:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-multiline.html

Logstash filtering unstructured Logs


Confusion about the uniqueness of an SDE

Question: Suppose we are given a SDE of the form
$$
dX_t=\sigma(t,X_t)dW_t + b(t,X_t)dt,\quad X_0=\xi.
$$
For simplicity, let $\xi$ be a constant.
My understanding is that

Strong solution: for a given probability space, filtration, and a Brownian motion (BM), if there exists $X$ that solves the above SDE, then $X$ is a strong solution.
Weak solution: if we can find a probability space, filtration, a BM, and $X$, then we say that the a tuple of these is a weak solution.

For the uniqueness, we say that

Unique in law: if the law of $X$ is unique. (E.g. a BM $B$ and $-B$ both solve the Tanaka SDE, and their laws are the same)

Pathwise uniqueness: if there are two solutions $X$ and $X'$, then $\mathbb{P}(X_t=X_t',\forall t)$.

My confusion is related to Ito's theorem and Yamada-Watanabe Theorem. Ito's theorem says that if $\sigma$ and $b$ are Lipshitz and of linear growth, then the solution exists and is unique. Yamada-Watanabe says that a weak solution + pathwise uniqueness is equivalent to  the existence of a unique strong solution.
My confusions are as follows.

In Ito's theorem, what does it mean by ``unique''? Is it pathwise uniqueness?
In Ito's theorem, what does it mean by ``solution''? Is is strong solution?
If Ito's theorem is about a strong and pathwise unique solution, then since strong solution is weak, Yamada-Watanabe seems to be moot.

I consulted Karatzas-Shereve, Rogers-Williams, and Klenke, but they use slightly different terminology and I am confused.
Answer: Indeed globally-Lipschitz regularity gives you strong-solution and  strong-uniqueness (see Karatzas-Shreve 5.1 "Strong solutions"). As explained in Karatzas-Shreve the difference between pathwise and strong-uniqueness is:

Pathwise uniqueness is said to hold  among solutions that spend zero time at 0 if whenever $(X,W), (X,\tilde{W} )$ are two weak solutions with  a common Brownian motion $W$ (relative to possibly different filtrations $\mathcal{F}_{t},\tilde{\mathcal{F}}_{t}$) on a common probability space and with common initial value, then $P(Xt = \tilde{X}_t \text{ for all }t \geq 0) = 1$.
Strong uniqueness almost the same above with the extra assumption "with respect to a common Brownian motion $W$ and with the same filtration" $\mathcal{F}_{t}$.

One theorem of Yamada-Watanabe proves that one can lift a weak-solution with pathwise-uniqueness to a strong solution.

They also prove that a strong-solution is possible with a weaker requirement on the coefficients (described in Karatzas-Shreve as you mentioned) (note that below they actually mean strong-uniqueness) eg. in  LECTURE NOTES ON THE YAMADA–WATANABE CONDITION
FOR THE PATHWISE UNIQUENESS OF SOLUTIONS OF
CERTAIN STOCHASTIC DIFFERENTIAL EQUATIONS.
For example, Revuz-Yor never uses the phrase "strong-uniqueness" but instead just talks about "pathwise-uniqueness" (which they define having the same filtration i.e. the strong uniqueness from Karatzas-Shrever).
Comment: @keepfrog other way around, if an SDE satisfies pathwise-uniqueness then it also gives strong-uniqueness because you can take $\mathcal{F}^1_{t}=\mathcal{F}^2_{t}$. Pathwise-uniqueness says if you have two solutions X1,X2 with filtrations $\mathcal{F}^1_{t},\mathcal{F}^2_{t}$, then they are in fact equal $P[X^1_t=X^2_t, \forall t\geq 0]=1$.
Comment: So Yamada-Watanabe theorem is more demanding i.e. to yield strong-existence one needs both weak-solution and pathwise-uniqueness.
Comment: also as they  mention in (5.3) remark 3.3 In Karatzas-Shreve, the Lipschitz-theorems can also be done with pathwise-uniqueness.
Comment: So, under the Karatzas-Shreve definition, strong uniqueness implies pathwise uniqueness? Is it possible that there is a pathwise unique weak solution but a strong solution, which exists by Yamada-Watanabe, is not strong unique?


Arpeggio sequences

Question: For my entire learning career as a violinist (25 years) I've been given arpeggios to practice with similar characteristics, always in a specific order.
1) there's the regular major (1 3 5 8 5 3 1).
2) Then there's 1 3 6 8 6 3 1.
3) Then 1 4 6 8 6 4 1.
4) Then diminished (1 min3 dim5 dim7 8 dim7 dim5 min3 1)
5) and then dominant (1 3 5 7 8 7 5 3 new1).
Does anyone know what the second and third arpeggios are named? And / or why string players (and maybe other instrumentalists) ended up practicing them in this order?
Answer: #2 is a minor triad in first inversion (it has the 3rd as the lowest note).  #3 is a major triad in second inversion (the 5th is lowest).
Doing them in this order forms a simple chord progression:  I vi IV vii°/V V.
Comment: Oh cool! So 2) is f# minor first inversion, and then 3) is d major in first inversion not second, right?
Comment: @JenniYael - it depends what key you're in! Sounds like you're in A maj.
Comment: Lol! Yes I used A major as an example for myself

Arpeggio sequences


while using python pd.DatetimeIndex how can I extract two instances?

Question: dt = pd.DatetimeIndex(df['datetime'])

df['date'] = dt.date
df['day'] = dt.day
df['month'] = dt.month
df['year'] = dt.year
df['hour'] = dt.hour
df['dow'] = dt.dayofweek
df['woy'] = dt.weekofyear
df['weeks_start'] = (dt - min(dt)).days / 7

Here I'm studying bicycle borrowing demand in Kaggle. With others' solution, I want to test with another column, which is year and month.
In turn what I want to create is df['year_month'] which shows concatenated version of year and month for example 2011_1.
But just doing str(df['year']) + str(df['month']) does not qualify such job since it is typed as pandas array.
How can I successfully concatenate these two as a new column and do the one hot encoding for this?
Comment: Would `dt.strftime('%Y_%m')` or `pd.Index(dt.strftime('%Y_%m'))` work?
Answer: IIUC, let's try:
dt.year.astype(str) + '_' + dt.month.astype(str)

Since, dt is aleady a datetimeindex object, you can access the year and month directly using .year and .month then cast them as strings using astype and add them together.  Output:
Index(['2017_1', '2017_2', '2017_3', '2017_4', '2017_5', '2017_6', '2017_7',
       '2017_8', '2017_9', '2017_10', '2017_11', '2017_12', '2018_1', '2018_2',
       '2018_3', '2018_4', '2018_5', '2018_6', '2018_7', '2018_8', '2018_9',
       '2018_10', '2018_11', '2018_12', '2019_1', '2019_2', '2019_3', '2019_4',
       '2019_5', '2019_6', '2019_7', '2019_8', '2019_9', '2019_10', '2019_11',
       '2019_12', '2020_1', '2020_2', '2020_3', '2020_4', '2020_5', '2020_6',
       '2020_7', '2020_8', '2020_9', '2020_10', '2020_11', '2020_12', '2021_1',
       '2021_2', '2021_3', '2021_4', '2021_5', '2021_6', '2021_7', '2021_8',
       '2021_9', '2021_10', '2021_11', '2021_12'],
      dtype='object')

Comment: this didn't worked at all
TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('<U11') dtype('<U11') dtype('<U11')
Comment: @jh.shin Okay,you can try df['year'].astype(str) + '_' + df['month'].astype(str)


ant.d - Form - How to show warnings with standard rules API?

Question: I have a generic rules for my form item:
<Form.Item
        label="Username"
        name="username"
        rules={[{ required: true, message: 'Please input your username!' }]}
      >
        <Input />
</Form.Item>

What I would like to do is for an empty input display not red, but orange formatting, similar to  validateStatus='warning'.
In other words - when the rule clicks - show orange warning like this:

Is that possible w\o explicitly interacting with form state via useForm? (wouldn't wanna do that)
Answer: If you add warningOnly: true to your rule, the user will not be blocked from submitting and you'll get the desired behavior.
More info at https://ant.design/components/form/#Rule
Answer: It seems that you can only set validateStatus to 'warning' to show warning style.
Here's how it works to set validateStatus in source code of FormItem. When validateStatus is undefined, mergedValidateStatus can only be 'validating', 'error', 'success'
let mergedValidateStatus: ValidateStatus = '';
if (validateStatus !== undefined) {
  mergedValidateStatus = validateStatus;
} else if (meta?.validating) {
  mergedValidateStatus = 'validating';
} else if (meta?.errors?.length || subErrorList.length) {
  mergedValidateStatus = 'error';
} else if (meta?.touched) {
  mergedValidateStatus = 'success';
}

const itemClassName = {
  [`${prefixCls}-item`]: true,
  [`${prefixCls}-item-with-help`]: domErrorVisible || !!help,
  [`${className}`]: !!className,

  // Status
  [`${prefixCls}-item-has-feedback`]: mergedValidateStatus && hasFeedback,
  [`${prefixCls}-item-has-success`]: mergedValidateStatus === 'success',
  [`${prefixCls}-item-has-warning`]: mergedValidateStatus === 'warning',
  [`${prefixCls}-item-has-error`]: mergedValidateStatus === 'error',
  [`${prefixCls}-item-is-validating`]: mergedValidateStatus === 'validating',
  [`${prefixCls}-item-hidden`]: hidden,
};

https://github.com/ant-design/ant-design/blob/8a264baf67322af5e786af13248311e89a1a8b05/components/form/FormItem.tsx#L178


Alguem pode me ajudar a escolher uma linha especifica da matriz ? (linguagem c/c++)

Question: Estou fazendo uma atividade e ela diz:

(função – matriz e vetor) Elabore um programa que leia uma matriz 4x4 e crie um vetor de 4 elementos
constituído dos elementos de uma linha da matriz escolhida pelo usuário
meu problema é q não sei fazer ele printar exatamente a linha escolhida pelo usuário.

Meu programa acabou sendo só isso
<code>#include <stdio.h>
int main() {

float matriz[4][4];
int d;
/* Entrada de dados na matriz */
printf(" Informe o valor de cada elemento -----\n");
for(int i=0; i<2; i++){
 for(int j=0; j<2; j++)
 {
 printf(" \n matA(%d,%d) = ", i+1,j+1);
 scanf("%f", &matriz[i][j]);
 }
}
// escolha da linha
printf("escolha a linha desejada:");
scanf("%d",&d);

/* Escrita da matriz - saída formatada */
printf(" \n A matriz informada eh = \n");
for(int i=0; i<2; i++)
{
 printf(" \n ");
 for(int j=0; j<2; j++)
 printf(" %4.1f ", matriz[i][j]);
}
}
</code>
Comment: Já pensou em não utilizar um loop para a linha (Iíndice `i`) e no lugar informar `d`? `for(int j=0; j<2; j++)  printf(" %4.1f ", matriz[d][j]);`
Answer: A solução é bem simples, você pega linha do usuário e substitui na linha que o for percorre, assim voce vai percorrer somente pela linha que o usuário escolher.
Aqui ele está fazendo com uma matriz 2x2 que foi seu codigo de exemplo:
<code>int main() {

float matriz[4][4];
float vetor[4];
int d;
/* Entrada de dados na matriz */
printf(" Informe o valor de cada elemento -----\n");
for(int i=0; i<2; i++){
 for(int j=0; j<2; j++)
 {
 printf(" \n matA(%d,%d) = ", i+1,j+1);
 scanf("%f", &matriz[i][j]);
 }
}
// escolha da linha
printf("escolha a linha desejada:");
scanf("%d",&d);

 printf(" \n ");
 for(int j=0; j<2; j++){
 printf(" %4.1f ", matriz[d][j]);
  vetor[j]=matriz[d][j];
 }
 //imprime vetor com os numeros da linha
 printf(" \n ");
 for(int j=0; j<2; j++){
 printf(" %4.1f",  vetor[j]);}

}````
</code>


CentOS VM only using a third of available memory

Question: I have a VM slice running CentOS, on Parallels.
Here is the output of free -m
[root@server ~]# free -m
             total       used       free     shared    buffers     cached
Mem:           960        272        687          0          0          0
-/+ buffers/cache:        272        687
Swap:            0          0          0

This is a LAMP server, and the database itself is over 1000 MB.
Shouldn't the "buffers" and "cached" be showing higher values, caching the database files? It looks like only 1/3 of memory is being used.
Comment: What database? What makes you think it should be cached?
Comment: MySQL, shouldn't Linux cache files in memory, if there is free memory?
Answer: The buffers and caches displayed are from the point-of-view of the OS.
To see what MySQL is caching you must run the following queries
MYSQL_CONN="-uroot -prootpassword"
BPSIZE=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL VARIABLES LIKE 'innodb_buffer_pool_size'" | awk '{print $2}'`
BPDATA=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL STATUS LIKE 'Innodb_buffer_pool_pages_data'" | awk '{print $2}'`
BPPAGE=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL STATUS LIKE 'Innodb_page_size'" | awk '{print $2}'`
BPDATABYTES=`echo ${BPDATA}*${BPPAGE}|bc`
BPPCTUSED=`echo ${BPDATABYTES}00/${BPSIZE}|bc`
KBSIZE=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL VARIABLES LIKE 'key_buffer_size'" | awk '{print $2}'`
KBUSED=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL STATUS LIKE 'Key_blocks_used'" | awk '{print $2}'`
KCSIZE=`mysql ${MYSQL_CONN} -ANe"SHOW GLOBAL STATUS LIKE 'key_cache_block_size'" | awk '{print $2}'`
KBDATABYTES=`echo ${KBUSED}*${KCSIZE}|bc`
KBPCTUSED=`echo ${KBDATABYTES}00/${KBSIZE}|bc`
echo "InnoDB Buffer Pool Size : ${BPSIZE}, (${BPPCTUSED}) Percent Full"
echo "MyISAM Key Buffer Size : ${KBSIZE}, (${KBPCTUSED}) Percent Full"

If you want to see how much data you actually have run this:
SELECT IFNULL(B.engine,'Total') "Storage Engine",
CONCAT(LPAD(REPLACE(FORMAT(B.DSize/POWER(1024,pw),3),',',''),17,' '),' ',
SUBSTR(' KMGTP',pw+1,1),'B') "Data Size",
CONCAT(LPAD(REPLACE(FORMAT(B.ISize/POWER(1024,pw),3),',',''),17,' '),' ',
SUBSTR(' KMGTP',pw+1,1),'B') "Index Size",
CONCAT(LPAD(REPLACE(FORMAT(B.TSize/POWER(1024,pw),3),',',''),17,' '),' ',
SUBSTR(' KMGTP',pw+1,1),'B') "Table Size"
FROM (SELECT engine,SUM(data_length) DSize,SUM(index_length) ISize,
SUM(data_length+index_length) TSize FROM information_schema.tables
WHERE table_schema NOT IN ('mysql','information_schema','performance_schema')
AND engine IS NOT NULL GROUP BY engine WITH ROLLUP) B,(SELECT 3 pw) A
ORDER BY TSize;

This will show how much InnoDB and MyISAM data you have in the MySQL Instance.
Please keep in mind

the sum of MyISAM Indexes is the Maximum key_buffer_size you can allocate for your instance
the sum of InnoDB Data and Index Pages is the Maximum innodb_buffer_pool_size for your instance

For more info on how to size the key_buffer_size and innodb_buffer_pool_size, please see my post in the DBA StackExchange.


Error when trying to attach an used address of shared-memory

Question: Receiving the message "Invalid argument" when using shmget with the second parameter not being NULL.
It compiles ok, but when executing, I get this error message.
I've been stuck on this all day long.  Hope you can help me! :)
#include <sys/ipc.h>
#include <sys/shm.h>
#include <stdlib.h>
#include <stdio.h>

int main()
{
    int idSharedMem;
    int *varSharedMem1;
    int *varSharedMem2;

    /* Create the shared memory */
    idSharedMem = shmget((key_t) 0001, sizeof(int), IPC_CREAT | 0666);

    if (idSharedMem == -1)
    {
        perror("shmget");
    }

    /* Allocate a memory address and attached it to a variable */
    varSharedMem1 = shmat(idSharedMem, NULL, 0);

    if (varSharedMem1 == (int *) -1)
    {
        perror("shmat1");
    }

    /* Sign a value to the variable */
    *varSharedMem1 = 5;

    /* Attach an existing allocated memory to another variable */
    varSharedMem2 = shmat(idSharedMem, varSharedMem1, 0);

    if (varSharedMem2 == (int *) -1)
    {
        /* PRINTS "shmat2: Invalid argument" */
        perror("shmat2");
    }

    /* Wanted it to print 5 */
    printf("Recovered value %d\n", *varSharedMem2);

    return(0);
}

Comment: You didn't state which function is giving you the error. If `shmget` -- what is the value of 'SHMMIN' on your platform? If it's less than `sizeof(int)`, this is documented and expected behavior.
Comment: Can you explain what the second call to `shmat` was supposed to do? The code doesn't make any sense, at least to me. And I can't figure out what its purpose is supposed to be. If you're trying to map the same segment again, why are you passing in `varSharedMem1`? There are a few things you could be trying to do. One is best done by `varSharedMem2=varSharedMem1;`. The other is best done by passing `NULL` as the second parameter. As is, I can't figure out what you are expecting to accomplish with that call.
Comment: @DavidSchwartz
`echo $SHMMIN` returns blank :(
Am I doing it right? I'm newbie to unix.

I know I can simply assign one pointer to another, but I have to do it by using the second parameter. This is what the second parameter is supoesed to do right?
At least it's what I've understood from manual.

Thanks for the answers!
Comment: Are you doing **what** right? I have no idea what you're trying to do. From what I can tell, it appears you are trying to create a second mapping right on top of the first mapping, but that makes no sense. If that's what you're trying to do, just don't do it -- it's already mapped. If you're trying to do something else, please explain what that something else *is*!
Comment: Thanks, David! It's what you told, but I could just understand well it on @nos post. The right thing I was wasking if i've done right was echoing the SHMMIN system variable :p
Answer: With shmat(idSharedMem, varSharedMem1, 0); you're trying to attach the segment at the location of varSharedMem1. However you have previously attached a segment at that location which will result in EINVAL. Linux provides a SHM_REMAP flag you can use to replace previously mapped segments.
shmat manpage:

The (Linux-specific) SHM_REMAP flag may be specified in shmflg to
  indicate that the mapping of the segment should replace any existing
  mapping in the range starting at shmaddr and continuing for
         the size of the segment.  (Normally an EINVAL error would result if a mapping already exists in this address range.)  In this
  case, shmaddr must not be NULL.
Comment: Hmm! Thanks nos!
I was working like if the second paramter was an used memory address, it would return me the same address I've passed to it, like the function was confirming that that address can be used.

But from what you've told, I can infer that I just need to pass the second parameter if I want to specify the new variable address, and it'll only work if it's not already used.

Thanks :D
Comment: What was strange to me was the message _Invalid argument_. It was looking like was a wrong variable type :/
Comment: Some reference, for those who will get here:
[link](http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/port/sysv_shmem.c?rev=1.55)
Answer: From shmat man page:
If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the
address equal to shmaddr rounded  down  to  the nearest multiple of SHMLBA.
Otherwise shmaddr must be a page-aligned address at which the attach occurs.

more from shmat man page:
Using shmat() with shmaddr equal to NULL is the preferred, portable way of
attaching a shared memory segment.

To make a second attach, just:
varSharedMem2 = shmat(idSharedMem, NULL, 0);

Comment: Thanks, Pepe! But I've already read the man page a dozen times, I really want to do it passing the second parameter not null.
Comment: @marcioggs you can do this but you must ensure your second parameter is page aligned
Comment: Anyway this second attach would try to get a free memory address, but this wouldn't be possible, since the shared-memory that has been created just hold enough space for a single int.
Comment: What do you mean by page aligned? Gonna Google for it.
Comment: Read this [link](http://www.devx.com/tips/Tip/13381) and `posix_memalign`man page
Comment: @marcioggs You have already attached the segment at the memory location of varSharedMem1(which is why it fails the 2. time) , why do you need to attach it at the same location again ?
Comment: I've thought of attached as "bringing the memory adress to the process", but it works more like "signing it to any process use"
Comment: @marcioggs You have a shared memory segment. shmat(xxx,NULL,0) will attach that segment at an a suitable address in your process and return the address that the system chose. shmat(xxx,varSharedMem1,0) will try to attach that segment to the address in your process stored by the varSharedMem1 pointer.
Comment: @nos It's was already very clear to me the way shmat works from your first post :) Thanks for this detailed explanation. Saved my day!


Is there a notion of "degree of discontinuity" in topology?

Question: We can defined the property of continuous functions between topological spaces. however, some discontinuities are worse than others. For example, if we have two functions $f,g:\{A,B,C\}\to \mathbb Z$, with the obvious topologies on these sets, and we know that $f(A)=1, f(B)=2, f(C)=4$, while $g(A)=0, g(B)=-4, g(C)=5$. Then in some sense, $g$ is "more discontinuous", since firstly, it goes down and up again, and secondly, it is discontinuous at multiple points, thirdly, the points are "farther away".
Comment: From the point of topology, there's no difference between the set $\{1,2\}$ and the set $\{1,100\}$. While the second has a bigger difference between the elements, "difference" is a term that is alien to topology. And so is "direction".
Comment: Topology is not going to give you the notion of "discontinuous" you are seeking here.  All your spaces are discrete, and every function between discrete topological spaces is continuous, and not discontinuous in any way.  If you have metric spaces, you maybe want to look at notions like distortion, or the best Lipschitz constant for a function (and its inverse?).
Comment: @celtschk, I noticed the curly brackets in $\{A,B,C\}$ didn't appear. Do you still agree with your statement? Here's why I intuitively think topology can say something about it: if we have the set $\{ -3, -2, -1, 0, 1, 2, 3\}$ with the standard topology on it, then $2$ is "closer" to $0$, than $3$ is to $0$, because every neighbourhood of $0$ that contains $3$, also contains $2$, but not the other way around. Doesn't that allow us to define a "degree of discontinuity" purely based on topology?
Comment: @Programmer2134: Yes. Topology is exactly what you get if you *forget* about things like difference. All topology sees from the set $\{-3,-2,-1,0,1,2,3\}$ is a set of seven separated elements. As far as topology is concerned, it could as well be the standard basis of $\mathbb R^7$, where any pair of vectors has the same distance. Topology cannot see the difference between both.
Comment: @celtschk, but if we have the standard topology on this set, then $0$ has the neighborhouds $\{0\}, \{0,1\},\{-1,0,1\}, \{0,2\},...$. Two of these neighbourhoods are $\{0,1,2\}$, and $\{0,1,2,3\}$, But there is no neighbourhood of $0$ that contains $3$ but not $2$. Doesn't this allow us to say that $2$ is closer to $0$ than $3$ is to $0$?
Comment: @Programmer2134: Wrong, $\{0,3\}$ is indeed a neighbourhood of $0$ that does not contain $2$.
Comment: @celtschk, why do you say $\{0,3\}$ is a neighborhood of $0$? Of course we can define the discrete topology on the set, and then it is a neighborhood trivially. But I'm assuming the topology that is generated by saying "a subset $X$ of $\mathbb Z$ is an open set iff it is not the case that for some $x\in \mathbb Z,\nin X$ and some $y,z\in X$, we have $y<x<z$". This assumes the standard order on $\mathbb Z$, and rules out $\{0,3\}$ as an open set. (Note that we're not using a metric here, and really not even an order, since we can drop the order once the topology is generated).
Comment: What you describe is not a topology, as the union of open sets and the intersection of finitely many open sets is open, by the definition of topology. So if $\{-1,0\}$ and $\{3,4,5\}$ are open sets, then $\{-1,0\}\cup \{3,4,5\} =\{-1,0,3,4,5\}$ is also open, and if then $\{0,1,2,3\}$ is also open, then so is $\{-1,0,3,4,5\}\cap\{0,1,2,3\}=\{0,3\}$. Therefore you cannot have a topology in which all of $\{-1,0\}$, $\{0,1,2,3\}$ and $\{3,4,5\}$ are open, but $\{0,3\}$ is not. Note that indeed the standard topology on $Z$ *is* the discrete topology.
Comment: @celtschk, Ah I was wrong. Interesting. So then we probably do need a metric, yes.
Answer: Suppose you have a function $f: X \to M$ where $M$ is a metric space and $X$ is a first countable topological space. Take $x \in X$ and a neighborhood base $(U_n)_n$ for $x$ with $U_1 \supseteq U_2 \supseteq U_3 \supseteq \dots$. On each $U_i$, we can define the oscillation of $f$ as $\sup_{y \in U_n} f(y)- \inf_{y \in U_n} f(y)$. Then $f$ is continuous at $x$ if and only if the oscillation over $U_n$ goes to $0$ as $n \to \infty$. You can measure how discontinuous $f$ is at $x$ by looking at the $\limsup$ of the oscillation over $U_n$ as $n \to \infty$. You can check that this matches up with your intuition.
The example you gave is pretty weird. I'm assuming $A,B,C$ are the three points in the topological space. If you have a discrete space, it doesn't make sense to talk about how discontinuous something is. Like I wouldn't say $g$ is more discontinuous than $f$ in the example you gave.
Comment: Thank you. You say we cant say anything about how discontinuous a function is in a discrete space. Wouldn't you say a hash function is "highly discontinuous"? a hash function is an example of a discontinuous function between topological spaces (metric spaces in fact), correct?
Comment: @Programmer2134: For a typical hash function, I would think of the codomain as some finite set of integers, e.g. $\{0, \dots, 2^{256}-1\}$, with the obvious metric.  But what is the domain, and what metric is it equipped with?
Comment: let's say the domain is also a finite set of integers, or a finite string of letters.  the metric could be for example, "the amount of changes to letters that need to be made to turn string $x$ into string $y$"
Comment: @Programmer2134 I feel like you're not talking about continuity but rather variation. If you have some function $f: \{1,2,\dots,n\} \to \{1,2,\dots,m\}$, it seems you care about $\sum_{j=1}^{n-1} |f(j+1)-f(j)|$.
Answer: For metric spaces there is the oscillation of $f$ at $x$ defined as
$$\operatorname{osc}(f,x) = \inf_{\delta >0} \operatorname{diam} f[B(x,\delta)] $$
see also here
It is $0$ iff $f$ is continuous at $x$. In your discrete example all fucntions are continuous. so no insight is gained.


Sobolev spaces involving time

Question: In his notes on the Cauchy problem in general relativity, Ringstrom has the following lemma:
Lemma A.5.  Let $I$ be an open interval. If $u \in L^\infty(I; H^k(\mathbb{R}^n))$ for some $k \geq 0$, then there is a function $U \in L^2_{\text{loc}}(I \times \mathbb{R}^n)$ such that for any $\phi \in C_0^\infty(I \times \mathbb{R}^n)$,
$$
\langle \phi, u \rangle = \int_I (\phi(t), u(t))_{L^2(\mathbb{R}^n)}\, dt = \int_{I \times \mathbb{R}^n} \phi U\, d(x, t).
$$
Furthemore, for every multi-index $\alpha$, there is a function $U_\alpha \in L^2_{\text{loc}}(I \times \mathbb{R}^n)$, such that for any $\phi \in C_0^\infty (I \times \mathbb{R}^n)$,
$$
\int_{I \times \mathbb{R}^n} \partial^\alpha \phi U\, d(x, t) = (-1)^{|\alpha|}\int_{I \times \mathbb{R}^n} \phi U_\alpha \, d(x, t),
$$
i.e. $U$ is weakly $k$-times differentiable in $x$. What follows is then a somewhat technical (though not super long) proof.
My question: (In brief: Why is this needed?) Intuitively, $U$ should just be $u$ itself. My question is, why isn't it? Should I view this as a technical lemma that basically lets us work with $U$ instead of $u$, since $u$ is a priori a "stranger" object, in that it actually takes a.e. $t \in I$ and spits out a function $u(t) \in H^k(\mathbb{R}^n)$? Is the issue here a kind of type-mismatch, in this sense? In that we are just trying to define a more easy-to-work-with object $U$ which is more or less $u$? Or is there something more going on that I'm not seeing?
Answer:
Should I view this as a technical lemma that basically lets us work with $U$ instead of $u$, since $u$ is a priori a "stranger" object, in that it actually takes a.e. $t \in I$ and spits out a function $u(t) \in H^k(\mathbb{R}^n)$?

Probably this. There is danger afoot when you consider $L^\infty$ Bochner spaces: tread carefully. Consider the following example (exercise 1.11 in Robinson, Rodrigo & Sadowski's The Three-Dimensional Navier-Stokes Equations)

Show that the function defined on $(0,1) \times(0,1)$ by
$$
f(x, t)=\left\{\begin{array}{ll}
0 & x<t \\
1 & x \geq t
\end{array}\right.
$$
which is clearly an element of $L^{\infty}((0,1) \times(0,1))$, is not an element of $L_{t}^{\infty}\left((0,1) ; L_{x}^{\infty}(0,1)\right) .$ [Hint: define $F:(0,1) \rightarrow L_{x}^{\infty}(0,1)$ by setting
$F(t)(x)=f(x, t)$, and show that $F^{-1}\left(B_{1 / 2}(F(t))\right)=\{t\}$ for any $t \in(0,1)$
Hence find an open subset $U$ of $L_{x}^{\infty}(0,1)$ such that $F^{-1}(U)$ is not measurable.] (This example is due to Juan Arias de Reyna.)

Not directly related, but 'switching the argument around' reminded me   also of the following (Paraphrasing of example 1.3.1 in Hamilton's The Inverse Function Theorem of Nash and Moser.) Let $C^k_p$ denote the Banach space of $C^k$ $2\pi$-periodic functions, and consider
$$\big( T (t)f\big)(x) = f(x+t)$$
Then

$T:\mathbb R\times C^0_p\to C^0_p$ is jointly continuous;
$T : \mathbb R \to L(C^0_p, C^0_p)$ is not continuous (wrt operator norm topology);
$T:\mathbb R \to L(C^1_p, C^0_p)$ is continuous;
$T:\mathbb R \to L(C^1_p, C^1_p)$ is not.

The lack of continuity in 2 stems from the lack of equicontinuity of the space all functions in $C^0_p$. So obvious sounding things still need to be checked.
Another (even less related) weird thing: if $X$ is infinite dimensional, then simple functions are not dense in $L^\infty(0,T; X)$...if you want to understand Bochner spaces carefully, the first few videos of this youtube lecture series by Amenta (which I thoroughly enjoyed) are relevant.
Comment: Thanks for the youtube link, the first video had me hooked!

Sobolev spaces involving time


updating entity with new attribute

Question: I have a 100 plus Entities of kind Employee in datastore. The key for the entity is username who logs in to the application. I need to add new attribute to Employee entity with some default value.
How do I update all the entities in one shot? I cannot login with each users account to update the entity with new attribute.
I prefer to use java to do that.
Answer: If what you want is a mass update, there is none in the datastore unfortunately.
I would suggest grabbing the full Employee table, and then iterating through your results, adding the default value to your new attribute, and then put the entity back.  Something like (might need some adjustements, this is just off the top of my head):
<code>Query q = new Query("Employee")
PreparedQuery pq = datastore.prepare(q);

for (Entity result : pq.asIterable()) {
    result.setProperty("newProperty","default Value");
    result.put();
}
</code>
And remember to update your model, adding the default value you want in it.  If you don't, any new data you put in won't have that default.

updating entity with new attribute


How to save a character with a line above (like this ō) in a file system?

Question: What file format should I use? It is for PHP to afterwards retrieve such a character and for HTML to display.
Comment: Are you trying to create a file with this character in the name or to save the character in the content of the file (such as a text file)? What file system?
Comment: As indicated in reply of quasi-same question, closed a few minutes ago, this is not so much a matter of file format than one of character encoding.  Also, the encoding used when saving must match the encoding used for display purposes.
Comment: I'd like to learn about how both to create a file with this character in the name and to save the character in the content of the file. Also both, ways for Windows and Linux.
Comment: mjv, can't these characters be coped from a web page and pasted in a plain .txt file?
Comment: @Coder123 Define "plain text file". The common [but obsolecent?] understanding of such a file was one where each character is saved as one and only one byte in the file, an read in like fashion, to be used in the context of an implicit codepage.  In that sense, probably not.  Now, if you understand a plain text file, as one that contains characters encoded in say utf-8 (which btw, mostly maps to 1 byte characters for most characters found in European languages), and which is then interpreted by an application that understands utf-8, then in such a file, yes.
Answer: You should be encoding the text as UTF8, then writing it to the file.
If you're doing this with PHP, then (I think) it'll be something like:
<code>$f = fopen("/path/to/file", "w");
fwrite($f, utf8_encode($some_string));
fclose($f);
</code>
(although, this assumes that PHP is using Unicode internally… But, then, it might not – my memory is kind of hazy)
Comment: The implicit input to PHP's utf8_encode() is an `ISO-8859-1` string, and this would therefore not answer Coder123's question, one of the reasons being (not the only one) that 8859-1 does not include any o-with-a-bar looking character.


Attachable Piano Stand

Question: I have a Yamaha P45 keyboard and it has screw holes at the bottom.  But I can't seem to find a piano stand that will attach to it.  The Yamaha L3C doesn't specify my type of piano and I don't think it would be strong enough.  Why would Yamaha put screw holes if they can't be used?
Are the Z type stands easy to knock the keyboard off? I can't find a Z type stand that's attachable to a keyboard. The 20 inch depth would seem to hold a keyboard on a little better than a 16 inch.
Answer: Most of the time, a piano as light as this would just be put onto an X-stand. I only use X-stands for the lightest of keyboards - and never for gigging! A Z-stand will do fine instead, or one with four legs is even better. But for home use an X-stand will be plenty good enough. The screw holes are probably for fixing it to the $100 wooden stand made by Yamaha for it. (A quick Google provided this).
Even on gigs, keyboards  are not usually attached to their stands. If you're in a situation where it's likely to get knocked off, then even attached, it's not a good situation to have it in! Velcro between keyboard and stand might be an answer.
Comment: Thanks Tim. I appreciate the comments.  I will go ahead and get a Z type stand and not be concerned with attaching it to the piano. They seems to offer a little more room for legs.  They appear to me to be a little more stable but I have no experience with them as I am a beginner.  I use it only at home.

Attachable Piano Stand


JavaScript - Dúvida ao manipular valores de div's com variáveis

Question: Boa tarde !
Estou com uma dúvida em relação a manipulação de valores de div's...
Ao arrastar uma imagem da div fundo1 (azul) ou a fundo2 (amarela) para div conteudo, gostaria que os valores presente dentro da div fundo1 (ou fundo2)
fosse para o textbox como na imagem a seguir:

Se for a imagem da div azul, o textbox ficará com o valor 1
Se for a imagem amarela, ficará com o valor 2.
Estou com problemas ao reconhecer o valor da div valorFundo1 (valor 1) e valorFundo2 (valor 2) dentro das div's e apresenta-la no textbox
Perdão a ignorância qualquer coisa e obrigado !!

//Drag'n Drop functions
var elementCounter = 0; // Para designar a ID do elemento dropado.

function allowDrop(ev) {
  if (ev.target.getAttribute("droppable") == "false") {
    ev.dataTransfer.dropEffect = "none"; // dropping is not allowed
    ev.preventDefault();
  } else {
    ev.dataTransfer.dropEffect = "all"; // drop it like it's hot
    ev.preventDefault();
  }
}

function drag(ev) {
  ev.dataTransfer.setData("text", ev.target.id);
  ev.dataTransfer.effectAllowed = "copy";
}

function drop(ev, div) {
  ev.preventDefault();
  var x = ev.clientX;
  var y = ev.clientY;
  var data = ev.dataTransfer.getData("text");
  var copyimg = document.createElement("img");
  var original = document.getElementById(data);
  var altura = document.getElementById("drag1").height;
  var largura = document.getElementById("drag1").width;
  var c = x - largura;
  var d = y - altura;
  copyimg.src = original.src;
  div.appendChild(copyimg);
  if (original.parentNode.id == "conteudo") {
    if (ev.target.tagName == "IMG") { // Se a imagem estiver dropando em uma imagem
      console.log(ev.target.tagName);
      original.parentNode.removeChild(original);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 70) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    } else { // Se não, é porque a imagem está sendo dropada em uma div
      original.parentNode.removeChild(original);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 50) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    }
  } else {
    if (ev.target.tagName == "DIV") { // Se a imagem está sendo movimentada dentro da div conteudo
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 50) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    } else { // Se não, é porque está sendo dropada na IMG
      console.log(ev.target.tagName);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 70) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    }
  }
}
.conteudo-externo {
  width: 55%;
  height: 60%;
  z-index: 3;
  background: red;
  border: 1px solid;
  float: left;
}

.conteudo {
  width: 320px;
  height: 300px;
  border-radius: 50%;
  border: 1px solid #000;
  z-index: 5;
  background: blue;
  margin: 35px auto;
}

.img {
  z-index: 1;
  width: 130px;
  height: 130px background-position:center;
  opacity: 0.5;
}

#fundo1 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: blue;
  border: 1px solid;
  float: left;
}

#fundo2 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: yellow;
  border: 1px solid;
  float: left;
}

}

    <input id="totalizador" name="totalizador" class="text" size="1" type="text" readonly="true" value="0" maxlength="999999" style=";border:0px solid #fff" />

 1

 2

Comment: Tanto faz se o drag cair dentro do circulo ou fora?
Comment: A parte vermelha de fora está como draggable="false", deixei para cair somente na parte azul (o que é interessante para nós)
Answer: Podes fazer isso juntando duas linhas:
var valorOrigem = original.nextElementSibling.textContent;
div.nextElementSibling.value = Number(div.nextElementSibling.value) + Number(valorOrigem);

//Drag'n Drop functions
var elementCounter = 0; // Para designar a ID do elemento dropado.

function allowDrop(ev) {
  if (ev.target.getAttribute("droppable") == "false") {
    ev.dataTransfer.dropEffect = "none"; // dropping is not allowed
    ev.preventDefault();
  } else {
    ev.dataTransfer.dropEffect = "all"; // drop it like it's hot
    ev.preventDefault();
  }
}

function drag(ev) {
  ev.dataTransfer.setData("text", ev.target.id);
  ev.dataTransfer.effectAllowed = "copy";
}

function drop(ev, div) {
  ev.preventDefault();
  var x = ev.clientX;
  var y = ev.clientY;
  var data = ev.dataTransfer.getData("text");
  var copyimg = document.createElement("img");
  var original = document.getElementById(data);
  var valorOrigem = original.nextElementSibling.textContent;
  div.nextElementSibling.value = Number(div.nextElementSibling.value) + Number(valorOrigem);
  var altura = document.getElementById("drag1").height;
  var largura = document.getElementById("drag1").width;
  var c = x - largura;
  var d = y - altura;
  copyimg.src = original.src;
  div.appendChild(copyimg);
  if (original.parentNode.id == "conteudo") {
    if (ev.target.tagName == "IMG") { // Se a imagem estiver dropando em uma imagem
      console.log(ev.target.tagName);
      original.parentNode.removeChild(original);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 70) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    } else { // Se não, é porque a imagem está sendo dropada em uma div
      original.parentNode.removeChild(original);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 50) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    }
  } else {
    if (ev.target.tagName == "DIV") { // Se a imagem está sendo movimentada dentro da div conteudo
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 50) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    } else { // Se não, é porque está sendo dropada na IMG
      console.log(ev.target.tagName);
      copyimg.id = "dropped_elem" + (++elementCounter);
      copyimg.setAttribute("style", "position: fixed; top: " + (y - 50) + "px; left:" + (x - 70) + "px;");
      copyimg.setAttribute('draggable', true);
      copyimg.setAttribute('ondragstart', "drag(event)");
    }
  }
}
.conteudo-externo {
  width: 55%;
  height: 60%;
  z-index: 3;
  background: red;
  border: 1px solid;
  float: left;
}

.conteudo {
  width: 320px;
  height: 300px;
  border-radius: 50%;
  border: 1px solid #000;
  z-index: 5;
  background: blue;
  margin: 35px auto;
}

.img {
  z-index: 1;
  width: 130px;
  height: 130px background-position:center;
  opacity: 0.5;
}

#fundo1 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: blue;
  border: 1px solid;
  float: left;
}

#fundo2 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: yellow;
  border: 1px solid;
  float: left;
}

}

    <input id="totalizador" name="totalizador" class="text" size="1" type="text" readonly="true" value="0" maxlength="999999" style=";border:0px solid #fff" />

 1

 2

Olhando para o código há muita repetição... isso pode ser feito com menos código.
Uma sugestão seria:

//Drag'n Drop functions
var elementCounter = 0; // Para designar a ID do elemento dropado.
var originalElement;

function onDragOver(ev) {
  var isDroppable = ev.target.getAttribute("droppable") !== "false";
  ev.dataTransfer.dropEffect = isDroppable ? "all" : "none";
  ev.preventDefault();
}

function onDragStart(ev) {
  originalElement = ev.target;
  ev.dataTransfer.setDragImage(originalElement, 15, 15);
}

function getCoords(ev) {
  var mouseCoords = {
    x: ev.clientX,
    y: ev.clientY
  }
  return mouseCoords

}

function drop(ev) {
  var valorOrigem = originalElement.nextElementSibling.textContent;
  var input = ev.target.nextElementSibling;
  input.value = Number(input.value) + Number(valorOrigem);

  var clone = originalElement.cloneNode(true);
  var mouseCoords = getCoords(ev);
  clone.style.left = mouseCoords.x - 15 + 'px';
  clone.style.top = mouseCoords.y - 15 + 'px';
  ev.target.appendChild(clone);
}
.conteudo-externo {
  width: 55%;
  height: 60%;
  z-index: 3;
  background: red;
  border: 1px solid;
  float: left;
}

.conteudo {
  width: 320px;
  height: 300px;
  border-radius: 50%;
  border: 1px solid #000;
  z-index: 5;
  background: blue;
  margin: 35px auto;
}

.conteudo [draggable] {
  position: absolute;
}

.img {
  z-index: 1;
  width: 130px;
  height: 130px background-position:center;
  opacity: 0.5;
}

#fundo1 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: blue;
  border: 1px solid;
  float: left;
}

#fundo2 {
  width: 200px;
  height: 200px;
  z-index: 3;
  background: yellow;
  border: 1px solid;
  float: left;
}

}

    <input id="totalizador" name="totalizador" class="text" size="1" type="text" readonly="true" value="0" maxlength="999999" style=";border:0px solid #fff" />

 1

 2

Comment: Vou estudar o código !!
Obrigado pela ajuda @Sergio !!
Comment: @Sora sim, se for um elemento único na página podes dar-lhe um ID. Senão é procurar por relações no DOM por exemplo.
Comment: @Sora se tens um elemento com ID podes usar `document.getElementById('a-id-do-element')`. Se fôr um `input` usas `.value`, se for uma `div` usas `.textContent`. Independentemente de onde esteja visto IDs serem únicos. Assim, para um `input` podias fazer: `document.getElementById('totalizador').value = 'novo valor';`
Comment: caso se o text estivesse fora da div conteudo, teria como eu atribuir o valor a ele mesmo assim??
Comment: Tens como me dar um exemplo de atribuição por favor? eu atribui o id do text como Totalizador, porém ao executar o código, retorna um erro de valor nulo. Seria por getElementBy??
Comment: $(totalizador).setAttribute("value", valorOriginal); algo assim?
Comment: @Sora é um `input`? ou uma `div`?
Comment: O `input` totalizador vai receber o valor de uma `div` (valor1) separada, que está dentro de uma outra div (`fundo1`)
Comment: Minha dúvida é se o input estiver fora dessas div's apresentadas como conteúdo ou conteúdo externo... Suponhamos que ela esteja em uma outra div chamada `totalFoot`
Comment: Aaaah, entendi, muito obrigado pela explicação !!! Tenha um ótimo dia @Sergio


Does Yoast SEO put "Focus Keyword" as H1 Tag?

Question: At the outset thanks you all for not assuming that I do coding. My question is related to my very simple and a first-timer's website http://enconengineers.in/ written in Drupal-8 with Zircon theme. While checking my Content with Yoast SEO module, it asks for "Focus Keyword" for the H1-Tag. It also red-flags about missing H-2 Tag. My query is does Yoast SEO automatically put the focus keyword as a H1-Tag in the content?
If not, shall appreciate if anyone can share the Steps/ways/setting-processes using which I can globally (for my website) define H1-Tag (which will always be a keyword (different for each content) in each content's Title Tag) and H2-Tag (which will be in 1st/2nd paragraph of each content) for all the content pages?
Answer: Without having to heavily modify the theme itself the most direct approach to solving your problem to have a better Yoast score would be to install and use the Meta Tag module in Drupal 8.
It sets many typical metatags in the head portion of every HTML page Drupal renders for visitors. If you need a custom meta tag for h1 focus word or h2 related items, I'm sure the metatag API supports creating your own metatags that can make use of Tokens.
Either use pre-existing Drupal 8 tokens provided by the Token module, or use the API provided by the Token to programmatically create tokens specific to your site that implement the functionality you need for a focus word or an H2 related token. This of course requires some PHP and web development experience, or hiring a web developer.


jQuery How to make a color palette with RGB?

Question: Not sure if the title explains itself, but probably not!
Anyway, I have 50 divs and I would like to set the background color from White to Dark Yellow.
The first div would be white and the last dark yellow.
The RGB value for white is 255, 255, 255
The RGB value for dark Yellow is 227, 151, 4
How could I do that with javascript (jQuery) so iterate through each div and make it gradually get darker?
Here's a screen shot

Thanks a lot
Comment: You're going to have to find / create some sort of formula for how each one of the R - G - B needs to decrease and iterate through it.
Comment: The RGB part shouldn't be hard, but you want the gradients and color-choosing effect as in the pic as well?
Comment: I created a loop and i am trying to iterate though the colors but doesnt work
Comment: Post the code of what you have so far
Answer: var divs = $('div'),
    len = divs.length;

var targ_R = 227,
    targ_G = 151,
    targ_B = 4,

    inc_R = (255 - targ_R) / len,
    inc_G = (255 - targ_G) / len,
    inc_B = (255 - targ_B) / len;

divs.css("backgroundColor", function(i, curr) {
    return "#" + toHex(255 - (i * inc_R)) +
                 toHex(255 - (i * inc_G)) +
                 toHex(255 - (i * inc_B));
});

function toHex(n) {
    var h = (~~n).toString(16);
    if (h.length < 2)
        h = "0" + h;
    return h;
}

DEMO: http://jsfiddle.net/RSyCM/
Comment: @peduarte: Wasn't sure if jQuery fixed any cross-browser issues for rgb, so I went with hex since I know it has wide support.
Comment: Last 2 divs are white (check with an outline).
Comment: Just one question, why did you conver them to hex? can't we just keep it as rgb?
Answer: Try this - DEMO
var r, g, b;

for (var i = 0, count = $("div").length; i < count; i++) {
    r = 255 - i * 3;
    g = 255 - i * 10;
    b = 255 - i * 25;

    $("div").eq(i).css("background", "rgb(" + r + ", " + g + ", " + b + ")");
}

UPDATE
To make the last one rgb(227, 151, 4) - DEMO
Comment: 3, 10 and 25 - that's just approximate. You can calculate them pretty precise, i.e. for red: `(255 - 227) / 50` - that's your multiplication value
Comment: Plus 1 for demo! Nearly worked. But I have 50 divs all together, and now it goes to red. Screen shot here: http://cl.ly/image/0D3T2w1O0y2f
Comment: I dont understand how you are coming up with the multiplication values (3, 10 and 25) or yours, Zoltan. there must be a formula somewhere. The last div needs to have the rgb values of 227, 151, 4
Answer: Here's a variation of Zoltan Toth's answer using the pusher.color library, in case you want to use a library to handle the parsing and blending.
DEMO
var color0 = pusher.color('white');
var color1 = pusher.color('rgb', 227, 151, 4);

for (var i = 0, count = $("div").length; i < count; i++) {
    var amount = i / (count-1);
    var result = color0.blend(color1, amount);
    $("div").eq(i).css("background", result.html());
}


MySql Convert to UTC

Question: I am trying to convert CET date to UTC Format in MySql. Whats wrong with this?
SELECT
CONVERT_TZ('2018-01-26T06:15:00+01:00', @@session.time_zone, '+00:00');

Expected:
2018-01-26T05:15:00Z
Actual:
2018-01-26T06:15:00Z
I guess it is not taking the +01:00 component
Answer: MySQL's date format doesn't have a timezone. It is interpreting your '2018-01-26T06:15:00+01:00' as '2018-01-26 06:15:00'.
The 1-st parameter is the time, the 2-nd parameter is the time zone for the 1-st and the 3-rd is the result time zone.
SELECT
  CONVERT_TZ('2018-01-26T06:15:00', '+01:00', '+00:00');

MySql Convert to UTC


Return e console.log não funcionam como esperado no Codecademy

Question: Estou fazendo curso de Javascript no Codecademy, fiz a função, ela está retornado o que o exercício pede, mas não passo.
Comment: Testa escrever o `oi` com letra grande... -> `Oi`. Esse code academy é um rôbot...
Answer: O return é uma instrução para a função, não tem ligação algum com o console.log.
O console.log é um metodo que foi "idealizado" faz pouco tempo (quando já existia javascript e navegadores) para depurar os scripts, você não usa ele para "produção" mas sim para desenvolvimento.
O correto seria "retornar" usando return e capturar este retorno com console.log
O return funciona assim
function test(){
    return "test";
}

console.log(test());

Então o código deveria ser:
function nameString(name)
{
     return "Oi, eu sou " + name;
}

console.log(nameString("Allan"));
console.log(nameString("Susie"));
console.log(nameString("Fred"));

Note que o código é executado até antes do return, mas o que vier depois não é executado:
function nameString(name)
{
     alert(1);//Isto será executado
     return "Oi, eu sou " + name;
     alert(2);//Isto NÃO será executado
}

Comment: @AllanSantos Isto o `return` é uma instrução para a função, não tem ligação algum com o `console.log`, o `console.log` é um metodo que foi inventado faz pouco tempo (quando já existia javascript) para depurar os scripts, você não usa ele para "produção" mas sim para desenvolvimento :)
Comment: Já tentei usar o return, mas aparece esta mensagem: "Oops, tente outra vez. Parece que voce nao exibiu nada no console! Voce se lembrou de usar console.log() no resultado de passar seu nome para a função nameString()?"
Comment: @AllanSantos você chamou assim `console.log(nameString("Allan"));`?
Comment: Amigo, muito obrigado.

Apenas pra deixar claro, quer dizer que o "return" não retorna pelo console ?
Answer: Tens de corrigir 2 coisas:

a função tem de dar return
a string que retornas tem de começar com letra grande

O code academy é um Robot, tens de fazer como ele quer.
#1 - Repara que a tua função devia ter return em vez de console.log. O que se pretende é que a função retorne e que depois faças console.log da invocação da função (do seu retorno).
O que se pretende é return "Oi, eu sou" + " " + name; e não console.log("Oi, eu sou" + " " + name);
Depois podes fazer console.log(nameString('Sergio')); ou assim var retorno = nameString('Sergio'); console.log(retorno );`
#2 - Repara que oi e Oi são strings diferentes. tens de começar com letra grande.
O código como deve ser:
var nameString = function (name) {
    return "Oi, eu sou" + " " + name;
};
console.log(nameString('Sergio'));