Best way to load balance across multiple static file servers for even an bandwidth distribution?

Question: First off, I'll explain my situation to you. I'm running a fairly popular website as a side project, so I can't really invest a ton of money into it. I currently have just one server with HAProxy in the front sending out normal requests to Apache, and all static file requests to Lighttpd. This is working really well because all php and post requests get handled by Apache, while all images are sent to the faster Lighttpd (the site is mostly images, so this is really important). It would be nice to not have to set up a sub-domain for serving the images, because short URLs are also really important, thus my reason for using HAProxy.
I've found a hosting provider that offers pretty cheap unmetered bandwidth that I've been using, the problem comes in when I start pushing out as much bandwidth as the 100mbs network card can handle, thus needing a second server.
I've put a lot of thought into my options, so I'll explain each one to you. Hopefully you could provide some insight into which one is the best option for me, or maybe there's another option out there that I haven't thought of yet.
Requirements:

Even bandwidth distribution is a must. I have a pretty powerful server, so scaling up is not an option. I need to scale out to gain more bandwidth.
Short URLs. I really don't wont to setup a subdomain, like img.example.com, to serve my images. example.com/image.jpg is how it is now, and how I would really like it to stay. But if there's no other way, then I understand.
The clostest server handling the request would be really nice, but not a must. Something to keep in mind.

HAProxy to loadbalance:

It would be really easy to do since I'm already using HAProxy anyway. However, I think the problem comes in when distributing bandwidth. I might be wrong on this, but doesn't HAProxy send the request to a server where the server processes it and then sends it back through HAProxy to the client? Thus, all traffic goes back out through the load balancer causing it to use as much bandwidth as all the servers combined.

DNS Round Robin:

This might be my best option. Just replicate the website across multiple servers and do what I'm doing now. The downside is that if one server goes down, clients are still sent to it. I would also need to replicate the site across the multiple servers. I was kind of hoping that I could have one main server that handles everything except static files, and then have a couple of static file servers. I also read that this was sort of the 'poor man's load balancing', and it would be nice to have something a little more sophisticated.

Direct Server Return:

It seems really complicated, but might be a good option. Would I still be able to send certain URLs to certain servers? Like right now with HAProxy, every URL that ends in the right file extension is sent to Lighttpd, while other extensions are sent to Apache. So I would need something similar. Like, all php requests are handled by the same server that's running the balancing software, while all jpg requests are sent out to multiple servers.

Ideally, if HAProxy supported Direct Server Return, then my problem would be solved. I also do not want to use a CDN, because they're really expensive, and this is just a side project after all.
Do you understand my problem? Let me know if I didn't explain something right or if you need more info.
Comment: This is Imgur and recently raised 40 million dollars. :O
Answer: Draw a picture of your request/response cycle for the application and isolate the bottleneck. You are correct that a single proxy distributing load to many application servers will require the aggregate bandwidth of all application servers. The classical solution is RR DNS. Google, Yahoo and Amazon all use this technique with a short TTL. I did some investigation a while back and documented my findings.
Another solution is to use a fancy-pants enterprise load balancing solution using virtual IP addressing to balance requests among multiple application servers with real IP addresses. I have worked with Netscaler and Stonesoft products. Both perform well but have terrible idiosyncrasies and are quite complex.
Comment: Thank you very much. Your survey results were very helpful. I think this is the solution I will finally come to. However, "Like any good researcher, I don't act until I have enough data.". :)
Comment: Thank you for the insight. Unfortunately an ironically the link to your findings seems to be down, can you fix it?
Comment: https://web.archive.org/web/20150520113110/http://lee.rockingtiger.com/posts/29
Answer: Some answers:

Yes, all traffic passes out through HAProxy, as it works as a HTTP level proxy. This will be the same even if HAProxy is installed on a separate server that load balances multiple back end servers. Thus if your hosting provider only supplies 100MBit network ports, and you're already pushing 100MBit, then you have a problem.
Regarding domain, the optimal thing would be to serve images from a different domain than your webapp -- not a subdomain, a different one, so that cookies are not sent along on image requests. See Steve Souders original work, or the implementation here on Stack Overflow. If short URLs are very important to you, maybe the best thing would be to move the webapp off the main URL, i.e. move the file management application to login.sitename.com ?

Do you need authentication on the image requests? If not, how about using something like Amazon S3? It is massively scalable, and the data transfer cost is fairly cheap. In this case I would use somthing like i.sitename.com as a DNS CNAME for the Amazon S3 bucket hostname, see Amazons docs. AFAIK you can't have the root domain name (sitename.com) as a CNAME, so you must use a subdomain like i.sitename.com for this.
You could also hash your images across multiple servers. I.e. you create a DNS structure like login.sitename.com and a.sitename.com ; b.sitename.com ; c.sitename.com et cetera. The "a." and "b." etc servers just contain a file system with images, and a lightweight HTTP server (you're already using Lighttpd, so continue using that. For a future project, I would propose to look at nginx as a better replacement.) When a user uploads an image, you create a hash of a unique identifier, perhaps his username, perhaps the filename, or a combination of multiple identifiers. From this hash, you determine which server to store the image on.
Edit I should have seen that hashing was already discussed. Essentially what I'm proposing here is just to use hashing on the hostname as well, to spread out network traffic evenly on multiple hosts.
I don't know how cheap you need this to be -- but when you're pushing 100MBit of network traffic, then "cheap and good" quickly turns out to be an illusion. Maybe you should look at getting a good business model first, something that provides recurring revenue, and then implement the appropriate technology afterwards?
Answer: I assume HAProxy is on the same server as your other applications?  You could break HAProxy out on to another system to run the requests through and have it send normal requests to one server, and image requests to another server.  The issue these is that all requests are still going to one box, and if you're saturating its bandwidth then that may not help you much.
You say short URLs are important.  Why?  Is it really that big of a deal to switch images from "example.com" to "i.example.com"?  You can set "i" to its own IP on its own server with Lighttpd and bypass HAProxy entirely, solving your throughput problem.  You would also get the benefit of the web browser allowing more requests open at once since it would consider them to be different domain names and could open more concurrent connections.  If the single "i" server got saturated you could employ DNS round-robin to add another one.  Hopefully by that time you're generating enough revenue to implement a better solution.
Comment: Yes, HAProxy is on the same server - I only have one so far. Even if I broke it out to another server, wouldn't all data still travel through the server with HAProxy, as I've explained above?

Short URLs are important because that's sort of the purpose to the site. It's a crossover between ImageShack and TinyPic. The longer the URL, the less point my site has. But as I said, if the only viable option is to setup a subdomain, then I'd just have to do it. I would really prefer not to though.
Answer: Does your hosting provider offer load balancing services? I think is the best solution.
Another way to do it, but it need to be tested, is rewrite (in lighty or apache) the requests. For example: example.com/file.html stays in apache and example.com/image.jpg redirects to i.example.com/image.jpg . All the requests will be managed through apache but reponses (upstream bandwidth) are going to the lighttpd server. The domain is transparent to the user. Still you need to test if apache can handle all the requests or maybe let lighttpd do this job.
You're right all the data pass through HAProxy so you can't (as far as I know) do direct server return with it.
UPDATE
Looking in HAproxy documentation I found the "redir" parameter. I don't know if it can works like apache rewrite but it can be useful. The documentation says:

Main use consists in increasing
  bandwidth for static servers by having
  the clients directly connect to them.

Maybe it works for your case.
Comment: Hey, thanks for the response. I've actually tried this already, and it doesn't work as well in practice as it does in theory. The reason is that Apache handles all the requests, so every time a user hits an image, Apache is spawned, looks at the url, then sends it to it lighty. Which is no different then just having Apache handle the image in the first place. I agree that a load balancer provided by my host is the best option, but it's also one of the most expensive. They charge per concurrent connection, and I get hundreds of them.
Comment: Is different in the way that lighty server will send the response directly to the client consuming his own bandwidth. Problem is that Apache server will handle a lot of requests. Check the update to my answer, i found another solution.
Answer: I'm assuming that with any sizable set of images you're not storing the images based on their original file name as you would run into name conflicts pretty quickly.
A lot of applications that deal with these types of problems use the hash of the file and a directory structure based on that hash.  The directory structure looks like the following where the directory path is the first two characters of the hash then the 2nd level directory is the next two characters in the hash.
<code>/image root/AA/AA/images
/image root/AA/AB/images
</code>
The benefit here is that hashes keep the distribution of files pretty even and it provides you a namespace that is easy to split up over multiple servers.  Basically you serve portions of the hash space from different servers and as you scale you can subdivide this further as required.
The downside is that hashes aren't perfect and there can be collisions.  I'm not sure how this is dealt with.  So that may take a bit of research on your part.  I imagine that a rewrite rule in the proxy should be able to take a hash say A3A8BBC83261.jpg and rewrite it to http://img3.domain.com/A3/A8/BBC83261.jpg.  You may not consider that to be a short url though.
Comment: Yes, that's exactly how I'm storing the images. However, the problem isn't with storage, it's with bandwidth distribution.
Comment: But if you store AA through 33 on one server and 34 through 99 on another server you will not only be balancing the storage problem out but also the bandwidth distribution.
Answer: In your post you mentioned that you felt DNS round robbin might be your best option but you were concerned about a single server failing...
If that's the case take a look at Simple Failover from JH Software.  I've used it in the past and it works very well.
http://www.simplefailover.com
Basically it monitors your servers and when it sees one go down it quickly rewrites the DNS to pull the dead server out of the rotation.
Here's a snippet from their website:

Simple Failover continuously monitors
  your servers to find out which are up
  and which are down, and then it
  dynamically updates your DNS records
  accordingly so that your domain name
  always points to a functional server.
It works with web-servers (HTTP),
  mail-servers (SMTP, IMAP, POP3),
  FTP-servers, and practically any other
  TCP/IP based server type.

As mentioned previously, I've used it in the past for both websites and mail servers.  It performed fairly well.  Failover was pretty quick in most cases (guessing 2-5min) and I'd say almost everyone failed over in less than 15 minutes.
Not necessarily PERFECT... but definitely quick and easy.
NOTE:  This is a windows product.  I'm not sure if they have a linux version or not but you can fail over any server you like since its DNS based.
In our case, we just tossed it on an XP machine, told the machine to reboot once a night, and it ran fine for years.


Coordenadas do Google Maps através de um Widget em um site

Question: Estou montando um sistema de cadastro baseado em geolocalização. Acontece que, em algumas vezes, o cadastro deverá ser feito por telefone, e a atendente é quem vai inserir a localização (coordenadas) do cliente.
Pelo site do Google Maps, simplesmente com um Botão Direito > O que é isto? se consegue as coordenadas exatas do local. Vejam a imagem:

O problema
Porém, não queria ter que usar a página do Google Maps, uma vez que é possível incluir o Widget na minha página. O problema é que polo widget, praticamente só dá pra dar zoom in/out; botão direito nem mesmo "funciona". Então, COMO CONSEGUIR AS COORDENADAS A PARTIR DELE?
Se NÃO FOR POSSÍVEL, qual a ALTERNATIVA de se fazer isto?
Exemplo da página:
Answer: Você pode extrair as coordenadas do centro do mapa na sua página chamando a funcão .getCenter() da API do Google Maps.
Assim o utilizador pode arrastar e centrar o mapa onde quer e depois extrair coordenadas com um botão ou outra ação.
Pode usar assim:
var map = new google.maps.Map(elementoMapa, {
    zoom: 6,
    center: new google.maps.LatLng(38.684748, -9.31572),
    mapTypeId: google.maps.MapTypeId.ROADMAP
});

E depois chamar map.getCenter(); para obter um objecto com as coordenadas do centro do mapa
E obter (no meu exemplo):
Q {d: -2.527066, e: -44.298419999999965, toString: function, b: function, equals: function…}
    d: -2.527066
    e: -44.298419999999965
    __proto__: Q

Exemplo
Answer: Use a API do Google Maps e veincule uma função ao evento click:
google.maps.event.addListener(marker, 'click', function () {
  console.log('Coordenadas:', marker.getPosition());
});

Answer: Eu implementei algo bem parecido em uma sistema uma vez. Você pode dar uma olhada e copiar se quiser.
How-to
Como pelo visto você já tem o widget do Google Maps do seu site, basta adicionar um campo para que o usuário possa digitar o endereço que a pessoa passar pelo telefone e dois campos hidden que irão guardar os valores de latitude e longitude. O campo de endereço vai ser do tipo autocomplete, igual ao Google Maps mesmo, onde você digita o endereço e as opções vão aparecendo.
<input id="address" name="school.address" type="text" placeholder="Digite o endereço da escola..." autocomplete="off">

<input id="lat" type="hidden" value="0">
<input id="lng" type="hidden" value="0">

Depois, no trecho de código que você inicializa os objetos do Google Maps, você deve criar um objeto places.Autocomplete, pasando como parâmetro o input que você criou para digitar o endereço.
map = new google.maps.Map(..., ...);

var addressInput = document.getElementById('address');
var autocomplete = new google.maps.places.Autocomplete(addressInput);
autocomplete.bindTo('bounds', map);

Feito isso, você precisa "escutar" o evento place_changed do componente de autocomplete, de modo que sempre que o usuário digitar um novo endereço, você possar pegar as coordenadas. Além de pegar as coordenadas no endereço, quando o evento ocorrer o mapa será centralizado no endereço, bem como um marker será criado no local.
google.maps.event.addListener(autocomplete, 'place_changed', function() {
    // centraliza na endereço digitado e ajusta o zoom
    map.setCenter(autocomplete.getPlace().geometry.location);
    maxZoomService.getMaxZoomAtLatLng(autocomplete.getPlace().geometry.location, function(response) {
        map.setZoom(response.status == google.maps.MaxZoomStatus.OK ? parseInt(response.zoom * .85) : 16);
    });

    // adiciona uma marker para o endereço
    addrMarker.setPosition(autocomplete.getPlace().geometry.location);
    addrMarker.setVisible(true);

    // chama outra função para setar os valores os inputs
    fillGeolocFields(autocomplete.getPlace().geometry.location);
});

Finalmente, essa é a função que preenche os campos hidden de latitude e longitude:
function fillGeolocFields(location) {
    $('#lat').val(location.lat());
    $('#lng').val(location.lng());
}

Agora com os valores de latitude e longitude armazenados os inputs, você pode submeter seu formulário normalmente.
Você pode ler a documentação do Place Autocomplete nos links:
https://developers.google.com/maps/documentation/javascript/places?hl=pt-br#places_autocomplete
https://developers.google.com/maps/documentation/javascript/examples/places-autocomplete?hl=pt-br


What test do I use comparing pre and post intervention data?

Question: I am doing a health related project at at hospital where a policy was implemented for patients on certain therapy. Data was collected for the number of patients and days of therapy pre-implementation, then data was collected for the same interval of time post-implementation. How should I go about data analysis?
https://i.stack.imgur.com/EZt4m.jpg
Comment: It would be helpful if you posted your data, but if not, tell us what the outcome measure is and what other variables you wish to include in the analysis. Is the outcome measured twice for each patient?
Comment: Why don't you post your data to an excel file ( or a simulation of that )and we will try and help you. If you don't want to post your real data then simply code/scale it .
Comment: I recently uploaded a link to imgur with my data!
Comment: can't understand/read your link .
Comment: Would the pre intervention be my control group?
Answer: More detail would be helpful. Is days of therapy the outcome you are interested in examining? Are the actual patients the same pre- and post-intervention or are they different groups of patients? If the former, then you have a within-subjects design, and you could use ANOVA. If the groups are different then you'd probably want to consider something like a simple t-test to see whether the groups significantly differ on the outcome.


What units are in the ring $\mathbb{Z}_2 \times \mathbb{Z}$?

Question: I'm stuck on this question.
I know how to find the units of rings for example, $\mathbb{Z}_7$ and $\mathbb{Z}_{15}$.  I need to find elements that are relatively prime to $7$ and $15$. Having trouble with the product example.
Comment: Can you find at least one to get started?
Comment: First: what is the multiplicative unit element in $\;\Bbb Z_2\times\Bbb Z\;$ ? Next, can you find a non unity unit in this ring?
Answer: Taking group of units preserves products, that's
\begin{align}
((\Bbb Z/2\Bbb Z)\times\Bbb Z)^\times
&=(\Bbb Z/2\Bbb Z)^\times\times\Bbb Z^\times\\
&=\{1+2\Bbb Z\}\times\{\pm 1\}
\end{align}
hence the units of $(\Bbb Z/2\Bbb Z)\times\Bbb Z$ are $(1+2\Bbb Z,\pm 1)$.


Igualar variável php a uma variável javascript

Question: Estou a tentar fazer algo deste género:

Ou seja, quero dar o valor a uma variável php de uma caixa de texto por javascript. O valor da caixa de texto chega bem aqui, mas não estou a conseguir igualar.
Comment: Cara, não é possível executar código PHP dentro de código Javascript, primeiro porque nem compila, segundo, por que o javascript executa somente no browser, e PHP somente no servidor, para você criar uma comunicação entre o browser e o servidor é necessário primeiro, estabelecer uma comunicação via HttpXmlRequest ou Ajax através de um `POST`
Comment: Já que está usando jquery, use ajax para enviar/receber valores do php.
Comment: ok, pensava que poderia ser possível. obrigado na mesma!
Comment: Achei o tutorial abaixo, só não testei pra ver se funciona. http://www.mauricioprogramador.com.br/posts/passar-variavel-javascript-para-php
Answer: PHP e JavaScript funcionam em momentos diferentes. O PHP gera a página e a partir daí só existe HTML e JavaScript. Assim não é possível igualar variàveis que pertencem a mundos diferentes: PHP do lado do servidor e JavaScript do lado do cliente.
Existem porém duas maneiras para poder comunicar "entre mundos". Uma delas, demasiado defenitiva para o seu caso, é fazer um formulário e passar a informação com o refresh da página.
A alternativa que procura aqui é AJAX. Uma ligação/chamada ao lado do servidor onde pode passar dados e receber passados alguns milisegundos. Um exemplo seria assim:
$.ajax({
    type: "POST",
    url: "seuFicheiro.php",
    data: {nomeVariavel: 'valor variável',
    success: function (data) {
        // aqui pode usar o que o PHP retorna
    }
});

e no lado do PHP algo como:
$nome = $_POST['nomeVariavel'];
// correr outro código que precise...
echo $resposta;

Este echo é o que é passado para a função success do AJAX do lado do cliente.
Espero que ajude a compreender o mecanismo.
Answer: Isto Não é possível.
Você poderia fazer o javascript requisitar um AJAX que gravaria o valor na sessão, e na próxima execução você teria este valor.
Alerta de gambiarra!
Comment: ok, pensava que poderia ser possível. obrigado na mesma!
Answer: Usando Javascript inline você pode fazer algo assim:

Mas lembre que, dependendo das suas necessidades, usar AJAX é o mais apropriado, como foi dito nas outras respostas.
Veja esta pergunta feita no Stack-EN

Atualização
Agora percebi que você quer fazer o contrário, passar uma variável JS para PHP. Neste caso só com AJAX ou submit de um form mesmo.
Comment: O contexto pede o contrário...
Comment: @DanielOmine Agora que percebi, atualizei a resposta, valeu o aviso. =)


Variance and Mean Relationship from Simulated Poisson Process with Sampling

Question: Background: I have a simple simulation of a sampled thresholded Poisson Process that arrives at a closed form solution but need help with the proof.
In my example, I am simulating 10,000 silicon wafers each containing 500 integrated circuits of area 'A0' (mmsq). Defects land according to a Poisson process with a defect density 'd0' (defects/mmsq). Each integrated circuit calculates a random Poisson variable based on 'A0' and 'd0' and if it returns a value greater than zero, the integrated circuit is deemed a failure (i.e 1 or more defects deems the part as a failure). From here, I compute the mean pass/fail result (wafer yield) for each of the 10,000 wafers. I then compute the grand mean wafer yield of all 10,000 wafers as well as the variance of the wafer yield. Lastly, I repeat this approach for varying combinations of 'A0' and 'd0'.
Interestingly, if I plot Wafer Yield Variance (Y) vs. Wafer Yield Grand Mean (X) with all the combinations of 'A0' and 'd0', it precisely fits the curve:
Y = (1/500)X(1-X)
If I change the number of integrated circuits per wafer 'N', the more general fit equation holds:
Y = (1/N)X(1-X)
I know this has to do with the fact that this is essentially a sampling distribution but I am not quite able to put the proof together on how to arrive at this fit equation from the basic variance equation:
Var(x) = (1/N)*Sum(x-u)
Can anyone lend a hand?
Answer: Helpful commentary from this post:
https://www.quora.com/Why-is-the-formula-for-the-variance-of-a-sample-proportion-p-1-p-n
Basically, this whole simulation is sampling of a proportion where each wafer's yield is a proportion of successes.
The variance of each wafer yield is Var(X/N), where 'X' is a binomial variable. Var(X) = N*p(1-p) so Var(X/N) is p(1-p)/N. The expected value of 'p' is the wafer yield grand mean which gets us to the final relationship I was seeing.


Payoff function

Question: $$
\mathrm{d}S_t=μS_t\,\mathrm{d}t+σS_t\,\mathrm{d}B_t
$$
The payoff function for a european call is:
$$
f(S,T)=(S(T)-K)⁺.
$$
When I graph this it is obvious it is continuous. How would I mathematically prove that it is actually continuous? Would i need to establish pathwise continuity?
Answer: The function $x\mapsto (x-K)_+$ is continuous because it is $1$-Lipchitz. Indeed
$$ |(x-K)_+-(y-K)_+| \le |x-y| $$
since $|u_+-v_+|\le |u-v|$ for any $(u,v)\in \mathbb{R}^2$. The solution of you SDE is pathwise continuous since the unique solution is
$$ S_t = S_0 e^{(\mu-\sigma^2 /2)t + \sigma B_t}$$
Then $T \mapsto (S_T-K)_+$ is also pathwise continuous.
To be more precise if $(\Omega,\mathcal{F},\mathcal{P})$ is your probability space then it means that there exists $\Omega' \in \mathcal{F}$ with $P(\Omega')=1$ such that for all $\omega \in \Omega'$, the latter function $T \mapsto (S_T(\omega)-K)_+$ is continuous.
Comment: ok thanks, what about the discrete Asian option payoff, it is not as straightforward [(1/n)∑S(t{i})-K]⁺
Comment: Different problem? Then ask a different question!
Comment: @JohnnyByr : for the discrete Asian what is the function you want to be continuous ?

Payoff function


Clonando elemento modificado com jQuery

Question: Em uma tabela HTML de referência, a tag
 é acessada e tem seus dados modificados [não incluso aqui por estar funcionando], e ao usar o método .clone() do jQuery para criar um deep copy para a tabela de clone, o conteúdo de
 é clonado corretamente mas o valor que foi modificado na tabela de referência não é enviado para a tabela de clone, o que é clonado são os dados iniciais estáticos por default, como clonar o que também está modificado no DOM da tabela de referência para a tabela de clone, o que está faltando?
O jQuery me auxiliou na construção da função abaixo e funciona, mas como mencionei, não cria o deep copy com o DOM modificado.

        <input type="checkbox" name="check">

      Conteúdo default

Answer: Pelo código que postou, como pode ser visto no exemplo abaixo, a clonagem funciona. O que faltou aí é que depois que faz o clone vc deveria pegar o seu conteúdo com o método html(). Repare que eu altero o conteúdo do footer da tabela ref e este conteúdo novo é clonado para o footer na tabela clone normalmente:

$('table.ref input[type="checkbox"]').on('change', function() {
  $('table.ref tfoot tr th').text('Outro Conteúdo !!!');  //novo conteúdo
  let $cloningFoot = $('table.ref tfoot').clone();
  $('table.clone tfoot').html($cloningFoot.html()); //pega o conteúdo e insere
});

        <input type="checkbox" name="check">

      Conteúdo default

Neste exemplo é mostrado no console que a clonagem funciona:

$('table.ref input[type="checkbox"]').on('change', function() {
  $('table.ref tfoot tr th').text('Outro Conteúdo !!!');
  let $cloningFoot = $('table.ref tfoot').clone();
  $('table.clone tfoot').html($cloningFoot.html());

  console.log('Tabela ref: ', $('table.ref').html());
  console.log('Tabela clone: ', $('table.clone').html());
});

        <input type="checkbox" name="check">

      Conteúdo default

Clonando elemento modificado com jQuery


Consigo executar qualquer query no SQL Server sem passar as credenciais, isso é normal?

Question: Estava procurando reiniciar o serviço do SQL Server, foi quando descobri uma forma de manipular o banco sem logar no sistema. Veja:

Abri o Management Studio.

Cancelei a autenticação.

3.Acesso o menu de Exibição > Servidores Registrados. Repare que o Object Explorer está desconectado.

Minha instância do SQL Server rodando...

Nova Consulta...

Dropando o banco...

Enfim, isso é meio estranho, alguém sabe como resolver isso? Afinal se alguém tiver acesso à minha máquina...
Comment: mas voce está a usar credenciais, está a usar "Windows credencials", pois esse servidor local está com setup para usar Windows Credencials. No **Security** > **Login/Authentication**  desse servidor, diga que só quer por palavra-chave e remova o Windows Credentials.
Comment: @balexandre entendo, na verdade é realmente um servidor local, mas costumo usar o SA para logar, especialmente no Visual Studio...
De qualquer maneira, no Login tem vários usuários, no caso devo deletar quais  deles?
Answer: Sendo um servidor local que está usando as credenciais do seu usuário local, a resposta é sim, isto é normal.
Note, porém, que é normal para um ambiente de desenvolvimento usado por um programador ou, no máximo, uma equipe de desenvolvimento ou testes.
Num ambiente de produção, um administrador de banco de dados deverá configurar corretamente as permissões mínimas necessárias para sistemas e usuários que acessam o banco de dados.


How to deal with standard deviation using percentage values

Question: I have the following issue: I have calculated a mean value of customer engagement over a sample of 30 individuals. We present them with material to consume and calculate the engagement as (number of videos/texts clicked on)/(number of videos/texts sent).
My values are a mean of 88.9% of all content has been consumed, so on average, 88.9% of the content sent to the customer (videos, texts etc.) have been clicked on, but my sdev is +/- 28.9%. Now I am wondering
a) if I have an error in my calculations since in my head that does not make that much sense, you can't consume more than 100% of the content provided, and
b) how would I show this as a graph? Again, in my head it does not make sense to show error bars that go beyond 100%?
Thank you all for your guidance!
Comment: It would be useful to give a few more details about what your data looks like. For example, it is surprising to me that the standard deviation is so high. Specifically, even a single draw from a binomial distribution with $p = 0.889$ has a standard deviation of $0.31$, which is only a bit larger than the $0.289$ you report.
Comment: What is your sample size $n?$  You say your 'standard deviation` is $0.289,$ How did you obtain that? Maybe your're mixing up standard deviation, standard error, and margin of error. @stats_model (+1) asks for more details. // If you're using a Wald 95% CI  it is possible to get an interval with endpoints outside $(0,1),$ But my guess is you're doing something wrong, and details would help us make sense of this.
Comment: Why not use my Answer, to the extent it may be helpful, to provide more detail in your Question--and possibly suggest a solution. Then maybe your question can be re-opened.
Comment: Hi @BruceET, I have added the details that hopefully clear that issue up and qualify the question for re-opening. I have not used your answer since I am located in an European country and I was, quite frankly, asleep when you posted it.
Answer: Example: Suppose you have $n = 1000$ randomly chosen subjects of whom $x = 889$ are 'engaged.'
Then the estimated proportion engaged in the population is $p = x/n = 0.889.$
Wald confidence intervals are intended for use with such large samples. A 95% Wald CI is of the form $\hat p \pm 1.96\sqrt{\frac{\hat p(1-\hat p)}{n}}.$ Using R as a calculator,
we get the 95% CI $(0.870, 0.908),$ rounded to three places.
n = 1000;  x = 889
p.est = x/n;  p.est
[1] 0.889
se = sqrt(p.est*(1-p.est)/n);  se
[1] 0.00993373
ci = p.est + qnorm(c(.025,.975))*se;  ci
[1] 0.8695302 0.9084698

If this does not answer your question, please say what part of it
seems different from what is being done in your course.
Note: There are many styles of CIs for a binomial proportion
(google confidence interval binomial proportion; perhaps see the
Wikipedia article, in particular). One additional style, called the Jeffries CI, relies on a Bayesian derivation, but has very good properties as a
frequentist CI. For the data above the 95% Jeffreys CI is $(0.868, 0.907),$ easily computed in R as shown below. This style of confidence interval never gives endpoints
outside of $(0,1).$
n = 1000;  x = 889
qbeta(c(.025,.975), x+.5, n-x+.5)
[1] 0.8684107 0.9073418

Comment: @P.Weyh Neither the standard deviation of a bunch of numbers, nor the standard error of the mean, impose any kind of restriction on what values can realistically be included in a confidence interval (or a prediction interval for that matter). Therefore, you can't state that 29 percentage points is in any way unrealistic. Just forget about the number 29 and try this alternative approach. The answer by BruceET gives you a better way to construct an interval based on your type of data. The second part even shows an alternative that restricts the interval to $(0, 1)$, ensuring realistic values.
Comment: Hi @BruceET, first of all, thank you for the answer. It is, however, not something that I am doing in a course but at work.

We have 30 customers that have, on average, consumed 89% of the material that we have provided them with, so 11% of the materials have been ignored or not clicked on. From that average, there has been a 29% sdev. Hope that clears it up
Comment: @FransRodenburg thank you for your comment. I am not saying that they are, I would like to know if they are, and am hoping to get an idea on how to interpret a sdev of ">100%" and how to go about it if I have a sample size of 30, which I added after edit. And I unfortunately do not know which interval would work for me, since I am not well versed in statistics. Therefore what I was hoping to get was an insight in how I can find out which interval to use, and what data I need to provide for someone to tell me which interval to use. Thank you for your help.


interpretation for non-normal boxplot

Question: I'm trying to understand this boxplot, where I'm rejecting null hypothesis, but it amazes me how can it reject it since it looks so similar, i.e. equal for the variable.
Does it mean if their medians are the same the hypothesis will also say it is the same?
Comment: I don't think boxplots are very helpful with discrete variables only taking a few distinct values. Its not hard to see that the second boxplot often  takes higher values than the first
Comment: The Mann-Whitney test is only a test of medians if the shape of the distributions is the same. Clearly, this is not the case here.
Comment: It might help some answerers illustrate why there seems to be a location difference if you were to tabulate the counts in each outcome value for each group (it's just 8 counts); it's hard to do from the plot.
Comment: When values are just 1, 2, 3, 4 medians and quartiles are usually one of these values or exceptionally half-way between two of them. That is not helpful. Paradoxically, or otherwise, when values are so coarse the most informative summary may be based on means instead.
Answer: The Mann-Whitney test is not in general a test of medians.  Instead, it is a test of whether the values in one group tend to be higher than the other group.
In your data Group 2 has few observations of "1" or "2", and relatively many of "3" and "4", when compared with Group 1.  This is formalized with the rank biserial coefficient and with the Mann-Whitney test.
Instead, if you want to compare the medians, you can use a test of medians.
A useful way to present this data is with a bar plot of the response categories ("1", "2", "3", "4") for each of the Groups.  A few possibilities are in links below.
i1.wp.com/statistical-research.com/wp-content/uploads/2012/12/groupedbar.likert.png
rcompanion.org/handbook/images/image061.png
rcompanion.org/handbook/images/image057.png

interpretation for non-normal boxplot


The meaning of the word "inversion"

Question: I'm looking for clarity over how to best describe two different meanings of the word "inversion" I have come across. The two different aspects I see are:

I have three notes C-E-G. The intervals between them are a major third followed by a minor third. By using octave equivalence this can be inverted to E-G-C (minor third followed by perfect fourth) and G-C-E (perfect fourth followed by major third). Other voicings of these inversions C-G-E, E-C-G and G-E-C also follow on from octave equivalence.
I have three notes C-E-G. The intervals between them are +4 semitones C-E and +3 semitones E-G. I can "invert" this by starting at C, going down 4 semitones to Ab, then going down 3 semitones to F. This gives me C-A♭-F.

The first meaning appears to be common usage. The second definition of inversion is the one used in pitch class set theory. They appear distinctly different to me and I think its confusing to use the same word to describe them.
I would like to know if there is another word or term I can use to differentiate between these two aspects?
Answer: Yeah, this can be very confusing. First of all, I would clarify that the standard understanding of the first type of inversion is too restricted. I can invert a root-position C-major chord by either putting the third in the bass, or the 5th in the bass. It doesn't have to be literally a m3-P4 to be a 1st inversion CM chord, it just has to be the notes of a CM chord, with E as the lowest pitch. Type 1 inversion is purely concerned with which member of the triad is lowest regardless of voicing, doubling, etc; the voicing and number of notes could be identical except for the bass note, and that would still be an inversion. The only intervals necessarily inverted, are the possible intervals above the bass.
I think the primary semantic confusion is removed as long as you refer to "1st inversion", "second inversion", "inversions of a CM chord", etc. That terminology doesn't exist in set theory, where we would call your example a T0I inversion of [0,4,7] (apologies if you learned a different notation for this, I'm following Straus' use in his Post-Tonal textbook). Since set class terminology is entirely octave- and inversionally-equivalent, there is no distinctions about what note is on bottom.
I've heard people refer to your type 1 as "bass inversion" and your type 2 as "pitch inversion", "mirror inversion" or "axis inversion," but I'm not sure how widespread that terminology is. Straus' solution is to refer to type 1 without the word inversion at all, he just calls the three positions of a triad "bass positions".
But remember, your type 2 is the most obvious example of pitch inversion—around a literal PC0 axis—but the mapping shows that it is less dissimilar from chord inversion than it might seem. If I move a CM chord from root position to second inversion, one way to visualize it is that the G moves from being above the C and E to being below (that's just one way to do it, but bear with me...). It was a P5 (pi7) above C and becomes a P4 (pi5) below C. It was a m3 (pi3) above E and becomes a M6 (pi9) below E. If you do T0I on [0,4,7], than any Gs that are pi7 above C become pi5 above C (F) and any Gs that are pi3 above E become pi9 above C (Ab). In both cases, you can see pi7s becoming pi5s and pi3s becoming pi9s, it's just a slightly more abstract relationship in the case of (037).
Type 1 inversion must always have the same PC content, Type 2 inversion almost always (except for a handful of inversionally-symmetric sets at just the right TnI values) have a least some different PCs, and often are an entirely new collection.
Comment: It does make sense. I was aiming to be concise so I avoided using the phrase "1st inversion", "2nd inversion" etc.I like the term "mirror inversion"; it seems to capture the notion that a pi7 _above_ becomes a pi7 _below_ etc. In some sense type 1 is more like 'rotation' or 'permutation' ... Both are of course equivalent when you only have two pitches.
Comment: "Rotation" and "permutation" are great terms for it, I'm going to start using that in post-tonal classes! But I don't think they're equivalent even with two pitches (except for the tritone). Rotating changes E–C to C–E while inverting changes E–C to C–Ab.
Comment: Good point. I was adding in a transposition on top. I was getting at the fact that E-C and C-A♭ are still both a pi8.
Comment: BTW I added extra cases to type 1, so I think the your comment on my definition being restricted probably does not hold any more? Maybe needs a light correction.
Comment: Ah, I hadn't seen that. But still, those other cases aren't additional inversions, just different voicings of the same inversions already mentioned. It's only a semantic distinction, but I think it actually gets right to the heart of the difference between the two "inversions". Type 1 inversion is purely concerned with which member of the triad is lowest regardless of voicing, doubling, etc.
Comment: Interesting. I've edited the question to say 'voicing' instead of 'inversion' for the additional type 1 examples. Your comment above ("Type 1 inversion is purely concerned with which member of the triad is lowest regardless of voicing, doubling, etc") is a useful clarification - could you add this to your answer?
Comment: OK, I added that, plus some additional clarification in the first paragraph and at the very end of the answer.
Comment: I use the term "chromatic inversion" for the second case, but it is not widely adopted. A more precise way to consider the difference might be to consider that the first case refers to reordering of a simultaneous collection of pitch classes, so I propose "pitch class" inversion (especially since the octave of any pitch besides the lowest is irrelevant), whereas the second form is more commonly used as an operation of motivic elaboration on individual pitches, imagined in relative succession even if realized in a simultaneity, so it could be termed "pitch".
Comment: Also, "rotation" is a great clarification I had meant to include on a new line instead of a new comment.

The meaning of the word "inversion"


Published React Native app pulls changes from private repository

Question: I've encountered a very mysterious and baffling phenomenon in the process of publishing a React Native app to both the Apple App Store and the Google Play Store.  It seems like witchcraft.
The app, which we have already published on both stores, updates every time we push changes to our private repository.
EDIT: The app, which we have already published on both stores, updates SOME OF THE TIME (not every time) we push changes to our private repository.
For example, let's say the app started out with page A and page B.  We publish the app, we'll call it "Witchcraft", to both app stores.  We then push an update to our private repository (on BitBucket) that includes the addition of page C.  The next day, someone (outside of the company) downloads our app from one of the app stores (without cloning our repository), and their install of "Witchcraft" includes page C.
What in the world is going on?
Here are some details:

The app was developed in React Native.
We used Expo for development and testing of the app.
We host our repository using BitBucket.
We are NOT developing in the master branch.
New users, when they install the app from either play store, seem to see ALMOST all the changes we've pushed to the repository, BUT...
There is one property of the repository that users do NOT acquire in their fresh installation - their installation correctly points to the production database (for retrieving user info, etc) instead of the development database, which is what is referenced in what is pushed to the repository.
We have NOT updated the builds on the app stores, so we do not expect these changes to appear in the app store builds.

We are very concerned because we do not want breaking changes being reflected in the production app.
We haven't the slightest idea what is going on.  Any insight would be appreciated.
Answer: It may be possible that you have OTA updates enabled, in your app.json add the following:
{
  "updates": {
    "enabled": false,
  }
}

Also check if at any point of the process you are doing expo publish maybe in git hook?


How to render rander rmarkdown file to markdown file with backtick styled code block

Question: I have a test.Rmd file.
```{r}
1 + 1
````

I render it to a markdown file with command rmarkdown::render("test.Rmd", rmarkdown::md_document()). The resulting code block is four space style. I want the backstick style.
```r
1 + 1
```

Answer: You can use the YAML header block to turn on Pandoc's backtick_code_blocks extension:
---
title: "Markdown file"
output:
    md_document:
        variant: markdown+backtick_code_blocks
---

Comment: Following your answer, I get what I want using the command `rmarkdown::render("test.Rmd", rmarkdown::md_document(variant = "markdown_strict+backtick_code_blocks"))`.


Load avg weirdness on Linux Ubuntu

Question: In past few days I've been trying to understand the weirdness which is happening in our infrastructure but I haven't been able to figure it our so I'm turning to you guys to give me some hints.
I've been noticing in Graphite, spikes in load_avg which are happening with deadly regularity approximately every 2 hours - it's not exactly 2 hours but it is very regular. I'm attaching a screenshot of this I took from Graphite

I've got stuck in investigating this - the regularity of this was leading me to thinking that it's some kind of cron job or something like that but there are NO cronjobs running on these servers - actually these are VMs running in Rackspace cloud. What I'm looking for is some kind of indication of might be causing these issues and how to investigate this further.
The servers are fairly idle - this is a staging environment so there's almost not traffic coming in/there should be no load on them. These are all 4 virtual cores VMs. What I know for sure is that we are taking a bunch of Graphite samples about every 10 seconds but if that's the cause of the load then I'd expect it to be constantly high rather than happening every 2 hours in waves in different servers.
Any help how to investigate this would be greatly appreciated!

Here are some data from sar for app01 - which is the first blue spike on the picture above - I wasnt able to draw ANY conclusions from the data. Also not that the bytes write spike you see happening every half an hour (NOT EVERY 2 HOURS) is due to chef-client running every 30 minutes. I'll try to gather more data even though I already have done that but couldn't really draw any conclusions from those either.
LOAD
09:55:01 PM   runq-sz  plist-sz   ldavg-1   ldavg-5  ldavg-15   blocked
10:05:01 PM         0       125      1.28      1.26      0.86         0
10:15:01 PM         0       125      0.71      1.08      0.98         0
10:25:01 PM         0       125      4.10      3.59      2.23         0
10:35:01 PM         0       125      0.43      0.94      1.46         3
10:45:01 PM         0       125      0.25      0.45      0.96         0
10:55:01 PM         0       125      0.15      0.27      0.63         0
11:05:01 PM         0       125      0.48      0.33      0.47         0
11:15:01 PM         0       125      0.07      0.28      0.40         0
11:25:01 PM         0       125      0.46      0.32      0.34         0
11:35:01 PM         2       130      0.38      0.47      0.42         0
11:45:01 PM         2       131      0.29      0.40      0.38         0
11:55:01 PM         2       131      0.47      0.53      0.46         0
11:59:01 PM         2       131      0.66      0.70      0.55         0
12:00:01 AM         2       131      0.81      0.74      0.57         0

CPU
09:55:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
10:05:01 PM     all      5.68      0.00      3.07      0.04      0.11     91.10
10:15:01 PM     all      5.01      0.00      1.70      0.01      0.07     93.21
10:25:01 PM     all      5.06      0.00      1.74      0.02      0.08     93.11
10:35:01 PM     all      5.74      0.00      2.95      0.06      0.13     91.12
10:45:01 PM     all      5.05      0.00      1.76      0.02      0.06     93.10
10:55:01 PM     all      5.02      0.00      1.73      0.02      0.09     93.13
11:05:01 PM     all      5.52      0.00      2.74      0.05      0.08     91.61
11:15:01 PM     all      4.98      0.00      1.76      0.01      0.08     93.17
11:25:01 PM     all      4.99      0.00      1.75      0.01      0.06     93.19
11:35:01 PM     all      5.45      0.00      2.70      0.04      0.05     91.76
11:45:01 PM     all      5.00      0.00      1.71      0.01      0.05     93.23
11:55:01 PM     all      5.02      0.00      1.72      0.01      0.06     93.19
11:59:01 PM     all      5.03      0.00      1.74      0.01      0.06     93.16
12:00:01 AM     all      4.91      0.00      1.68      0.01      0.08     93.33

IO
09:55:01 PM       tps      rtps      wtps   bread/s   bwrtn/s
10:05:01 PM      8.88      0.15      8.72      1.21    422.38
10:15:01 PM      1.49      0.00      1.49      0.00     28.48
10:25:01 PM      1.54      0.00      1.54      0.03     29.61
10:35:01 PM      8.35      0.04      8.31      0.32    411.71
10:45:01 PM      1.58      0.00      1.58      0.00     30.04
10:55:01 PM      1.52      0.00      1.52      0.00     28.36
11:05:01 PM      8.32      0.01      8.31      0.08    410.30
11:15:01 PM      1.54      0.01      1.52      0.43     29.07
11:25:01 PM      1.47      0.00      1.47      0.00     28.39
11:35:01 PM      8.28      0.00      8.28      0.00    410.97
11:45:01 PM      1.49      0.00      1.49      0.00     28.35
11:55:01 PM      1.46      0.00      1.46      0.00     27.93
11:59:01 PM      1.35      0.00      1.35      0.00     26.83
12:00:01 AM      1.60      0.00      1.60      0.00     29.87

NETWORK:
10:25:01 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s
10:35:01 PM        lo      8.36      8.36      2.18      2.18      0.00      0.00      0.00
10:35:01 PM      eth1      7.07      4.77      5.24      2.42      0.00      0.00      0.00
10:35:01 PM      eth0      2.30      1.99      0.24      0.51      0.00      0.00      0.00
10:45:01 PM        lo      8.35      8.35      2.18      2.18      0.00      0.00      0.00
10:45:01 PM      eth1      3.69      3.45      0.65      2.22      0.00      0.00      0.00
10:45:01 PM      eth0      1.50      1.33      0.15      0.36      0.00      0.00      0.00
10:55:01 PM        lo      8.36      8.36      2.18      2.18      0.00      0.00      0.00
10:55:01 PM      eth1      3.66      3.40      0.64      2.19      0.00      0.00      0.00
10:55:01 PM      eth0      0.79      0.87      0.08      0.29      0.00      0.00      0.00
11:05:01 PM        lo      8.36      8.36      2.18      2.18      0.00      0.00      0.00
11:05:01 PM      eth1      7.29      4.73      5.25      2.41      0.00      0.00      0.00
11:05:01 PM      eth0      0.82      0.89      0.09      0.29      0.00      0.00      0.00
11:15:01 PM        lo      8.34      8.34      2.18      2.18      0.00      0.00      0.00
11:15:01 PM      eth1      3.67      3.30      0.64      2.19      0.00      0.00      0.00
11:15:01 PM      eth0      1.27      1.21      0.11      0.34      0.00      0.00      0.00
11:25:01 PM        lo      8.32      8.32      2.18      2.18      0.00      0.00      0.00
11:25:01 PM      eth1      3.43      3.35      0.63      2.20      0.00      0.00      0.00
11:25:01 PM      eth0      1.13      1.09      0.10      0.32      0.00      0.00      0.00
11:35:01 PM        lo      8.36      8.36      2.18      2.18      0.00      0.00      0.00
11:35:01 PM      eth1      7.16      4.68      5.25      2.40      0.00      0.00      0.00
11:35:01 PM      eth0      1.15      1.12      0.11      0.32      0.00      0.00      0.00
11:45:01 PM        lo      8.37      8.37      2.18      2.18      0.00      0.00      0.00
11:45:01 PM      eth1      3.71      3.51      0.65      2.20      0.00      0.00      0.00
11:45:01 PM      eth0      0.75      0.86      0.08      0.29      0.00      0.00      0.00
11:55:01 PM        lo      8.30      8.30      2.18      2.18      0.00      0.00      0.00
11:55:01 PM      eth1      3.65      3.37      0.64      2.20      0.00      0.00      0.00
11:55:01 PM      eth0      0.74      0.84      0.08      0.28      0.00      0.00      0.00

For people curious about cronjobs. Here's the summary of all cronjobs set up on the server ( I picked app01 but this is happening on a few other servers too with the same cronjobs set up)
$ ls -ltr /etc/cron*
-rw-r--r-- 1 root root  722 Apr  2  2012 /etc/crontab

/etc/cron.monthly:
total 0

/etc/cron.hourly:
total 0

/etc/cron.weekly:
total 8
-rwxr-xr-x 1 root root 730 Dec 31  2011 apt-xapian-index
-rwxr-xr-x 1 root root 907 Mar 31  2012 man-db

/etc/cron.daily:
total 68
-rwxr-xr-x 1 root root  2417 Jul  1  2011 popularity-contest
-rwxr-xr-x 1 root root   606 Aug 17  2011 mlocate
-rwxr-xr-x 1 root root   372 Oct  4  2011 logrotate
-rwxr-xr-x 1 root root   469 Dec 16  2011 sysstat
-rwxr-xr-x 1 root root   314 Mar 30  2012 aptitude
-rwxr-xr-x 1 root root   502 Mar 31  2012 bsdmainutils
-rwxr-xr-x 1 root root  1365 Mar 31  2012 man-db
-rwxr-xr-x 1 root root  2947 Apr  2  2012 standard
-rwxr-xr-x 1 root root   249 Apr  9  2012 passwd
-rwxr-xr-x 1 root root   219 Apr 10  2012 apport
-rwxr-xr-x 1 root root   256 Apr 12  2012 dpkg
-rwxr-xr-x 1 root root   214 Apr 20  2012 update-notifier-common
-rwxr-xr-x 1 root root 15399 Apr 20  2012 apt
-rwxr-xr-x 1 root root  1154 Jun  5  2012 ntp

/etc/cron.d:
total 4
-rw-r--r-- 1 root root 395 Jan  6 18:27 sysstat
$ sudo ls -ltr /var/spool/cron/crontabs
total 0
$

As you can see there are no HOURLY cronjobs. Only daily/weekly etc.
I have gathered bunch of stats (vmstat, mpstat, iostat) - however hard I'm trying I just can't see any leads which would suggest any VM component misbehaving...I'm starting to lean towards potential problems at hypervisor. Feel free to have a look at the stats The gist starts with sar -q output around the "offending" time and then you can see vm,mp and iostats....
Basically it's still a total mystery to me ...
Comment: Do you have any interactive data you can share to investigate further (i.e., what do 'top', 'htop' and 'iotop' display during the recurring load spikes)?  Also, have your checked your application logs during the times in question to see if they are exhibiting any odd behavior?  Additionally, do you have any hosts with similar configurations not hosted on public cloud infrastructure, and if so, are they exhibiting similar behavior?
Comment: In terms of app logs - there is nothing happening. The only log entries it contains are checks from monitoring which are happening every minute - basically the monitoring system hits the main site and reports the result code - apart from that logs are completely empty.
Also as you can see there is a variety of hosts above - this is happening on all of them (redis, app server, chef server etc.)
Comment: you assume regularity, but the data you show don't show any regularly happening spikes.. please be more specific as to the exact period it's showing regularity at (over several days perhaps? on the picture, there is no regularity.)  you could run a "top -n 1" every 1mn or so and keep them in a file, and this could help see what other processes are competing for cpu at the same time a spike occurs. If App1 is a internet facing app, maybe it's just someone getting access to it and forcing that behavior? add a regular "netstat -an" logging too (every minutes?)
Comment: Did you see the attached screenshot ? If that doesn't show regularitu I don't know what does. I've now increased sampling period for sar so I'm sampling every 5 minutes. The regularity on the pic is more than obvious - its happening every two hours. This is a staging environment with NO traffic at all - as you surely can see from the above sar outputs for network stats.
Comment: Not an answer, but we have several misbehaving vmware hosts that exhibit similar symptoms.  Our perf team identified the offending component as the storage subsystem.  In our case that is a SAN.
Comment: Have you tried using [psacct](http://www.cyberciti.biz/tips/howto-log-user-activity-using-process-accounting.html) to narrow it down?
Comment: It's definitely not crontab: I have now removed ALL cronjobs from all servers - besides that, the ones which were in crontab were just running cronjobs from /etc/cron.daily(weekly|monthly).
Comment: Some things stick out a bit here. Your load goes up, but the number of processes you have does not change, nor does your runq sz change from zero -- granted this is an average value. Additionally there is no IO changes showing. Can you provide context switching stats on your host and if possible (usually not logged in SAR) IRQ stats?
Comment: @MIfe thats exactly the same thinking as I used when investigating - all apart form IRQs and context switches. I'll gather those and post them here as soon I'll have them.
Comment: @gyre sar -w shows context switching. sar -B might be useful for pagefaults too.
Comment: actually...vmstat contains context switches in 12th column and interrupts in 11th
Comment: Server load shot up to 4.5+ at one point, but cpu idle remain >90% even in 10sec monitoring interval over +/-5min of that specific time(08:50-08:55 PM). That does not make sense at all.
Comment: Unless there is a bunch of short lived processes ? I'm just thinking...
Comment: Assuming [this](http://serverfault.com/a/429332/138643) is correct, those numbers are average of the time interval, which is 10sec.
Comment: Those logs had the sampling time of 10 seconds. Also the Graphite output above has a sampling time of 10 second too.
Comment: @gyre Yes, that is the puzzling part. The number from vmstat and mpstat are matching each other, but neither show anything close to a 1+ load. On a 4-cpu system, `load > 4` should be matched with entries having `%idle=0` (or single digit %idle), but we are seeing `%idle=97` with `%wa=0` instead.
Comment: @gyre Any chance your server/application in a similar situation to this question: http://serverfault.com/a/365700/138643 ? Do you have a log for network connections in the same timeframe?
Comment: This is a very interesting insight @JohnSiu , I'll try to gather some information on this. Basically the post is suggesting that the Send-Q is going over the moon ?  Unfortunately I don't have the network stats but it's easy to collect them as this is still happening :)
Comment: In your gist.github.com file, at 21:00:07, right after load spike, the xvda await shoot to the roof (59.43), is that behaviour consistent(always happen right after load spike)?
Comment: let us [continue this discussion in chat](http://chat.stackexchange.com/rooms/7067/discussion-between-gyre-and-john-siu)
Comment: I will take a look into this again. But the gist doesn't point to anything that pops out immediately. BTW, do you know what profiling tools are available in the distro. I mainly work on RHEL, so a naive question here. Particularly, I would like to see something like perf or oprofile.
Comment: Wondering if there is any progress on this problem.
Answer: Interesting.
Firstly, can you increase the frequency of sar logging. In stead of 10 minutes, try to log every minute. The sysstat cronjob is configurable.
Next, try to script out the following commands.
ps auxf > /tmp/ps.out
vmstat 1 50 > /tmp/vm.out
mpstat -P ALL 1 50 > /tmp/mp.out
iostat -xdk 1 50 > /tmp/io.out
cat /proc/meminfo > /tmp/meminfo.out

Gather this set of data either at every iteration when the load average increases manually or through cron. It would be good to have data of at least one full working day.
Now, I understand that the servers are idle but still some applications must be running. What are they?
Is it possible that you can run some profiling tool, like perf or oprofile.
Has any server hardware component being changed? Even something as innocuous as a firmware upgrade or software upgrade.
Hey, one question. What is the scheduler you are running. I believe it is cfq, any chance you can change it to noop. Put elevator=noop in the kernel command line parameter and reboot the system and see whether it improves it.
Comment: I added a small edit about scheduler. please see the result.
Answer: Log Top Processes
Since the occurrence is very regular, setup cron job to monitor top processes during those period
#app01
20-59 0/2 * * * root /usr/bin/top -b -n 1 | /usr/bin/head -n 15 >> /var/log/top.log

Change 20-59 to * will log the whole hour for every even number hours. Cron job will run once per minute in either case.
You may want to add the top.log file to log rotation so it will not take up all space in case you forget to disable it.
Check Log file
Search log file entries at high load period
Take following load entry as example
10:25:01 PM         0       125      4.10      3.59      2.23         0

Do
grep ' 22:2' /var/log/*
grep ' 22:2' /var/log/apache2/*

That will show all log entries for 22:2x:xx. May have to include other log directories.
Sun Jan  6 21:00:07 2013: xvda w_await spike
xvda Chart - The w_await spike is at Sun Jan  6 21:00:07 2013
Answer: One thing I would definitely check:

vSphere graphs for the same pattern, maybe another VM on the same host is eating the CPUs (thus the load on your VM increases as it takes more time to process the same amount of data with a constant flow due to less CPU time being available for your VM).

Edit: Didn't get it the first time :) You're running on Rackspace, so no control on the hypervisor, yet it might be worth to ask rackspace if they could check if this pattern is common on other VMs on the same host.
Comment: I am suspicious of that too - it wouldn't be the first time Rackspace cloud would be causing some kind of madness. I doubt they monitor any of their hypervisor servers though - I mean in terms of misbehaving VMs, however I'd like to rule out any "internal" possibilities before I turn to the last resort - Rackspace support.
Comment: Would hypervisor performance affect a VM's self-apparent load-average?  This leads me to think about how the load average is calculated.  Could this possibly be some effect of green/power-saving feature periodically shifting work around to fewer cores unbeknownst to the OS?  Or how about dynamically changing clock-rate based on e.g. environmental inputs?
Comment: Load average is calculated by the scheduling algorithm, in simple words, if you have 100 tasks in your processing queue and the hypervisor is 100% efficient at executing 10 tasks per 1 seconds, then you need 10 seconds to execute 100 tasks, if your hypervisor is only 50% efficient (maybe CPU overpprovisioning) it will take 20 seconds to execute the same amount of tasks, thus leading to increased load. Complete explanation: http://blog.scoutapp.com/articles/2009/07/31/understanding-load-averages

Load avg weirdness on Linux Ubuntu


Question about defining sequence via transfinite induction.

Question: I have sequence $A_i$ where $i\in\mathbb{N}$ and every $A_i$ is countable.
Let $A=\bigcup_{i\in\mathbb{N}}A_i$.
I want to find new sequence of countable subsets of $A$ such that $C_i\subseteq A_i$ and $\forall n\neq m:C_n\cap C_m=\emptyset$.
I do it that way. First, by Zelmero theorem, I take $(A,\leq)$ to be well-ordered set.
Now I define a injective sequence ($B_i$) of elements of $A$ such that:
the smallest $b_1\in A$ such that $b_1\in A_1$
the smallest $b_2\in A\setminus\{b_1\}$ such that $b_2\in A_1$
the smallest $b_3\in A\setminus\{b_1,b_2\}$ such that $b_3\in A_2$
the smallest $b_4\in A\setminus\{b_1,b_2,b_3\}$ such that $b_4\in A_1$
the smallest $b_5\in A\setminus\{b_1,b_2,b_3,b_4\}$ such that $b_5\in A_2$
the smallest $b_6\in A\setminus\{b_1,b_2,b_3,b_4,b_5\}$ such that $b_6\in A_3$
$\dots$
Although not formally defined such sequence exist. Now let $d_i$ be fractal sequence. Then we take order isomorphism $f$, such that $f(d_i)=b_i$ and take $C_n=\{f(d_i):d_i=n\}$. Now I believe we have found sequence we were looking for.
Is such reasoning correct?

Possibly improved answer
Let $A=\bigcup_{i\in\mathbb{N}}A_i$. And let $(A,\leq)$ be well ordered set.
$B=\Sigma_{n=0}^\infty n$ where addition is meant as defined for ordinal numbers.
Now $B\subset\omega\times\omega$
Let's define order isomorphism $f:\omega\to B$ and projection $h:B\to\omega$ such that $h(x,y)=y$. Finally let's take $b_n=g\circ f(n)$
If everything is correct, then $b_n$ is what I called fractal sequence earlier (but now starting with 0). So $b_0=0,b_1=0,b_2=1,b_3=0,b_4=1,b_5=2,b_6=0,b_7=1\dots$
Now let's define another injective sequence:
$d_1=\min_{(A,\leq)}\{x:x\in A_{b_1}\}$
$\dots$
$d_n=\min_{(A,\leq)}\{x:x\in A_{b_n}$ and $\forall m<b_n: x\neq b_m\}$
Finally, let's define sequence of subsets of elements of sequence $A_i$
$$
C_n=\{d_k:k\in h^{-1}(n)\}
$$
Comment: What is a "doubly fractal sequence"?
Comment: @NoahSchweber,  it is fractal sequence. I've read somewhere that is also called doubly fractal and it sticked with me.
Comment: When you say "fractal sequence," do you mean [this](https://en.wikipedia.org/wiki/Fractal_sequence)? It's not a term I've seen used commonly ...
Comment: @NoahSchweber, yeah. What is more common term? That's the place I found this name.
Comment: @NoahSchweber, actually it's not fractal sequence, but only example of it 1,1,2,1,2,3,1,2,3,4,.... so it would be useful to have name for it.
Answer: As written, I neither see how this would yield a well-defined sequence (the crux lies in "...") nor do I understand what $f$ (and thus - by extension - $C_n$) is supposed to be.
It seems that you are looking for a sequence $(C_n \mid n \in \mathbb N)$ of pairwise disjoint sets such that $C_n \subseteq A_n$ and such that $A = \bigcup_{n \in \mathbb N} C_n$. This is easily arranged:
Recursively define

$C_0 := A_0$ (or begin with $C_1 := A_1$ in case you don't consider $0$ to be a natural number) and
$C_{n+1} := A_{n+1} \setminus \bigcup_{i \le n} A_i$.

Clearly $C_n \subseteq A_n$ and thus $\bigcup_{n \in \mathbb N} C_n \subseteq A$. It's also easy to see that the sequence $(C_n \mid n \in \mathbb N)$ is pairwise disjoint. So let me show that $A \subseteq \bigcup_{n \in \mathbb N} C_n$:
Fix $x \in A$ and let $n \in \mathbb N$ be minimal such that $x \in A_n$. Then $x \not \in \bigcup_{i \le n - 1} A_i$ and thus $x \in C_n \subseteq \bigcup_{m \in \mathbb N} C_m$.
Comment: I need $C_n$ to be countable subsets. I've improved my answer.
Comment: @SekstusEmpiryk $C_n$ is countable, since $C_n \subseteq A_n$ and $A_n$ is countable.


Scores of six soccer matches

Question: In the first round of the city soccer tournament, the teams in group A finished as follows:
<code>Team        Goals For           Goals Against           Points
Naranja     4                   2                       7
Bleu        5                   2                       6
Midori      1                   3                       3
Gelb        1                   4                       1
</code>
A victory earns three points, a tie one point, and a loss no points. Each team played the other three once. What were the scores of all the matches?
What I'm doing is making different combinations and doing guess and check. Is there a simpler way to do this (mathematical way)?
Answer: First, figure out how many games each team won, lost, and tied, based only on their total points (the third column). For example, Naranja has 7 points.  How can you make 7 points by adding up 0's, 1's, and 3's?
Then use that, plus the number of goals scored, to deduce the goals for / goals against for each team's three games.
Then pair up the teams that were in matches with the same scores.
Answer: This can be done pretty systematically. Since N has 7 points, they must have won twice and tied once. Since B has 6 points, they must have won twice and lost once. Since G has one point, they must have tied once and lost twice. As for M with three points, they could have either tied thrice or won once and lost twice. Since B never tied any game, M cannot have tied thrice. So, we know:

N: WWT
B: WWL
M: WLL
G: TLL

Since only one tie occurred, N and G must have tied, so N beat B and M. This accounts for B's single loss, so B beat M and G. This accounts for M's two losses, so M beat G.
As for scores, we begin by noting that M only has one goal for, so it must have been during their winning game against G: M 1-0 G. Since M has three goals against, their other two games were 0-1 and 0-2, but we don't yet know which against whom. G's tie with N was either 0-0 or 1-1, and G's loss against B was either 1-3 or 0-2. Summarizing, we have four cases:

M 0-1 N and M 0-2 B and G 0-0 N and G 1-3 B
M 0-2 N and M 0-1 B and G 0-0 N and G 1-3 B
M 0-1 N and M 0-2 B and G 1-1 N and G 0-2 B
M 0-2 N and M 0-1 B and G 1-1 N and G 0-2 B

Case 1.
B scored 5 goals against G and M, hence no goals on N. But N ceded 2 goals from B because it shut out M and G. Contradiction!
Case 2.
Similarly, G and M scored 0 against N, so N suffered 2 goals from B.
But B scored 4 against G and M, so it scored 1 on N. Contradiction!
Case 3.
B scored 4 goals against G and M, hence 1 goal on N.
B allowed no goals from G and M, so it allowed 2 from N.
Conversely, N scored 2 goals against G and M, hence 2 goals on B.
And N lost one goal to G, so it lost the other to B.
Consistent!
Case 4.
B scored 3 goals against G and M, hence 2 goals on N.
But N let in a goal against G and M, so it must have let only 1 in against B.
Contradiction!
Thus, there is one solution:

N 2-1 B
N 1-0 M
N 1-1 G
B 2-0 M
B 2-0 G
M 1-0 G.

And indeed, we see that N has 4 for, 2 against; B has 5 for, 2 against; M has 1 for, 3 against, and G has 1 for, 4 against.

Scores of six soccer matches


Allegro Debug -Abort() Has Been Called

Question: Eu tentei fazer uma animação e o Visual Studio dá o erro Debug -Abort() Has Been Called.
Código:
#include <allegro5\allegro.h>
#include <allegro5\allegro_native_dialog.h>
#include <allegro5/allegro_image.h>

const int WIDTH = 1000;
const int HEIGHT = 600;
bool exit = false;

const int maxFrame = 12;
int curFrame = 0;
int frameCount = 0;
int frameDelay = 5;

int main()
{
    ALLEGRO_DISPLAY *display = NULL;
    ALLEGRO_EVENT_QUEUE *event_queue = NULL;
    ALLEGRO_TIMER *timer;
    ALLEGRO_BITMAP *image[maxFrame];

    if(!al_init())
    {
        return -1;
    }

    display = al_create_display(WIDTH, HEIGHT);

    if(!display)
    {
        return -1;
    }

    al_install_keyboard;
    al_init_image_addon;

    image[0] = al_load_bitmap("walking w0000.bmp");
    image[1] = al_load_bitmap("walking w0001.bmp");
    image[2] = al_load_bitmap("walking w0002.bmp");
    image[3] = al_load_bitmap("walking w0003.bmp");
    image[4] = al_load_bitmap("walking w0004.bmp");
    image[5] = al_load_bitmap("walking w0005.bmp");
    image[6] = al_load_bitmap("walking w0006.bmp");
    image[7] = al_load_bitmap("walking w0007.bmp");
    image[8] = al_load_bitmap("walking w0008.bmp");
    image[9] = al_load_bitmap("walking w0009.bmp");
    image[10] = al_load_bitmap("walking w0010.bmp");
    image[11] = al_load_bitmap("walking w0011.bmp");

    for(int i = 0; i < maxFrame; i++)
    {
        al_convert_mask_to_alpha(image[i], al_map_rgb(106, 76, 48));
    }

    event_queue = al_create_event_queue();
    timer = al_create_timer(1.0 / 60);

    al_register_event_source(event_queue, al_get_timer_event_source(timer));
    al_register_event_source(event_queue, al_get_keyboard_event_source());

    al_start_timer(timer);

    while(!exit)
    {
        ALLEGRO_EVENT ev;
        al_wait_for_event(event_queue, &ev);

        if(ev.type == ALLEGRO_EVENT_KEY_DOWN)
        {
            switch(ev.keyboard.keycode)
            {
            case ALLEGRO_KEY_ESCAPE:
                exit = true;
                break;
            case ALLEGRO_KEY_LEFT:

                break;
            case ALLEGRO_KEY_RIGHT:

                break;
            case ALLEGRO_KEY_UP:

                break;
            case ALLEGRO_KEY_DOWN:

                break;
            }
        }
        else if(ev.type == ALLEGRO_EVENT_TIMER)
        {
            if(++frameCount >= frameDelay)
            {
                if(++curFrame >= maxFrame)
                {
                    curFrame = 0;
                }
                frameCount = 0;
            }
        }

        al_draw_bitmap(image[curFrame], 200, 100, 0);

        al_flip_display();
        al_clear_to_color(al_map_rgb(0, 0, 0));

    }

    for(int i = 0; i < maxFrame; i++)
    {
        al_destroy_bitmap(image[i]);
    }
    al_destroy_event_queue(event_queue);
    al_destroy_display(display);

    return 0;
}

Erro completo:
'Sobrevivencia.exe': Loaded 'C:\Users\Lucas-PC\Desktop\Arquivos\Sobrevivencia\Debug\Sobrevivencia.exe', Symbols loaded.
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\ntdll.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\kernel32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\KernelBase.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\allegro-5.0.10-monolith-md-debug.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\msvcr100d.dll', Symbols loaded.
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\psapi.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\ole32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\shlwapi.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\winmm.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\shell32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\WinSxS\x86_microsoft.windows.gdiplus_6595b64144ccf1df_1.1.9600.17415_none_dad8722c5bcc2d8f\GdiPlus.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\user32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\comdlg32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\gdi32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\advapi32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\opengl32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\combase.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\msvcrt.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\rpcrt4.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\sechost.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\winmmbase.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\WinSxS\x86_microsoft.windows.common-controls_6595b64144ccf1df_5.82.9600.17810_none_7c5b6194aa0716f1\comctl32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\glu32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\ddraw.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\sspicli.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\cfgmgr32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\devobj.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\dciman32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\cryptbase.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\SHCore.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\bcryptprimitives.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\imm32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\msctf.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\d3d9.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\version.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\dwmapi.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\uxtheme.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\aticfx32.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\atiu9pag.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Unloaded 'C:\Windows\SysWOW64\atiu9pag.dll'
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\atiu9pag.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\atiumdag.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\atiumdva.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\powrprof.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\kernel.appcore.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Program Files (x86)\Internet Download Manager\idmmkb.dll', Cannot find or open the PDB file
'Sobrevivencia.exe': Loaded 'C:\Windows\SysWOW64\oleaut32.dll', Cannot find or open the PDB file
Sobrevivencia.exe has triggered a breakpoint
The thread 'Win32 Thread' (0x14c0) has exited with code 3 (0x3).
The thread 'Win32 Thread' (0xcfc) has exited with code 3 (0x3).
The thread 'Win32 Thread' (0x17e0) has exited with code 3 (0x3).
The thread 'Win32 Thread' (0x140c) has exited with code 3 (0x3).
The thread 'Win32 Thread' (0x804) has exited with code 3 (0x3).
The program '[4472] Sobrevivencia.exe: Native' has exited with code 3 (0x3).

Comment: As mensagens `Cannot find or open the PDB file` provavelmente não são importantes. A parte que importa acho que é a partir do `Sobrevivencia.exe has triggered a breakpoint`.
Comment: Imagina oque pode ser ??
Comment: Euu não sei. Talvez alguém que saiba venha aqui...
Answer: O primeiro erro está na inclusão dos arquivos do Allegro 5:
#include <allegro5\allegro.h>
#include <allegro5\allegro_native_dialog.h>
#include <allegro5/allegro_image.h>

Você está utilizando erroneamente o '\' para incluir os dois primeiros arquivos. Use '/' no lugar de '\' quando se referir a arquivos. (Se quiser saber o motivo para isso, basicamente '\' é um caractere especial em C e C++ que realiza diversas operações com strings, por isso ele é evitado.)
Seu segundo erro está na falta de inicialização dos addons que você incluiu. Sempre, sempre que for fazer qualquer programa com Allegro 5, inicialize seus arquivos. No caso os arquivos que precisam de inicialização são o allegro.h e o allegro_image.h. Para inicializar o allegro.h, ponha no começo do programa o seguinte macro:
int main() {
     al_init();

Para inicializar o allegro_image.h, ponha após o al_init() a seguinte função:
     al_init_image_addon();

O último erro do seu programa se refere à falta de referência ao Allegro 5 e suas DLLs no Visual Studio. Para resolver esse problema, clique com o botão direito no seu projeto do Visual Studio, vá em propriedades, depois selecione Configuration Properties>>Linker>>Input, e no campo "Additional Dependencies" insira o nome da sua biblioteca atual (no caso suponho que você queira que seja uma versão de debug da biblioteca), ou seja: "allegro-5.x.x-monolith-md-debug.lib" (Substitua o "x.x" pela sua versão de Allegro 5. A minha por exemplo é 5.0.10).
Edição: Outro erro gravíssimo que você cometeu foi:
al_install_keyboard;
al_init_image_addon;

Você esqueceu de adicionar os parênteses após os nomes das funções. O compilador vai achar que você está querendo dizer outra coisa ao simplesmente por o nome da função no código. A maneira correta é:
al_install_keyboard();
al_init_image_addon();

Corrigidos estes erros, é quase certo que seu programa funcionará.

Allegro Debug -Abort() Has Been Called


How to screen out non-virtual machine ip address

Question: I'm using the code below
local_addr.sin_addr.s_addr = INADDR_ANY;
, but i find that the default address is an ip of vmware,which is not what i want.
As we know, we can enumerate the ip addresses by hostinfo->h_addr_list, but how to screen out the non-virtual-machine ip addresses?
Comment: I don't quite understand the question. If what you mean is distinguish a VM IP address from a non-VM IP address, then there's no way to do so having only the IP address.
Comment: It is also possible to configure VMWare to not use virtual IPs at all, it can share the host's physical IPs instead.
Answer: i've solved this problem by enumerate network adapters by GetAdaptersInfo/getifaddrs


Aplicação para login em outro site (Facebook), erro "InvalidSubmitError"

Question: Meu código é esse:
import robobrowser
import re

url = 'https://m.facebook.com'
loggedin_title = 'Facebook' # isto vai servir para confirmarmos que estamos loggedin, vendo o titulo da pagina para onde fomos redirecionados

browser = robobrowser.RoboBrowser(history=True)
browser.open(url)

form = browser.get_form(id='login_form')
form['email'].value = 'USERNAME'
form['pass'].value = 'PASSWORD'
browser.submit_form(form)

redirect_title = re.compile('<title>(.*?)</title>').search(str(browser.parsed)).group(1)

if(redirect_title == loggedin_title):
    print('[+] SUCCESS')
    print('Username: {}\nPassword: {}'.format(form['email'].value, form['pass'].value))
else:
    print('[-] LOGIN FAILED')

Mas obtenho este erro e não consigo solucioná-lo:

Warning (from warnings module):
  File "C:\Users\EDUARDO\AppData\Local\Programs\Python\Python36\lib\site-packages\bs4\__init__.py", line 181
    markup_type=markup_type))
UserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.

The code that caused this warning is on line 1 of the file <string>. To get rid of this warning, change code that looks like this:

 BeautifulSoup(YOUR_MARKUP})

to this:

 BeautifulSoup(YOUR_MARKUP, "lxml")

Traceback (most recent call last):
  File "C:\Users\ED\Documents\Python\Python\Python\LOGIN\login.py", line 13, in <module>
    browser.submit_form(form)
  File "C:\Users\ED\AppData\Local\Programs\Python\Python36\lib\site-packages\robobrowser\browser.py", line 339, in submit_form
    payload = form.serialize(submit=submit)
  File "C:\Users\ED\AppData\Local\Programs\Python\Python36\lib\site-packages\robobrowser\forms\form.py", line 226, in serialize
    include_fields = prepare_fields(self.fields, self.submit_fields, submit)
  File "C:\Users\ED\AppData\Local\Programs\Python\Python36\lib\site-packages\robobrowser\forms\form.py", line 154, in prepare_fields
    raise exceptions.InvalidSubmitError()
robobrowser.exceptions.InvalidSubmitError

Comment: Possível duplicata de [Login no facebook com python](https://pt.stackoverflow.com/questions/142790/login-no-facebook-com-python)
Comment: @prmottajr, eu respondi a essa, mas de facto precisava de ser atualizada, e o parser devia ser passado neste caso
Answer: Coloca o parametro submit com o botão correspondente:
...
browser.submit_form(form, submit=form['login'])
...

Ficando assim:
import robobrowser
import re

url = 'https://m.facebook.com'
loggedin_title = 'Facebook' # isto vai servir para confirmarmos que estamos loggedin, vendo o titulo da pagina para onde fomos redirecionados

browser = robobrowser.RoboBrowser(history=True, parser='html.parser')
browser.open(url)

form = browser.get_form(id='login_form')
form['email'].value = 'USERNAME'
form['pass'].value = 'PASSWORD'
browser.submit_form(form, submit=form['login']) # <-- acresentar aqui

redirect_title = re.compile('<title>(.*?)</title>').search(str(browser.parsed)).group(1)

if(redirect_title == loggedin_title):
    print('[+] SUCCESS')
    print('Username: {}\nPassword: {}'.format(form['email'].value, form['pass'].value))
else:
    print('[-] LOGIN FAILED')

DOCS
Comment: Cheguei a instalar uma versão anterior do Python pra tentar resolver, mas não adiantou. Com sua ajuda melhorou, mas ainda há um erro a ser resolvido. Ele completa a leitura do código, mas ainda traz o erro relacionado ao BeautifulSoup:
Comment: @Scienceandsociety Mas fizeste o login? Isso é só um warning
Comment: @Scienceandsociety, acho que o próprio robobrowser é que está a importar mal o beautifulsoup. faz update a esse pacote ou desintala e instala outravez
Comment: Sim, o login foi feito. Esse erro ainda incomoda. Já fiz o upgrade do pacote, mas não funciona. No próprio erro, ele diz pra trocar BeautifulSoup(YOUR_MARKUP}) por BeautifulSoup(YOUR_MARKUP, "html.parser") no arquivo "C:\Python34\lib\site-packages\bs4\__init__.py", o problema é que na linha indicada não tem isso "BeautifulSoup(YOUR_MARKUP})"
Comment: Editei a resposta @Scienceandsociety, com uma possível solução: `robobrowser.RoboBrowser(history=True, parser='html.parser')`
Comment: Valeu Miguel! Funcionou perfeitamente! Pensei que essa edição tivesse que ser feita no arquivo da biblioteca. Obrigado!
Comment: @Scienceandsociety de nada, eu também não sabia, tive de ir ver os docs


What is the shield-like gear the umpires are wearing at the IPL?

Question: The umpires in this(2016) IPL are wearing a shield like gear on their hands. What is it called?
I have read from a couple of articles that it is for deflecting the ball, thus protecting themselves. But, is it really so? If yes, then what is it called?
If No, then what purpose does it serve, and what is it called?
Picture for reference:
Comment: Can you add a link?
Comment: @KenGraham Added a picture in the post above, for reference :)
Answer: At the moment, this isn't standard equipment for umpires; only Bruce Oxenford is using it at the moment as far as I'm aware. As such, I'm guessing this is a personal piece of equipment that Oxenford is using and as such it doesn't have a formal name, but yes, it's obviously for the umpire to use as protection from the ball.
For what it's worth, this isn't the first time Oxenford has used the "shield" - he also used it in the Australia vs West Indies World T20 warm up match earlier in 2016.


Integral = 0 implies function = 0?

Question: If $f : \mathbb{R}^2 \to \mathbb{R}$ is continuous and such that
$$
\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} \! f(x,t) \phi(x,t) \, \mathrm{d}x \, \mathrm{d}t = 0
$$
for all $\phi \in C_c^{\infty}(\mathbb{R}^2)$ (that is, for all compactly supported smooth functions), then is it true that $f$ must be identically equal to $0$ ?
I thought taking, in particular, $\phi_n \in C_c^{\infty}(\mathbb{R}^2)$ to be equal to $0$ outside the disk of radius $n$ centered at the origin, to $f$ where $f \geq 0$ and to $-f$ where $f < 0$ would permit us to prove this, but unfortunately it seems like such $\phi_n$ need not be $C^{\infty}$...
Comment: This sounds like the fundamental lemma of variational calculus (in two dimensions), particularly the the du Bois-Raymond lemma. See here: http://en.wikipedia.org/wiki/Fundamental_lemma_of_calculus_of_variations
Comment: Can you approximate $f$ by functions in $C_c^\infty(\Bbb R^2)$?
Answer: We can assume that $f$ has compact support. If not, take a $C_c^\infty$ partition of unity $\{\psi_n\}$ and show that $\psi_n(x)f(x)=0$ as below. Then $f=\sum\limits_n\psi_n(x)f(x)=0$.
Given an $\epsilon\gt0$, we can find a $\phi\in C_c^\infty$ so that
$$
\int_{-\infty}^\infty\int_{-\infty}^\infty|f(x,t)-\phi(x,t)|^2\,\mathrm{d}x\,\mathrm{d}t\le\epsilon
$$
The hypothesis then implies
$$
\begin{align}
&\int_{-\infty}^\infty\int_{-\infty}^\infty|f(x,t)|^2\,\mathrm{d}x\,\mathrm{d}t\\
&\le\int_{-\infty}^\infty\int_{-\infty}^\infty|f(x,t)|^2+|\phi(x,t)|^2\,\mathrm{d}x\,\mathrm{d}t\\
&=\int_{-\infty}^\infty\int_{-\infty}^\infty|f(x,t)-\phi(x,t)|^2\,\mathrm{d}x\,\mathrm{d}t\\[6pt]
&\le\epsilon
\end{align}
$$
Since $\epsilon$ is arbitrary, we have
$$
\int_{-\infty}^\infty\int_{-\infty}^\infty|f(x,t)|^2\,\mathrm{d}x\,\mathrm{d}t=0
$$
Comment: How do you know such a $\phi \in C_c^{\infty}$ can be found ?
Comment: To be accurate, we need to multiply $f$ by a $C_c^\infty$ partition of unity $\{\psi_n\}$, and then show each $\psi_n(x)f(x)=0$ as above, and then add them back up.
Comment: Okay. Thanks for the answer. It is clever to consider the square of the function... I won't forget this idea !
Answer: A nonvanishing continuous function has a nonzero value in a disk $B(x, R)$ of radius $R$ around some $x.$ Now, take $\phi$ a partition of unity function (aka bump function) whose support lies in $B(x, R)$ What happens?

Integral = 0 implies function = 0?


How do I capitalized any two letter word in a string in addition to the first letter of any word?

Question: I need to capitalized the first letter of each word in a string and also capitalized both letters the string if the length of the word is two.
Example input:
dr. david BOWIE md

Example Output:
Dr. David Bowie MD

I started of with something like this:
TextInfo tCase = new CultureInfo("en-US", false).TextInfo;
return tCase.ToTitleCase(input.ToLower());

Not sure how to pull this off.
Comment: What about names where part of the name is only 2 letters?
Comment: Very true, did not think of that. That sucks. I think I might just add specific salutation for specif string I need converted to upper.
Thanks for pointing that out.
Answer: You could try this, using Split and Join:
var input = "dr. david BOWIE md";
TextInfo tCase = new CultureInfo("en-US", false).TextInfo;
var result =  tCase.ToTitleCase(input.ToLower());

result = string.Join(" ", result.Split(' ')
               .Select(i => i.Length == 2 ? i.ToUpperInvariant() : i));

Output:

Dr. David Bowie MD
Answer: You can do this:
TextInfo tCase = new CultureInfo("en-US", false).TextInfo;
string s = "dr. david BOWIE md";
var ss = string.Join(" ", s.Split(' ').Select(u => u.Length == 2 ? u.ToUpper() : tCase.ToTitleCase(u.ToLower())).ToList());
Console.WriteLine(ss);

Outputs:

Dr. David Bowie MD
Answer: String s = "dr. david BOWIE md";
s= ConvertToMyCase(s);

public string ConvertToMyCase(string s)
{
  s = System.Threading.Thread.CurrentThread.CurrentCulture.TextInfo.ToTitleCase(s.toLower());
  List<string> my = new List<string>();
  string[] separators = new string[] {",", ".", "!", "\'", " ", "\'s"};
  foreach (string word in s.Split(separators, StringSplitOptions.RemoveEmptyEntries))
  {
    if(word.Length == 2)
    {
       word.ToUpper();
    }
  }
return s;
}

Answer: var str = "dr. david BOWIE md";
var strList = new List<string>();
str.Split(' ').ToList().ForEach(s =>
{
    strList.Add(s.Length == 2 ? s.ToUpper() : string.Format("{0}{1}", char.ToUpper(s[0]), s.Substring(1).ToLower()));
});
var output = string.Join(" ", strList.ToArray());
Console.WriteLine(output);

OUTPUT:
Dr. David Bowie MD

Answer: I would extend your solution with Regex to a one linear solution:
string result = Regex.Replace(new CultureInfo("en-US", false).TextInfo.ToTitleCase(input.ToLower()), @"(?i)\b([a-z]{2})(?!.)\b", m => m.ToString().ToUpper());
Console.WriteLine(result);

Output:
Dr. David Bowie MD


Free vector space and its quotient

Question: Given $\phi_1: V\times W \to F[V\times W]$ defined by $\phi_1(v,w)=(v,w)$, where $F[V\times W]$ is the free vector space. $\phi_2: F[V\times W] \to F[V\times W]/Y$ is defined by $\phi_2(v,w)=[(v,w)]$, where Y is so chosen that $\phi:  V\times W \to  F[V\times W]/Y ,\phi:(v,w) \mapsto [(v,w)]$, is linear. Then $\phi= \phi_1 \circ \phi_2$.
Questions:
Firstly, what can we say about the basis for the free vector space $F[V\times W]$ (in terms of the bases for $V $ and $W$)? I know this question may be a little stupid, but these stuff is from a linear algebra course and we haven't talked about things like free R-module yet. These maps are constructed in order to show $F[V\times W]/Y$ satisfies the universal property of tensor product.
Moreover, can we say any of $\phi, \phi_1, \phi_2$ is surjective? Given that not all elements in $F[V\times W]/Y= V\otimes W$  can be written as $v\otimes w= [(v,w)]$, I doubt if $\phi$ is surjective.
Answer: All the pairs $(v,w)$ are a free basis for $F[V\times W]$ so an element $T\in F[V\times W]$ is a linear combination
$$T=\sum_{k,l} T^{kl}(v_k,w_l)$$
for certain scalars.
To construct $V\otimes W$ you need to quotient $F[V\times W]$ with the vector subspace $S$ generated by the relations
$$a(v,w)-(av,w),$$
$$a(v,w)-(v,aw),$$
$$(v_1+v_2,w)-(v_1,w)-(v_2,w)\qquad \mbox{and}$$
$$(v,w_1+w_2)-(v,w_1)-(v,w_2).$$
Then $v\otimes w=(v,w)+S$

Free vector space and its quotient


Modal window on html5 + transition

Question: I use modal windows on html5 for my menu, but transition doesn't work. Help me find my errors.
-webkit-transition: opacity 400ms ease-in;
-moz-transition: opacity 400ms ease-in;
transition: opacity 400ms ease-in;

and here is example in online redactor
Answer: Your display: none is hiding the dialog immediately, before the transition starts.
Answer: You need to remove the display:none in .modalDialog and set the opacity to 0.
Comment: In this case all page will be not clickable for opera T_T
Comment: just move the modal outside the window before it is active. I.e. you can set top:100% in .modalDialog and then move it to top:0 on .modalDialog:target

Modal window on html5 + transition


How to test multiple events in RxTest

Question: We are currently implementing unit tests for our ViewModel.
When a requestInAppPayment Input request comes in from the view model, subscription is implemented only when the instance property isPurchasing filter is false (to prevent multiple touches on the purchase button).
I want to write a test case for the filter when the purchase button is pressed while the purchase is in progress, but I don't know how to do it.
I want to write a test case that filters when the purchase button is clicked while the purchase is in progress.
in ViewMode
requestInAppPayment
            .filter { [weak self] _ in
                self?.isPurchasing == false
            }
            .subscribe(with: self, onNext: { owner, inAppPayment in
                owner.isPurchasing = true
                owner.showIndicator.onNext(())
                owner.requestInAppPurchaseUseCase.execute(productID: inAppPayment.productID)
            })
            .disposed(by: disposeBag)

 requestInAppPurchaseUseCase.purchaseSuccess
            .do(onNext: { [weak self] _ in
                self?.isPurchasing = false
                self?.hideIndicator.onNext(())
            })

in Unit Test
       let observer = scheduler.createObserver(String.self)
       let stubPaymentItem = InAppPayment()

        scheduler.createHotObservable([
            .next(10, stubPaymentItem),
            .next(20, stubPaymentItem)
        ])
        .bind(to: viewModel.input.requestInAppPurchase)
        .disposed(by: disposeBag)

        viewModel.output.showPaymentResultPage
            .map { "success" }
            .drive(observer)
            .disposed(by: disposeBag)

        scheduler.start()

// result = [.next(10, "success"), .next(20, "success")],
// The logic I think is [.next(10, "success")]

    XCTAssertEqual(observer.events, [
          .next(10, "success")
        ])

Comment: Hi @Jiss. There's some information missing... There is no `showPaymentResultPage` in the view model (it seems much of the view model is missing outright.) How is an `InAppPayment` object generated in the actual app?
Comment: If making sure you don't execute an IAP while one is in progress is your goal, then you are working too hard. You only need to use `flatMapFirst` in order to prevent multiple touches on the purchase button.
Comment: hi @DanielT. First of all, I didn't have much information, but thank you for showing the example code.  The purchaseSuccess in the ViewMode output is originally showPaymentResultPage.

I'm not familiar with RxSwift, so I didn't know about the existence of flatMapFirst Operator. The code handled the test normally using flatMapFirst. Thanks again for your reply..!
Answer: Here is a fully working example based on what little you provided of the unit test alone. I did not use your view model code at all, I just wrote a view model that passes the test.
I used the test harness I have available here: https://gist.github.com/danielt1263/bd449100764e3166644f7a38bca86c96
class ViewModelTests: XCTestCase {
    func test() {
        let scheduler = TestScheduler(initialClock: 0)
        let disposeBag = DisposeBag()
        let iapObserver = scheduler.createObserver(InAppPayment.self)
        let mockUseCase = MockRIAP(scheduler: scheduler)
        let viewModel = ViewModel(requestInAppPurchaseUseCase: mockUseCase)
        // I had to add all of the above to make the test compile.
        let observer = scheduler.createObserver(String.self)
        let stubPaymentItem = InAppPayment(id: 25) // I had to give the object an ID

        scheduler.createObservable(timeline: "-AA", values: ["A": stubPaymentItem]) // waits one second then emits two stubPaymentItems one second apart.
        .bind(to: viewModel.input.requestInAppPurchase)
        .disposed(by: disposeBag)

        viewModel.output.showPaymentResultPage
            .map { "success" }
            .drive(observer)
            .disposed(by: disposeBag)

        scheduler.start()

        XCTAssertEqual(observer.events, [
            .next(2, "success") // success emits at the 2 second mark because the trigger waits a second and then the mock waits a second. The second trigger is ignored.
        ])
    }
}

class MockRIAP: RequestInAppPurchaseUseCase {
    let args: TestableObserver<InAppPayment.ID>
    let _execute: (InAppPayment.ID) -> Observable<IAPResponse>

    init(scheduler: TestScheduler) {
        args = scheduler.createObserver(InAppPayment.ID.self)
        _execute = scheduler.mock(args: args, values: ["A": IAPResponse()], timelineSelector: { _ in "-A|" }) // waits one second then emits an IAPResponse
    }
    func execute(id: InAppPayment.ID) -> RxSwift.Observable<IAPResponse> {
        _execute(id)
    }
}

Here is the production code to make the above work:
protocol RequestInAppPurchaseUseCase {
    func execute(id: InAppPayment.ID) -> Observable<IAPResponse>
}

struct ViewModel {
    struct Input {
        let requestInAppPurchase: AnyObserver<InAppPayment>
    }
    struct Output {
        let showPaymentResultPage: Driver<Void> // a Driver<Void> doesn't make sense. Why would you want to emit the previous result? Better would be a Signal<Void>.
    }
    let input: Input
    let output: Output
}

extension ViewModel {
    init(requestInAppPurchaseUseCase: RequestInAppPurchaseUseCase) {
        let _requestInAppPurchase = PublishSubject<InAppPayment>()
        let showResults = _requestInAppPurchase
            .flatMapFirst { purchase in
                requestInAppPurchaseUseCase.execute(id: purchase.id)
            }
            .map { _ in }
            .asDriver(onErrorDriveWith: .empty())

        input = Input(requestInAppPurchase: _requestInAppPurchase.asObserver())
        output = Output(showPaymentResultPage: showResults)
    }
}

struct InAppPayment: Identifiable {
    let id: Int
}

struct IAPResponse { }

How to test multiple events in RxTest