Unable to call GIPHY API

Question: I have a text box and a search button. Based on user input, I want to query Giphy api to fetch matching Gifs. but my ajax call always going to error.
Can anybody help me,
 $('#button2').click(function() {
   var srchParam = $('#srcCriteria').val();

   $.ajax({
    url: "http://api.giphy.com/v1/gifs/search?q=dog&api_key=dc6zaTOxFJmzC",
    type: "GET",
    contentType: "application/json; charset=utf-8",
    dataType: "json",
    success: function(response) {
   alert("hello");
    //console.log(response.data[0].bitly_url);
   },
  error: function(xhr, status, error) {
     alert("bye");
   }
 });
});

Here is the jsfiddle link: https://jsfiddle.net/Appy169/faedybhf/12/
Comment: Why do you do `var srchParam = $('#srcCriteria').val();` and use `$('#srcCriteria').val();` in the get parameter?
Answer: Use the "upgrade-insecure-requests" CSP directive in the HTML head
<meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests">

My issue was the embed_url link (https) redirects to a giffy s3 bucket (http) causing a 'mixed content' error. By setting the "Content-Security-Policy" to "upgrade-insecure-requests", all requests made from your html will be over https (except for 'navigational')
https://developers.google.com/web/fundamentals/security/prevent-mixed-content/fixing-mixed-content
Answer: The problem lies in url on jsfiddle. If you check the console you will see this message:

Blocked loading mixed active content "http://api.giphy.com/v1/gifs/search?q=test&api_key=dc6zaTOxFJmzC"

What this means you ask? It means that you are calling http from https.
Just for testing you can change the url to //api.giphy.com/v1/gifs/search?q=" + $('#srcCriteria').val() +  "&api_key=dc6zaTOxFJmzC
and you will see it works.
Updated working JSFIDDLE.
Comment: @ArpitaDutta, The url you are now using it's not actually the url to an image, but to a tool/page of the giphy site. Use `response.data[0].images.fixed_height.url` https://jsfiddle.net/faedybhf/30/. Inspect the object `response.data[0].images` you can see that there are multiple sizes. Including original which you can get using `response.data[0].images.original.url`
Comment: You are a genious. Thank you AGAIN
Comment: check the latest jsfiddle. the images are not loading.. But when opened in new tab they shows up https://jsfiddle.net/Appy169/faedybhf/29/
Answer: Remove Http from your URL.
var srchParam = $('#srcCriteria').val();
$.ajax({
    url: "//api.giphy.com/v1/gifs/search?q=" + srchParam  +  "&api_key=dc6zaTOxFJmzC",
    type: "GET",
    success: function(response) {
    //alert("hello");
        console.log(response.data);
    },
   error: function (e) {
   alert(e);
   }
});

JSFiddle Example
For more information visit Mixed Content Error (Http/Https)
Comment: Also had a similar problem, but it may help others to know that these URL's are case sensitive. If the case is not respected then you may get a "This content is not available" gif.

Unable to call GIPHY API


php - Get compressed contents using cURL

Question: I need to get content of various web pages. Some of them are compressed using different methods (gzip, deflate, etc). I've searched on the Internet and found the solution for gzipped content:
<code>$ch = curl_init("http://games2k.net/");
curl_setopt($ch,CURLOPT_ENCODING , "gzip");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
</code>
However, this only works for a single method. I need a solution that works for as many compression methods as possible (preferably all of them) or at least the most popular two which are gzip and deflate.
Thanks very much for your help!
Comment: Deflated content  is gzippped
Comment: Thank you @Anigel for pointing that out for me. I found that gzip actually uses DEFLATE algorithm. But people always say there are 2 popular HTTP compression methods: gzip and deflate. This really makes me confused. Anyway, I just want to know if **gzip** works for both methods or not T__T
Answer: did you try
<code>$ch = curl_init("http://games2k.net/");
curl_setopt($ch,CURLOPT_ENCODING , "");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output;
</code>
keep the quotes empty
Comment: Thank you very much, @Deepanshu! I seems that leaving the param empty works well for now. I hardly encounter compression methods other than gzip, though, and I'm still looking for more solutions and explanations.


Please explain this Map key-type constraint recipe to me

Question: I am new to OCaml and such languages.
I have been experimenting with Map and ended up with:
type thing = ThingA | ThingB

module ThingMap = Map.Make(String)

let things = [
    ("a", ThingA);
    ("b", ThingB)
  ] |> List.to_seq
    |> ThingMap.of_seq

So far so good, I've got a map of strings to things.
Then I was reading that Map.Make(String) only enforces the type of the keys, the values could be any type.
I found this recipe for enforcing that values should be of thing type:
let m: thing ThingMap.t ref = ref ThingMap.empty

The recipe seems useful but I don't understand how it works.
Three questions:

what does m represent? it has type type thing ThingMap.t ref but that doesn't mean much to me... is it useful for anything? can I just rename it to _?
does declaring this in a module enforce the constraint for all instances of ThingMap in any module? (is "instances" the right word?)
what's up with the ref and empty? how does this work?

Update
After experimenting a bit more with benefit of @Jeffrey Scofield's helpful answer I can see that this recipe is not needed in my case (initialising an immutable Map from a list of key-value pairs).
First I observe that I get a type error if subsequent elements of the list do not have same type as the first element:
let things = [
    ("a", ThingA);
    ("b", 123)
  ] |> List.to_seq
    |> ThingMap.of_seq

Error: This expression has type int but an expression was expected of type
         thing

Cool, but that's not enough because if the "wrong" element is in the first position I get the "wrong" type error for the constraint I intended:
let things = [
    ("a", 123);
    ("b", ThingB)
  ] |> List.to_seq
    |> ThingMap.of_seq

Error: This expression has type thing but an expression was expected of type
         int

I can fix this by understanding that the part after the colon in let m: thing ThingMap.t ... is a type signature, so I can get the result I want by adding a signature to things:
let things: thing ThingMap.t = [
    ("a", 123);
    ("b", ThingB)
  ] |> List.to_seq
    |> ThingMap.of_seq

Error: This expression has type int but an expression was expected of type
         thing

Here the first item has wrong type, but I get the type error I wanted.
Answer: m is a global variable (a name) that is declared by the let. This is not a sweeping declaration that all values of type ThingMap contain values of type thing. It's just a declaration of one such map (accessed through a reference). You could later use ThingMap.t with values of any type.
# let m : thing ThingMap.t ref = ref ThingMap.empty;;
val m : thing ThingMap.t ref = {contents = <abstr>}
# let n : int ThingMap.t ref = ref ThingMap.empty;;
val n : int ThingMap.t ref = {contents = <abstr>}

Here I declared a value m, a reference to a map from strings to things. Then I declared a value n, a reference to a map from strings to ints.
The left side of let is a pattern. You can certainly use _ for this pattern but then you would have no name by which to refer to your map.
This declaration doesn't enforce any constraints on anything except the specific variable m.
If you don't know what references are, you should learn about them. In essence a reference is a mutable cell that always contains a value of one type, but it can be modified to contain different values of the type. (On the other hand, it's good when starting with OCaml to try to code without using references.)
ThingMap.empty is a map with no keys (and hence no values).
If you really want to control the values used in ThingMap, you can define it in a module that specifies types for the functions that access the map. Once you've defined ThingMap.t as a visible type, there's no way to change its meaning to make it more restricted.
If you just want a handy name for the type of maps from strings to things, you can give it a name:
type thingmap = thing ThingMap.t

Update
Here is a previous SO discussion of OCaml (immutable) variable bindings and references:
What is the difference between let-bindings and references in OCaml?
Comment: Yes the second `m` is a different `m`. Any uses of the previous `m` (in function definitions say) will not change their value, for example. This is getting too complicated for this "comment" area. A tutorial on OCaml might be informative.
Comment: Thank you, this helps a lot. Am I right to think of `m` in this example as an "instance"? I understood that Maps are immutable. Does the `ref` get around that and mean I could mutate `m` by adding keys to it?  That would explain why this recipe is useful, if it was restricting values type of an immutable empty Map that would not be very useful.
Comment: The problem with "instance" is that it has a technical OO meaning. This part of OCaml isn't object oriented, so it's better to think of `m` as a reference to a value of type `thing ThingMap.t`. You can mutate `m`, yes. You can set it to different values, which is the point of using a reference. Note that it's just `m` that changes. The maps are immutable, they don't change. Just `m` changes.
Comment: If the maps are still immutable and we're just changing which one `m` points to, how is `let m: thing ThingMap.t ref = ref ThingMap.empty;;`  different from `let m: thing ThingMap.t = ThingMap.empty;;` ?
Comment: If you don't declare `m` as a reference it is immutable. You can't change the `m` in this new declaration. It will always be the empty map. OCaml "variables" are immutable. Calling them variables is a bit of a misnomer but is the historical practice in lambda calculus.
Comment: But I can subsequently `let m: thing ThingMap.t = ThingMap.of_seq (List.to_seq [("a", ThingA)]);;` and `m` will then be a map with one binding in it. I guess what you are saying is that without `ref` this is now a "different" `m`? Which is a semantic that may have consequences in my program. Whereas doing the same with a `ref` would assign a new value _to the original `m`_ ?


Not IN ou Not EXISTS qual usar?

Question: Já vi alguns respostas aqui com o uso Not In e algumas com o uso do Not Exists para responder uma mesma pergunta.
Eu não sei a diferença entre ambas e gostaria de saber sobre:

Qual dos dois é melhor quando se considera o desempenho?
Em quais ocasiões devo usar um e outro?

Se possível de exemplos com Query!
Comment: Eu uso NOT EXISTS. Esta [resposta](http://stackoverflow.com/a/11074428/2556111) no SOen explica bem o que pode ser diferente no uso de cada uma delas.
Comment: Isso gerou um debate aqui na empresa ontem mesmo rs, o DBA comentando sobre o uso do `not in` e do `not exist` em um script que criamos
Comment: @MarceloBonifazio eu geralmente uso o not in, mais não sei qual dois é mais eficiente.
Comment: Eu uso o `not in` também. Não sei responder se tem melhor desempenho ou não.
Comment: Aqui tens uma outra razão para considerar evitar o uso do NOT IN (ou do IN): http://stackoverflow.com/questions/4722220/sql-in-clause-1000-item-limit
Comment: Obrigado pessoal.
Comment: [esse artigo no manual do MySQL](https://dev.mysql.com/doc/refman/5.5/en/subquery-optimization-with-exists.html) (em inglês) descreve algumas condições em que é possível otimizar consultas com *subqueries* usando `EXISTS`
Comment: @Sanção Obrigado.
Answer: Isso vai depender da situação.  O not in é recomendado quando você já tem os valores e passa como parametro, por exemplo:
select * from tbl where status_id not in (1,3,7)

Agora se você for fazer um subselect recomendo usar o not exists, pois no primeiro resultado que ele encontrar ele já irá validar e passar para o próximo registro.
select * from user u where not exists (select 1 from player p where p.user_id = u.id limit 1)

Como eu não preciso dos campos da tabela player o retorno 1 fica mais rapido do que *. O limite 1 (muda a forma de escrever de acordo com o banco de dados) vai fazer com que ao encontrar 1 registro já seja o suficiente para dizer que existe. Update: em algumas situações o limit 1 pode deixar mais rápido de acordo com esta resposta.
Espero que tenha ficado claro.
Desempenho: como o Paulo citou na resposta abaixo, o not exists permite a utilização de índice, o que em tabelas com muitos registros poderá melhorar em muito o desempenho em relação ao not in.
Comment: Muito boa a resposta.
Comment: Muito obrigado! :)
Comment: Ótimo exemplo!!
Answer: Complementando a resposta acima. O objetivo agora é encontrar todos os empregados que não são gestores. Vamos ver como podemos conseguir isso usando o NOT IN e NOT EXISTS.

NOT IN
SQL> select count(*) from emp_master where emp_nbr not in ( select mgr_nbr from emp_master );
COUNT(*)
———-
0

NOT EXISTS
SQL> select count(*) from emp_master T1 where not exists ( select 1 from emp_master T2 where t2.mgr_nbr = t1.emp_nbr );

COUNT(*)
———-
9

Agora, existem 9 pessoas que não são gestores. Assim, você pode ver claramente a diferença que valores NULL fazem desde que NULL! = NULL no SQL, a cláusula NOT IN não retorna nenhum registro de volta.
Implicações de desempenho:
Ao usar NOT IN, a consulta executa varreduras aninhados de tabela, enquanto que para NOT EXISTS, a consulta pode utilizar um índice dentro da sub-consulta.
Answer: EXISTS é mais performático do que o IN, porém não vem a ser uma boa pratica, depende da sua necessidade....
O in "cria" uma estrutura de or's na sua condição sql, exp:
select * from tabela1
where idtabela in (select idtabela from tabela2 where condicao = 2)

Digamos que existam 5 condições = 2 de idtabela nessa tabela2 , o bd interpretaria o in como:
select * from tabela1 t
inner join tabela2 t2 on t2.idtabela = t.idtabela
where idtabela = 1 or idtabela = 2 or idtabela = 3 or idtabela = 4 or idtabela = 5

Agora ao usar o exists o bd "força" um relacionamento entre as duas tabelas
Ficando mais ou menos assim:
select * from tabela1 t
inner join tabela2 t2 on t2.idtabela = t.idtabela
where condicao = 2

Answer: A resposta depende primeiro de qual é o gerenciador de banco de dados (sgbd) pois as implementações internas podem não ser as mesmas para NOT IN e NOT EXISTS.
No caso do SQL Server recentemente publiquei o artigo "Qual é mais rápido: NOT IN ou NOT EXISTS?" em que é tratada a questão de "como saber quais elementos estão no conjunto A mas não estão no conjunto B". Além de NOT IN e NOT EXISTS são tratadas de outras opções como EXCEPT, LEFT OUTER JOIN e OUTER APPLY.
Em primeiro lugar, sempre é recomendado analisar o plano de execução. E é preciso ter em mente que o plano de execução gerado varia de acordo com vários fatores, inclusive com as características dos dados.
Em minha opinião não há como definir a priori qual método é o mais rápido.
Ao analisar outros artigos sobre o assunto percebe-se que em alguns foram utilizados casos em que a situação é otimizada para se obter determinada conclusão. Por exemplo, no artigo "Should I use NOT IN, OUTER APPLY, LEFT OUTER JOIN, EXCEPT, or NOT EXISTS?", de Aaron Bertrand, ele afirma logo no início do artigo que "Instead of NOT IN, use a correlated NOT EXISTS for this query pattern. Always". Ou seja, segundo ele NOT EXISTS sempre é mais rápido. Entretanto, ele cita "for this query pattern". Que padrão de consulta? Consulta direta entre somente duas tabelas e que possuam índices que permitam a junção direta entre as tabelas. Ora, aí é mamão com açúcar, né?
Já no artigo "Consider using [NOT] EXISTS instead of [NOT] IN with a subquery", de Phil Streiff, consta que "It used to be that the EXISTS logical operator was faster than IN, when comparing data sets using a subquery. (…) However, the query optimizer now treats EXISTS and IN the same way, whenever it can, so you're unlikely to see any significant performance differences".

Not IN ou Not EXISTS qual usar?


How to compare whether the coefficients of two independent variables statistically different from each other?

Question: If I have two independent variables and they are dummy variable along with other independent variables and I run a linear probability model, I want to compare whether the coefficients of two dummy variables are statistically different from each other. I do not know how to compare that. Could someone help me on that?
Comment: An alternative to the answer you have is by [reparameterization of the model into parameter and difference of the two](http://stats.stackexchange.com/questions/93524/testing-whether-two-regression-coefficients-are-significantly-different-in-r-id), by suppling the basline dummy and the sum of the two dummies. and a direct test of the coefficient for the difference. (In fact given the fact that being a dummy makes no difference to the approach, this Q is arguably a duplicate.)
Comment: Hint: think through the statistical procedure to determine statistical significance of coefficients. What is the coefficient estimate compared against?
Comment: what statistical procedure are you referring to do you mean the z statistic is coefficient divided by its standard error?
Comment: Sorry - that was a bit too elliptical. The test that you're looking for is the Wald test (http://stats.stackexchange.com/questions/92912/how-to-calculate-p-values-in-logistic-regression-with-gradient-descent-algorithm/92940). Usually, you compare the coefficient estimate to zero... But not in this case.
Comment: The Wald test seems to be used for one variable for it parameter and the proposed value at least based on wiki...but I have to compare two DIFFERENT dummy variables though...
Comment: When determining statistical significance of a single parameter, we compare whether it is sufficiently "far" from zero to reject the null that it is zero. But what is the null in this case, when you are comparing two coefficients to eachother?
Comment: there are two null in mind let say they are A and B...one scenario for null is A + B = 0. second scenario is A*B = 0... Are these the right approach?
Answer:
I want to compare whether the coefficients of two dummy variables are statistically different from each other

I understand that you want to test the hypotheses $$H_0: A=B$$
$$H_1: A\not=B$$.
So we are performing inference on the quantity $A-B$ and evaluating whether it is sufficiently far from 0 relative to its standard error. Ordinarily, in the single-parameter case, we just compare, e.g., $A$ to its standard error. But we have two parameters here. So we have
$$\sqrt{\frac{(A-B)^2}{\text{var}(A)+\text{var}(B)-2\text{cov}(A,B)}}$$
as our test statistic, where $\text{cov}(A,B)$ is the covaraince of $A$ and $B$. Now we compare the test statistic to a standard normal distribution.
Your last comment:

there are two null in mind let say they are A and B...one scenario for null is A + B = 0. second scenario is A*B = 0... Are these the right appraoch?

confuses me, because it suggests that your null hypothesis is not that $H_0:A=B$, but instead that $H_0:A=-B$, or $H_0:A=1/B$. While not typical, this certainly might make sense for your particular research topic.
Comment: The Stata command for a Wald test is `test`. The last hypothesis $H_0 : A = 1/B$ is a non-linear Wald test, the Stata command for that is `testnl`.
Comment: Based on what you said, I can set a null A - B = 0 and then use F test and If I can show that they are statistically significant ... it means they are statistically different from each other?
Comment: No, this is still a Wald test. But you have the general idea right: we are testing whether parameter $A$ is different from $B$.
Comment: what is the command for Wald test?
Comment: I'm afraid I'm not a STATA user.
Comment: Quick question: Shouldn't the variance in the denominator for $H_0: A=B$ be $V(A) + V(B) - 2\cdot \text{Cov}(A,B)$? That is the variance of the difference of two random variables.
Comment: @AndyW I think you might be correct, Andy. After I posted this answer I went searching for a textbook reference, but I haven't yet found it. Thanks for pointing that out.


A Simple probability question

Question: Suppose you are correct 90% of the time.
But in a particular case of 30 questions, you only get 15 correct. What are the odds of that?
Comment: The question can be answered if we model it by thinking of $30$ rolls of a $10$-sided die. That may not be a reasonable model, if many of the questions are about the $10\%$ that we do not know. And being aware that things are going rather badly may affect performance.
Comment: You're probably intending for someone to tell you the *probability*, not the *odds* of that occurring. If you interpret 0.5 as a probability, that means 50% of the time. If you interpret 0.5 as odds, that is odds of 1:2, which means "twice as likely to happen than not," which translates to a probability of one third (33.333...%) that the event happens.
Comment: Welcome to math.SE: since you are new, I wanted to let you know a few things about the site. In order to get the best possible answers, it is helpful if you say in what context you encountered the problem, and what your thoughts on it are; this will prevent people from telling you things you already know, and help them give their answers at the right level.
Answer: If we assume that the correctness of your answers to the questions are independent (getting one question correct does not affect your chances of getting any other question correct), then (as said in Alex's answer) we can use a Binomial probability distribution to get our answer.
Here's how it works: If the correctness of your answers to each question are independent, this implies (by the laws of probability) that the probability of getting one answer right and another answer wrong is equal to the product of the probabilities of getting each answer right individually. (If $A$ and $B$ are independent events, then $P(A \cap B) = P(A) P(B)$.) If you are correct 90% of the time, and each of the thirty questions are independent, then the probability of getting a specific question right is $90\%$, and the probability of getting a specific question wrong is $100\% - 90\% = 10\%$. So any single arrangement of 15 correct answers and 15 incorrect answer has a probability equal to the product of fifteen $90\%$'s and fifteen $10\%$'s; that is:
$$
P(\textrm{a specific arrangement of 15 right and 15 wrong answers}) = (90\%)^{15}(10\%)^{15}=(0.9)^{15}(0.1)^{15}
$$
However, there is more than one way to arrange 15 correct answers and 15 incorrect answers. (For example, instead of getting all the first fifteen right, you get the first fourteen right and the sixteenth right.) In fact, the number of arrangements of 15 correct and 15 incorrect answers out of 30 answers is $\binom{30}{15}$ (thirty choose fifteen), which equals $\frac{30!}{15! 15!} = 155\,117\,520$. Each of these arrangements is disjoint (meaning you can't get more than one arrangement at the same time), which implies (by the laws of probability) that the probability of one arrangement, or another, or another, or another (and so on until the last arrangement) is equal to the sum of the probabilities. (If $A$ and $B$ are disjoint events, then $P(A \cup B) = P(A) + P(B)$.) Each arrangement has the same probability of happening, so the probability of that one of those 15517520 arrangements occurring is the probability of a single arrangement, multiplied by 155117520. So, now, the probability of any of the 155117520 arrangements of 15 correct and 15 incorrect answers is:
$$
\binom{30}{15}(90\%)^{15}(10\%)^{15} = (155117520)(0.9)^{15}(0.1)^{15} = 3.193732180051435815048 \times 10^{-8} \approx 0.000000032
$$
In general: The Binomial probability distributions make this a lot shorter because it puts all that stuff into one formula. The probability of getting $k$ successes (here, $k$ questions right) out of $n$ independent attempts (here, out of $n$ questions asked) with a true probability of success represented as $p$ (here, a $(100 \times p)\%$ chance of getting a question right) is:
$$
\binom{n}{k} p^k (1-p)^{n-k} = \frac{n!}{k! (n-k)!} p^k (1-p)^{n-k}
$$
Final note: The above demonstrates the probability of getting exactly 15 right and 15 wrong. The probability of getting 15 or less right and 15 or more wrong is a slightly higher. You can get this by using the above formula (again using $n = 30$ and $p = 0.9$) for $k=15$, $k=14$, $k=13$, and so on down to $k=0$, and adding all those probabilities together (because these are also disjoint events). This probability is $3.559479269444171697124 \times 10^{-8} \approx 0.000000036$.
Comment: Thanks for your helpful explanation! It is easier to understand and I can see the multiple ways to look at the question.
Comment: You're welcome! Be sure to upvote the question since you found it helpful. Thanks!
Comment: I tried to upvote, but I have insufficient reputation ;-(
Comment: Perhaps anyone who happens along here can click instead because I cannot. Please?!

In any case, I greatly appreciate your beautiful and thorough explanation. I was still confused by Alex's answer..
Comment: Ah! Of course! I forgot about that. If you feel inclined, you may mark my answer as accepted, but you don't have to.
Comment: I clicked on the checkmark. Thanks for the newbie tip.
Comment: You're surely welcome. Thank you, too.
Answer: Binomial distribution $B(30,0.9)$
$$\binom{30}{15}\left ( \frac{9}{10}\right )^{15}\left ( \frac{1}{10}\right )^{15}\approx3.19373 \cdot 10^{-8}$$
Comment: Thanks for your answer @Alex I had to do 1 / 3.19*10^(-8) because I wanted it to be expressed that way?
Comment: @Kevinsb, do you mean you want to express it as "a one-in-____ chance of getting 15 questions right"?
Comment: @Firefeather Yes.
Comment: @Kevinsb, that would work; that works out to about 31311329, so you could say that there is about a 1 in 31,311,329 chance of getting 15 questions right.
Comment: @Firefeather Thanks! Yes, that seems like a helpful way to think about very long odds.

A Simple probability question


Obter a estrutura de uma tabela através de uma consulta

Question: Através do PHPMyAdmin, acedendo a uma tabela, podemos ver a estrutura da mesma no separador structure que nos lista os campos, os tipos de dados que cada um aceita, entre outras informações.

Como posso obter através de uma consulta os detalhes de determinada tabela na base de dados, mais especificamente a estrutura da mesma, os seus campos e o tipo de dados que cada campo aceita?
Answer: Faça assim:
DESCRIBE tabela;

Ou assim:
SHOW CREATE TABLE tabela;

Answer: Se entendi é DESCRIBE nome_da_tabela
ref.: https://dev.mysql.com/doc/refman/5.0/en/getting-information.html
Answer: Essas informações pode ser obtidas através do information_schema
SELECT * FROM information_schema.columns WHERE table_name = 'nome_da_tabela'

forma abrevida:
describe nome_da_tabela


Encontrar o centro de cada circulo na imagem

Question: Tenho essa imagem

e queria pegar o x e o y do centro de cada circulo. Já tentei de várias maneiras, mas em nenhuma tive sucesso. Minha pergunta é se existe alguma lib java para fazer isso ou algum algoritmo específico, se sim por favor me indique.
Comment: Seu problema é interessante, você pode utilizar esta técnica http://pt.stackoverflow.com/questions/105891/ajuda-processamento-de-imagem-remo%C3%A7%C3%A3o-de-fundo e depois percorrer os pixels da imagem para encontrar apenas o pixel da cor dos circulos. Depois encontra a fórmula matemática para encontrar o centro. O que estou passando é muito superficial, mas é uma ideia de início
Comment: mas não adianta eu mudar a cor de fundo. Vou ficar com o mesmo problema com cores diferentes
Comment: Então desta forma você vai precisar de pontos (pixels) com coordenadas x e y que estejam na boda do circulo, estes links podem te ajudar, experimenta dar uma olhada http://stackoverflow.com/questions/4103405/what-is-the-algorithm-for-finding-the-center-of-a-circle-from-three-points e http://paulbourke.net/geometry/circlesphere/
Answer: A resposta que você já tem é muito boa e fornece um algoritmo simples e muito eficiente para a imagem de exemplo fornecida. Entretanto, ele pode não funcionar bem para imagens mais complexas, contendo, por exemplo, ruído ou círculos sobrepostos ou incompletos (nos cantos da imagem). Como você não forneceu detalhes do seu domínio de problema, talvez essas variações possam ocorrer.
Nesse sentido, uma alternativa muito bacana é a Transformada de Hough. Esse algoritmo permite encontrar em uma imagem qualquer tipo de forma que seja parametrizável por meio de uma equação matemática (tais como retas, círculos, elípses, etc).
Princípio Matemático
O princípio matemático desse algoritmo é facilmente explicado considerando o exemplo mais simples: a identificação de retas em uma imagem. Considere os gráficos abaixo:

Suponha que uma imagem contém um conjunto de pontos que, se ligados, traçam uma reta (imagem da esquerda). A equação geral de uma reta é . Assim, em um ponto qualquer da imagem, de coordenadas , há infinita retas que passam por ele (as retas mais claras na imagem): basta variar os parâmetros  (a inclinação da reta) e  (a altura que ela corta no eixo ) para os valores  fixos. Naturalmente, isso vale para qualquer outro ponto. E como se pode perceber pelo desenho da esquerda, há apenas uma reta que passa por todos os pontos (a reta mais escura na imagem). Isso significa que para todos esses pontos, a equação dessa reta específica tem o mesmo valor de  e .
Ao invés de representar cada ponto no espaço , nós os representarmos no espaço  (também chamado de espaço paramétrico, ilustrado na figura da direita). Nesse espaço, um ponto do espaço original agora é descrito por uma reta (devido ao ajuste na equação para considerar os parâmetros como variáveis) com todas as combinações de valores  e  que, lembre-se, descrevem todas as retas possíveis que passam naquele ponto no espaço original . Desse modo, quando existe uma reta que liga todos os pontos no espaço original, as retas no espaço de parâmetro que descrevem cada um dos pontos coincidem em um local exato: nos valores dos parâmetros  e  que justamente satisfazem a equação da reta no espaço original.
Assim, o algoritmo simplesmente trabalha no espaço paramétrico e vai testando, ponto a ponto e para cada possível valor de  e , onde se encontram esses cruzamentos. Ele faz isso construindo uma matriz z-dimensional (vide figura abaixo) onde z é o número de parâmetros na equação representativa da forma que se busca. Essa matriz armazena "contagens" de ocorrências dos valores dos parâmetros (tanto que é comumente chamada de matriz de acumuladores) em intervalos discretos. Por exemplo, no caso da equação da reta anterior, a matriz é a seguinte:

Os valores dessa matriz são inicializados para 0 e conforme o algoritmo é executado eles são incrementados toda vez que um ponto qualquer da imagem original tiver uma reta com parâmetros  e  no intervalo utilizado. No final, basta escolher o par de parâmetros com mais "votos" (com maior acumulador). Ele indicará a melhor reta estimada que passa por todos os pontos analisados.

Eu espero que essa explicação seja suficiente. Mas se precisar de uma
  explicação passo-a-passo, esse vídeo (em inglês) pode ser de muita
  ajuda.

É mais comum utilizar na detecção de retas a representação da equação na forma polar (, em que  representa o vetor normal à reta e  é o ângulo desse vetor em relação ao eixo horizontal), porque assim se evita ter de tratar casos em que o valor do parâmetro  é infinito (a reta está inclinada em 90º).
No caso da detecção de círculos (o seu interesse), utiliza-se comumente a equação da circunferência na forma paramétrica (onde  são as coordenadas do centro do círculo,  é seu raio e  é o ângulo em relação ao eixo horizontal):

Esse algoritmo tem como vantagens ser razoavelmente simples e fácil de implementar, ser robusto à ruído (pois o ruído gera menos "votos"/acumuladores) e a dados parciais (os dados de apenas parte de um círculo, por exemplo, ainda assim geram bastante acumuladores para um mesmo conjunto de parâmetros). A principal desvantagem é que ele se torna computacionalmente custoso conforme o número de parâmetros cresce. No exemplo da reta a matriz era bidimensional pois utilizava-se 2 parâmetros. No caso do círculo são 3, fazendo com que agora tenhamos uma matriz tridimensional para manipular.
Implementação
Eu construí um código de exemplo, da forma mais simples e didática possível. Ele segue abaixo. Vale notar o seguinte:
1 - Ele foi feito pensando mesmo na didática. Então não está tão otimizado quanto poderia. Por exemplo, o espaço de armazenamento da matriz de acumuladores poderia ser diminuido conforme a configuração de raio mínimo e máximo, e eu simplesmente aloco a área máxima mesmo que não a utilize completamente. Além disso, a ordenação dos círculos encontrados e a eliminação de círculos semelhantes foi feita de forma bem simples e direta, com mais de uma varredura da matriz de acumuladores. Certamente há formas melhor de fazer isso.
2 - A definição dos limites de parâmetros foi feita de forma empírica. Por exemplo, no caso do ângulo eu simplesmente dividi a matriz em 360 graus, em intervalos de 1 grau. E no caso do raio, ele vai do mínimo configurado ao máximo calculado (a metade da diagonal da imagem), também em intervalos de 1 pixel. Essa não é necessariamente a forma correta de se fazer. Analisando código existente, percebe-se que é mais comum fazer o intervalo angular ser 1/8 do menor raio (admito que não sei exatamente a razão disso, mas certamente tem a ver com desempenho de armazenamento vs desempenho de resultado de detecção).
3 - Apesar de esse ser um algoritmo simples e fácil de implementar, na prática não vale a pena reimplementá-lo. Há inúmeras implementações disponíveis, e a mais indicada é a do OpenCV - que também pode ser utilizado em Java.
Eis o código:
import javax.imageio.ImageIO;
import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;

import java.awt.BasicStroke;
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.Image;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.net.URL;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.Map;
import java.util.TreeMap;

public class Teste {

    /**
     * Inner class para representação de círculos.
     */
    public static class Circle {
        /** Coordenada x do centro do círculo. */
        public int x;

        /** Coordenada y do centro do círculo. */
        public int y;

        /** Raio do círculo em pixels. */
        public int r;

        /**
         * Construtor da classe.
         * @param x Coordenada x do centro do círculo.
         * @param y Coordenada y do centro do círculo.
         * @param r Raio do círculo em pixels.
         */
        public Circle(int x, int y, int r) {
            this.x = x;
            this.y = y;
            this.r = r;
        }

        /**
         * Verifica se os círculos são similares segundo um limiar.
         * @param oOther Instância do outro círculo para comparação.
         * @param iThreshold Inteiro com o limiar para comparação.
         * @return Verdadeiro se os círculos são similares segundo
         * o limiar dado, falso caso contrário.
         */
        public boolean similarTo(Circle oOther, int iThreshold) {
            return Math.abs(oOther.x - x) <= iThreshold &&
                   Math.abs(oOther.y - y) <= iThreshold &&
                   Math.abs(oOther.r - r) <= iThreshold;
        }
    };

    /**
     * Cria uma janela para exibição da imagem dada.
     * @param oImage Objeto BufferedImage com a imagem a ser exibida.
     * @param sTitle String com o título da janela.
     * @return Instância de JFrame com a janela criada.
     */
    private static JFrame showImage(Image oImage, String sTitle) {
        JFrame oWindow = new JFrame();
        JPanel oPanel = new JPanel(new BorderLayout());
        JLabel oWidget = new JLabel(new ImageIcon(oImage));
        oPanel.add(oWidget);
        oWindow.add(oPanel);
        oWindow.setVisible(true);
        oWindow.setTitle(sTitle);
        oWindow.pack();
        oWindow.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);

        return oWindow;
    }

    /** Máscara para cálculo do gradiente por convolução no eixo X. */
    private static int[][] SOBEL_X = {
                                        { -1, 0, +1 },
                                        { -2, 0, +2 },
                                        { -1, 0, +1 },
                                     };

    /** Máscara para cálculo do gradiente por convolução no eixo Y. */
    private static int[][] SOBEL_Y = {
                                        { +1, +2, +1 },
                                        {  0,  0,  0 },
                                        { -1, -2, -1 },
                                     };

    /**
     * Processa a imagem (por convolução) para extração das bordas.
     * @param oImage BufferedImage com a instância da imagem original.
     * @return BufferedImage com a imagem binária contendo apenas as bordas (em
     * branco) sobre o fundo (em preto).
     */
    private static BufferedImage getEdges(BufferedImage oImage) {
        BufferedImage oRet = new BufferedImage(oImage.getWidth(), oImage.getHeight(), oImage.getType());

        // Percorre a imagem para fazer a convolução
        for(int x = 0; x < oImage.getWidth(); x++) {
            for(int y = 0; y < oImage.getHeight(); y++) {

                // Calcula as somas ponderadas pelas máscaras no pixel atual
                int iSumX = 0, iSumY = 0;
                for(int i = 0; i < 3; i++) {
                    for(int j = 0; j < 3; j++) {
                        int xl = x + (i - 1);
                        int yl = y + (j - 1);

                        int iValue;
                        if(xl >= 0 && xl < oImage.getWidth() && yl >= 0 && yl < oImage.getHeight()) {
                            Color oColor = new Color(oImage.getRGB(xl, yl));
                            iValue = (int) Math.ceil((0.21 * oColor.getRed() + 0.72 * oColor.getGreen()  + 0.07 * oColor.getBlue()));
                        }
                        else { // Considera pixels fora da imagem com o mesmo valor do pixel atual
                            Color oColor = new Color(oImage.getRGB(x, y));
                            iValue = (int) Math.ceil((0.21 * oColor.getRed() + 0.72 * oColor.getGreen()  + 0.07 * oColor.getBlue()));
                        }

                        iSumX += iValue * SOBEL_X[i][j];
                        iSumY += iValue * SOBEL_Y[i][j];
                    }
                }

                // Calcula a magnitude do gradiente no pixel e atualiza a imagem de retorno
                int iMag = (int) Math.ceil(Math.sqrt(Math.pow(iSumX, 2) + Math.pow(iSumY, 2)));
                oRet.setRGB(x, y, iMag > 254 ? Color.WHITE.getRGB() : Color.BLACK.getRGB());
            }
        }

        return oRet;
    }

    /**
     * Cria uma tabela de referência para os cálculos envolvendo seno e cosseno,
     * como forma de otimizar o desempenho do algoritmo.
     * @param iMaxRadius Inteiro com o maior raio buscado (utilizado na definição
     * do tamanho da estrutura de dados).
     * @return Matriz de 3 dimensões com os dados do seno e cosseno dos diferentes
     * ângulos possíveis, já multiplicados pelos valores de raio possíveis.
     */
    private static int[][][] buildLookupTable(int iMaxRadius) {
        // A tabela tem espaço para 2 valores (do seno e do cosseno), n raios
        // (sendo n a metade da diagonal da imagem = raio do maior círculo
        // possível) e 360 graus (theta).
        int aTable[][][] = new int[2][iMaxRadius + 1][361];

        for(int iRadius = 0; iRadius <= iMaxRadius; iRadius++) {
            for(int iAngle = 0; iAngle <= 360; iAngle++) {

                double dRad = Math.PI * iAngle / 180; // ângulo em radianos
                int iCos = (int) (iRadius * Math.cos(dRad));
                int iSin = (int) (iRadius * Math.sin(dRad));

                aTable[0][iRadius][iAngle] = iSin;
                aTable[1][iRadius][iAngle] = iCos;
            }
        }

        return aTable;
    }

    /**
     * Processa a imagem de bordas (pela transformada de Hough de círculos) para extração
     * dos círculos.
     * @param oImage BufferedImage com a instância da imagem binária com as bordas.
     * @param iMinRadius Inteiro com o menor raio a ser procurado.
     * @param iMaxRadius Inteiro com o maior raio a ser procurado.
     * @param iMaxCircles Inteiro com o número máximo de círculos a ser devolvido.
     * @return
     */
    private static Circle[] getCircles(BufferedImage oImage, int iMinRadius, int iMaxRadius, int iMaxCircles) {

        // Constroi a tabela de lookup (pra agilizar os cálculos)
        int aTable[][][] = buildLookupTable(iMaxRadius);

        // Calcula os acumuladores na imagem, segundo a equação paramétrica do círculo
        int aAcc[][][] = new int[oImage.getWidth()][oImage.getHeight()][iMaxRadius+1];

        for(int x = 0; x < oImage.getWidth(); x++) {
            for(int y = 0; y < oImage.getHeight(); y++) {

                // Ignora pixels que não contêm bordas
                if(oImage.getRGB(x, y) == Color.BLACK.getRGB())
                    continue;

                for(int iRadius = iMinRadius; iRadius <= iMaxRadius; iRadius++) {
                    for(int iAngle = 0; iAngle <= 360; iAngle++) {

                        int a = x + aTable[1][iRadius][iAngle];
                        int b = y + aTable[0][iRadius][iAngle];

                        if(a >= 0 && a < oImage.getWidth() && b >= 0 && b < oImage.getHeight())
                            aAcc[a][b][iRadius]++;
                    }
                }
            }
        }

        // Cria estrutura ordenada pelos acumuladores
        Map<Integer, Circle> mAcc = new TreeMap<Integer, Circle>(new Comparator<Integer>()
        {
            @Override
            public int compare(Integer i, Integer j) {
                return j.compareTo(i);
             }
        });

        for(int a = 0; a < aAcc.length; a++) {
            for(int b = 0; b < aAcc[a].length; b++) {
                for(int r = 0; r < aAcc[a][b].length; r++) {
                    int iAcc = aAcc[a][b][r];
                    mAcc.put(iAcc, new Circle(a, b, r));
                }
            }
        }

        // Move os dados para um array, já removendo as duplicidades
        ArrayList<Circle> aRet = new ArrayList<Circle>();
        for (Map.Entry<Integer, Circle> oCur: mAcc.entrySet()) {

            // Procura no ArrayList por um círculo parecido
            // Se existir, ignora. Caso contrário, adiciona.
            boolean bExist = false;
            for(Circle oExist: aRet) {
                if(oCur.getValue().similarTo(oExist, 10))
                {
                    bExist = true;
                    break;
                }
            }

            if(!bExist)
                aRet.add(oCur.getValue());
        }

        // Devolve o número de círculos desejado
        return aRet.subList(0, iMaxCircles).toArray(new Circle[0]);
    }

    public static void main(String[] args)  {

        // Carrega a imagem original
        BufferedImage oImage = null;
        try {
            URL oURL = new URL("https://i.stack.imgur.com/FWgw0.png");
            oImage = ImageIO.read(oURL);
        }
        catch (IOException e) {
            System.out.println("Oops! Não foi possível carregar a imagem.");
            System.exit(-1);
        }

        // Exibe a imagem original
        JFrame oSource = showImage(oImage, "Imagem original");

        // Processa a imagem original para detecção das bordas
        BufferedImage oEdges = getEdges(oImage);

        // Exibe a imagem com as bordas detectadas
        JFrame oTarget = showImage(oEdges, "Imagem com as bordas");
        oTarget.setLocation(oSource.getLocation().x + oSource.getWidth(), oSource.getLocation().y);

        // Aplica a transformada de Hough para o círculo
        int iMinRadius = 10;
        int iMaxRadius = (int) Math.ceil(Math.sqrt(Math.pow(oImage.getWidth(), 2) + Math.pow(oImage.getHeight(), 2)) / 2);
        int iMaxCircles = 4;
        Circle[] aCircles = getCircles(oEdges, iMinRadius, iMaxRadius, iMaxCircles);

        // Desenha os círculos na imagem original
        Graphics2D g = oImage.createGraphics();
        g.setPaint(Color.BLACK);
        g.setStroke(new BasicStroke(2, BasicStroke.CAP_ROUND, BasicStroke.JOIN_ROUND));
        for(int i = 0; i < aCircles.length; i++) {
            Circle oCircle = aCircles[i];
            g.drawLine(oCircle.x-5, oCircle.y, oCircle.x+5, oCircle.y);
            g.drawLine(oCircle.x, oCircle.y-5, oCircle.x, oCircle.y+5);
            g.drawOval(oCircle.x - oCircle.r, oCircle.y - oCircle.r, 2 * oCircle.r, 2 * oCircle.r);
        }
        g.dispose();

        // Exibe a imagem com as anotações dos círculos encontrados
        JFrame oFinal = showImage(oImage, "Imagem original anotada");
        oFinal.setLocation(oTarget.getLocation().x + oTarget.getWidth(), oTarget.getLocation().y);
    }

}

Como o código funciona
O primeiro passo é limitar os dados a serem processados. Se a Transformada de Hough fosse aplicada sobre todos os pixels da imagem original, ia demorar demais e potencialmente gerar muita confusão (muitos acumuladores parecidos). Assim, primeiramente eu produzo uma imagem binária (contendo só cor preta e branca) apenas com as bordas encontradas na imagem. O melhor modo de fazer isso é aplicar o filtro Sobel por meio de convolução no domínio espacial seguido de uma limiarização simples (não vou explicar esse processo em detalhes por falta de espaço, mas leia os links da Wikipedia e analise o código pra entender).

Note essa linha de código no método getEdges:
oRet.setRGB(x, y, iMag > 254 ? Color.WHITE.getRGB() : Color.BLACK.getRGB());

O valor 254 ali é o limiar utilizado para limiarização. Ou seja, eu
  praticamente ignoro (faço ser igual à cor preta) qualquer valor de
  magnitude do gradiente que seja menor ou igual a 254. Isso deixa as bordas mais
  finas, e reduz informação (pixels de borda) na imagem resultante. Vc pode/deve
  ajustar esse limiar conforme o seu domínio de problema.

Então, a transformada de Hough é aplicada apenas aos pixels em branco na imagem de bordas (ou seja, apenas aos pixels de borda). Essa abordagem é muito comum na literatura, pois esses são os pixels essenciais para a detecção de formas. :)
O resultado da execução desse código para a sua imagem de exemplo (lida diretamente da sua postagem) é esse:

Eu fiz uma imagem de exemplo com algumas variações que podem ser difícies de detectar no algoritmo mais simples anterior:

O resultado do código para essa imagem é esse:

Note que você precisa alterar para 7 o número máximo de círculos na chamada do método getCircles:
. . .
int iMinRadius = 10;
int iMaxRadius = (int) Math.ceil(Math.sqrt(Math.pow(oImage.getWidth(), 2) + Math.pow(oImage.getHeight(), 2)) / 2);
int iMaxCircles = 7; // ALTERE AQUI CONFORME O NÚMERO DE CÍRCULOS EXISTENTES
Circle[] aCircles = getCircles(oEdges, iMinRadius, iMaxRadius, iMaxCircles);
. . .

Sobre a importância do processamento intermediário
O filtro de Sobel aplicado antes da transformada de Hough é fundamental para a qualidade do resultado. Outra alternativa bastante utilizada é o filtro de Canny. Ainda assim, dependendo do tipo de imagem utilizada, talvez você precise também aplicar outros filtros para redução de rúido (usando, por exemplo, um filtro Gaussiano ou até mesmo filtros mais simples como a média e mediana.
Um exemplo é para a detecção de moedas de Real na imagem abaixo. Como apenas a detecção de bordas ainda deixa muitos pixels de borda devido à textura das moedas, o resultado final (para a detecção de 10 círculos com o algoritmo acima) não é exatamente o esperado:

Note que um círculo pequeno na moeda de 5 centavos mais à direita foi detectado com muitos acumuladores. Esse erro talvez deva-se às escolhas empíricas descritas anteriormente, e talvez pudesse ser corrigido com o ajuste do raio mínimo (se você souber de antemão o tamanho da menor moeda na imagem, por exemplo). Mas note como a imagem de bordas tem muitos detalhes. Eles fazem não somente o processamento demorar mais como também não ajudam na detecção de círculos ideais.
Comment: obrigado pela dedicação. Isso não foi só uma resposta, foi uma aula. Em alguns testes ficou um pouco lento o processo, mesmo fazendo suas indicações. Isso só ocorre quando tem vários círculos na imagem. Mas mesmo assim, sua resposta é perfeita para essa pergunta. obrigado!
Comment: Por nada. É, a lentidão é um problema sério porque o código não foi feito pensando em desempenho. rs Você pode tentar melhorar o código (sugiro medir onde demora mais) - ou use mesmo o do OpenCV. :)
Answer: Esse problema parece ser muito difícil, mas não é.
Antes de apresentar a minha solução, aconselho que use essas bibliotecas do java:
import javax.imageio.ImageIO
import java.awt.image.BufferedImage

Bem, minha solução é essa:
Leitura:

Antes de tudo, crie uma matriz com as dimensões da imagem de entrada.
Ande pixel por pixel da imagem de entrada, e coloque zeros nas coordenadas da matriz que representam o branco na foto, e uns para as que representam o vermelho. No final, você terá uma matriz binária que representa a foto, 0 == branco, 1 == vermelho.

Processamento:

Ande por cada coordenada da matriz vendo se a coordenada atual é 1, se for, quer dizer que você acabou de tangenciar um circulo, ou seja, tocou no ponto "mais alto" dele(Visto de um referencial 2D), guarde essa coordenada em algum lugar. Bem, pensa comigo, se eu estou no Brasil e cavo um buraco em linha reta e perpendicular ao plano que eu estou em pé, vou sair na outra extremidade da Terra(Tókio?), certo? Se eu medir o comprimento desse "túnel", eu tenho o diâmetro da Terra(considerando a Terra um circulo perfeito), não tenho? Então, finge que o pixel que estamos parados é o Brasil e que o nosso circulo é a Terra, vamos cavar um buraco até sair dele. Como que você vai fazer isso? Faz um for ou while, e nesse loop, você vai andar só em Y, ou seja, vai andar pra "baixo", enquanto o conteúdo da coordenada atual for 1, você não chegou no Japão ainda, quando a coordenada for 0, bem vindo à Tokio, se você deu azar, vai dar de cara com Kim jong Yun na Korea do Norte, ai já era kkkk. Dentro desse Loop adicione um contador, esse contador vai ter o diâmetro do circulo quando você chegar em tókio, ou seja, sair do circulo. Se você achou o diâmetro, 99.9% do problema já está morto. Você tem o ponto de tangencia e o diâmetro, com isso, você consegue achar o centro, pois o centro vai ser = (X , coordenada Y do ponto de tangencia menos o raio).
Agora você tem que excluir o circulo que você acabou de calcular o centro, como você faz isso? Muito tranquilo, como eu disse, achou o diâmetro já era. Você vai calcular um quadrado no qual o nosso circulo seja uma circunferência inscrita nele. Partindo do centro, você vai somar e diminuir o raio tanto em X quanto em Y, ou seja, você vai ter quatro coordenadas no final dos calculos. TODOS os pontos dentro dessa área delimitada por essas coordenadas vão ser setados pra 0, ou seja, nosso circulo foi excluído e podemos fazer tudo isso de novo para o próximo.

Sacou a ideia? Vai andando pixel a pixel até tocar o circulo, NECESSARIAMENTE o primeiro ponto que você tocar vai ser o mais "alto", ou seja, se você andar pra "baixo" vai chegar na outra extremidade percorrendo todo o diâmetro.
Abraço!
Comment: valeu cara, deu um pouco mais de trabalho, porque os círculos não eram perfeitos, mas com essa ideia deu certo.
Answer: Acho que tudo é simples, muita complicação!!
//************************************************************
//* (hBitmap, x, y, Width, Height)  imagem em preto e branco.
//************************************************************

HB_FUNC (BT_BITMAPBLACKONWHITE)
{
   HDC memDC;
   HBITMAP hBitmap;
   register INT x, y;
   INT x1, y1, Width, Height, c;
   COLORREF nCor;

   hBitmap  = (HBITMAP)  hb_parnl (1);
   x1       = (INT)      hb_parnl (2);
   y1       = (INT)      hb_parnl (3);
   Width    = (INT)      hb_parnl (4);
   Height   = (INT)      hb_parnl (5);

   memDC = CreateCompatibleDC(NULL);
   SelectObject(memDC, hBitmap);

   for (y = y1; y < (y1 + Height); y++)
   {
     for (x = x1; x < (x1 + Width); x++)
     {

       nCor = GetPixel(memDC, x, y);

       if (nCor < RGB(50,50,50))
       {
         SetPixel(memDC, x, y, RGB(255,255,255));
       }
       else if (nCor > RGB(60,60,60) & nCor <= RGB(255,255,255))
       {
         SetPixel(memDC, x, y, RGB(0,0,0));
       }
       else
         SetPixel(memDC, x, y, RGB(255,255,255));
     }
   }

   DeleteDC(memDC);
}

código completo em C.
Sem complicações e cálculos. Lembre-se teoria é uma coisa pratica é outra.
Comment: Não postei o código completo. te explico!.faço um loop pegando os pixel, depois comparo eles entre eles mesmos, o pixel que não estiver dentro de um valor RGB, é alterado para a cor que eu quiser, transfor. uma imagem complexa em preto e branco. existe outro loop que escreve em preto e branco. manipular cada pixel.Estou estud. sobre reconhec.facial, estou desenv. o meu próprio código. e vc não vai acreditar a linguagem é CA-Clipper, C, e Harbour Minigui.PS:vc pegar pixel por pixel funciona mais fica muito lento, então vc deve utilizar step, fazendo com que a leitura e escrita fique mais rápido
Comment: Como posso colocar exemplos, coloquei minha imagem em 0 e 1. onde posso colocar o executável, para que execute, para ver o resultado. me de umas dicas, sou novo neste site!!! Executável vacinado e pequeno. Windows 32 bits.


Can you bound the third moment from the second moment?

Question: Suppose $X$ is a random real variable with zero mean and finite second moment $\langle X^2\rangle$. Under what conditions can we give a bound (upper/lower) for the third moment $\langle X^3\rangle$?
Comment: What kinds of "conditions" do you have in mind?  You need *something,* because the third moment can be undefined (consider the Student $t$ distribution with 3 df, for instance), it can be infinite, and it can be negatively infinite: there are no general bounds.
Comment: That actually answers my question, thanks @whuber
Answer: There is no such universal bound.
Here is a cute fact to put this question in perspective:

Given real numbers $k^\prime \gt k \ge 0,$ there exist random variables $X$ that have a finite $k^\text{th}$ moment but infinite $k^{\prime\,\text{th}}$ moment.

It doesn't matter whether we consider these raw moments or central moments: see How to prove that the $r^\text{th}$ raw moment exists if and only if the $r^\text{th}$ central moment exists?.
To prove the fact, let $X$ be supported on $[1,\infty)$ with a density proportional to $(1 + x^2)^{-(k^\prime+1) /2}.$  Since $k^\prime \gt 0,$ this density integrates to a finite number, showing such distributions exist.
Because $1+x^2 \gt x^2,$
$$\mu_{k}(X) \propto \int_1^\infty \frac{x^k\,\mathrm{d}x}{(1 + x^2)^{(k^\prime+1)/2}} \le \int_1^\infty x^{k - k^\prime - 1}\,\mathrm{d}x = \frac{1}{k^\prime - k} \le \infty$$
but since $1+x^2 \le 2x^2,$
$$\mu_{k^\prime}(X) \propto \int_1^\infty \frac{x^{k^\prime}\,\mathrm{d}x}{(1 + x^2)^{(k^\prime+1)/2}} \gt \int_1^\infty \frac{x^{k^\prime}\,\mathrm{d}x}{(2 x^2)^{(k^\prime+1)/2}} \propto \int_1^\infty \frac{\mathrm{d}x}{x} = \lim_{x\to\infty}\log(x)$$
diverges, QED.
The question concerns the case $k^\prime=3$ and $k=2.$  To apply the foregoing analysis, let $\sigma^k$ be the given $k^{\text{th}}$ moment and let $X$ be the random variable described above.  Then the $k^\text{th}$ moment of $$ \frac{\sigma\,X}{\mu_k(X)^{1/k}} $$ equals $\sigma^k$ but its $k^{\prime\,\text{th}}$ moment is infinite.
Comment: +1, whuber. I changed the linked post's title and hence this minor edit to your post.


Heroku performance measure

Question: I have Java EE application running on heroku. I need measure load of server CPU. app was runing on my virtual server, I used command top. Can I measure load of CPU on heroku server? I need measure server CPU load when I send a lot of requests by JMeter from my desktop.
Answer: You could go for simply installing a Heroku addon designed for application performance monitoring. You can find a list of such addons in the Analytics section here.
I would personally use New Relic for performance monitoring and resource usage. Just call heroku addons:add newrelic in your command line and it installs automatically.
Comment: +1, this looks a better and right solution than mine.
Answer: Is it possible to enable snmp port on heroku server? With that, you can install mrtg or other similar tools on your client, and get CPU load, memory usage, network usage and other information.
MRTG:  A tool to monitor the traffic load on network-links. Raw data collected via SNMP.

Heroku performance measure


Checking a property that involve vector fields

Question: I´d like to know if it is valid this solution. I think not, but I can´t find the reason.
TASK:
Let $h : \mathbb{R}^n \to \mathbb{R}$ be a differentiable map. For $p \in \mathbb{R}^n$ critic point of $h$ consider $$\phi : T_p(\mathbb{R}^n) \times T_p(\mathbb{R}^n) \to \mathbb{R}$$ $${([((\mathbb{R}^n,Id),v)],[((\mathbb{R}^n,Id),w)])} \mapsto X(Y(h))(p)$$ being $X,Y$ vector fields of $\mathbb{R}^n$ such that $$X(p)=(p,[((\mathbb{R}^n,Id),v)]), \hspace{0.4cm} Y(p) = (p,[((\mathbb{R}^n,Id),w)]).$$ Prove that $\phi$ is bilinear.
POSSIBLE SOLUTION:
Let be $\xi_1 = [((\mathbb{R}^n),Id), v_1)]$, $\xi_2 = [((\mathbb{R}^n),Id), v_2)]$ and $\nu = [((\mathbb{R}^n),Id), w)]$. We get $$\phi(\xi_1 + \xi_2,\nu) = X(Y(h))(p) = \partial (Y(h) \circ Id^{-1})_p(v) = \partial (Y(h))_p(v) = \partial (Y(h))_p(v_1+v_2) = \partial (Y(h))_p(v_1) + \partial (Y(h))_p(v_2) = X_1(Y(h))+X_2(Y(h)) = \phi(\xi_1,\nu) + \phi(x_2,\nu).$$
It is valid?
Thanks!
Answer: I suppose that $X(Y(h))(p) = X(p)(Y(h)) \neq X(Y(h)(p))$, since $Y(h)$ is a differentiable map different to $Y(h)(p) \in \mathbb{R}$ is a real number.


Is it possible to set a top-level only/except in a .gitlab-ci.yml file?

Question: I have three stages in my CI file, they all have only/except like this:
<code>test:
  only:
    - tags
  except:
    - branches
  script:
    - npm run test
</code>
Seems redundant to have the only/except in three places. Is there a way to set this at the top level of the script config? Don't see anything like that in the docs.
Answer: You can use the map merging feature: https://docs.gitlab.com/ee/ci/yaml/#special-yaml-features
<code>.job_template: &job_definition
  only:
    - tags
  except:
    - branches

test1:
    <<: *job_definition
    script:
      - npm run test

test2:
    <<: *job_definition
    script:
      - # ...
</code>
Comment: Amazing feature!


Question about R-algebra "generated by"

Question: Background:
In the context of Atiyah-Macdonald (so all rings are considered commutative and unital) an $R$-algebra, $A$, is defined to be a ring $A$ with a homomorphism $f\colon R \to A$. As mentioned in the text, the point is that the homomorphism induces a compatible $R$-mod structure on the ring $A$.
First, I believe that this definition actually defines an associative algebra but that is not my point here. I have also seen the alternate definition of an $R$-algebra that does not mention the homomorphism $f$, but simply requires that $A$ have both a ring structure and a compatible $R$-mod structure. That is fine with me.
Question:
Out of the three objects rings, modules, and algebras, the last is the one that I have the least experience with. Something that is particularly confusing to me is the notion of an algebra being generated by a set of elements. The way this "generating" process is defined is not intuitive to me. The notions of an ideal being generated by a set of elements or a module generated by a set of elements seems to be the intuitive process to me, just analogues of 'span' from linear algebra - taking linear combinations with coefficients from the appropriate place. This is what I naturally think of anytime an algebraic object is said to be generated by a set of elements. However, Atiyah-Macdonald defines the $R$-algebra $A$ to be generated by a set $B$ if every element of $A$ can be expressed as a polynomial in the elements from $B$ and coefficients in $f(R)$ (if the definition not mentioning the homomorphism $f$ is used, define $f(r) = r\cdot 1$, then use $f(R)$ as stated). It seems to mean that, if you wanted a explicit operational description of "algebra generated by", it would be something like:
The $R$-algebra $A$ is generated by $\{x_0\}$ if for all $a \in A$, $$a = \sum_{i=0}^n r_i x_0^n$$ for some $n \in \mathbb{N}$ and $r_i \in f(R)$.
My question is why is the notion of 'generated' for algebras so much different than the notion of that algebra being generated as a module? For example, $k[x]$ requires infinitely many generators as a $k$-mod but requires only one as a $k$-algebra? I suspect that my lack of experience with algebras, and how they are used, is why it is not intuitive to me. The notion for algebras seems to be connected to integral dependence?
Comment: The notion of generation is pretty much the same for groups, modules and algebras. $x_1,\ldots,x_n$ generate a group if the only subgroup containing them is the whole group. $x_1,\ldots,x_n$ generate a module if the only submodule containing them is the whole module. $x_1,\ldots,x_n$ generate an algebra if the only subalgeba containing them is the whole algebra.
Comment: "I believe that this definition actually defines an associative algebra" Just as you observed that in the context of that book rings are commutative, well, algebras are associative. If that were your point, it would be a silly one!
Comment: The definition A-M uses and the alternative one you mentioned are exactly the same. If you have an algebra on the sense you mean, defining f(x)=x.1 (where on the right the dot denotes the module structure) you get an algebra in the book's sense.
Comment: Right, I just mentioned the bit about associative because I tagged this 'algebras' and was referring to the object as an algebra, so I was just letting the more informed reader know that I am aware I am dealing with a smaller class of objects, associative algebras, and to the second comment, yes, I also mentioned I have seen the proof of equivalence and I even mention defining the morphism $f(x) = x\cdot 1$ if the alternate definition is used.
Answer: You're thinking about "generating" wrong; it doesn't have anything particularly to do with linear combinations.  A subset $S$ of an algebraic structure $A$ "generates" an element $a\in A$ if you can produce $a$ by starting with the elements of $S$ and repeatedly using the operations of $A$.
In the case that $A$ is an $R$-module, the operations you have are addition and scalar multiplication by any element of $R$.  So for instance, if $s_1,s_2,\dots$ are elements of $S$ and $r_1,r_2,\dots$ are elements of $R$, some elements you can generate from $S$ are: $$s_1+s_2$$
$$r_4(r_1r_2(s_1+r_3s_2)+s_3)$$
$$r_3(s_1+s_2+r_1s_3+r_2s_4)+r_4r_5s_5+s_1$$
At a first glance, these might look a lot more complicated than just linear combinations!  However, since scalar multiplication is associative and distributes over addition, you can always reduce any such combination to a linear combination.  For instance, the second element above is equal to the linear combination $$r_5s_1+r_6s_2+r_4s_3$$ where $r_5=r_4r_1r_2$ and $r_6=r_4r_1r_2r_3$.
In the case that $A$ is an $R$-algebra, you have one additional operation: besides addition and multiplication by elements of $R$, you can also multiply elements of $A$ together.  So some elements which you can generate from $S$ are: $$s_1s_2$$
$$r_1(s_1+s_2)(s_3+r_2s_4)+s_4^3$$
$$(r_1s_1+r_2s_2)^2s_3+s_1$$
This time we can't just reduce everything to linear combinations of the $s_i$, because we might be multiplying multiple $s_i$ together, as in $s_1s_2$.  However, we still can use the ring axioms to reduce everything down to a linear combination of products of the $s_i$.  For instance, the second element above is equal to $$r_1s_1s_3+r_3s_1s_4+r_1s_2s_3+r_3s_2s_4+s_4^3$$ where $r_3=r_1r_2$.  This is a linear combination of the products $s_1s_3, s_1s_4,s_2s_3,s_2s_4,$ and $s_4^3$.  Another name for "linear combination of products" is "polynomial".  Indeed, the expression above is just $f(s_1,s_2,s_3,s_4)$ where $f$ is $$f(x,y,z,w)=r_1xz+r_3xw+r_1yz+r_3yw+w^3,$$ a polynomial in four variables with coefficients in $R$.  So in general, the elements of the algebra generated by $S$ are all elements of $A$ which can be written as polynomials (in multiple variables) with coefficients in $R$ evaluated at elements of $S$.
Comment: Yeah, I glossed over the fact that "products" include the empty product which gives you the unit element.
Comment: @AlJebr: Yes, that's correct.
Comment: Wow, you actually basically cleared it up for me in the first paragraph. "using the operations of $A$". So in the $k[x]$ example we can get the $x^2$ with the ring structure, but just using the module structure we need $x^2$ as a separate generator. Of course, I believe we have to define the empty product in order to generate the ground field for this example to work but thats a nonissue. I don't know why this wasn't more obvious to me? I think part of my issue was that I was also comparing to the definition of an ideal
Comment: "generated by", where we do have the ring structure but it is actually defined more like the 'linear combination' method I mention. However, you could define it more like how it is for an algebra, it just wouldn't matter since you can build higher powers anyways by choosing the generating as the ring element you are multiplying by.
Comment: Great answer by the way, still not sure what the heck I was thinking but this is exactly what I needed to clear little road block up.
Comment: No worries, even A-M doesn't mention it (that I have noticed). Thanks again!
Comment: @EricWofsey Does this mean that if $A$ is a finitely generated $R$-algebra, then $A =R[a_1, \dots, a_n]$ (or maybe just isomorphic?) for some elements $a_i \in A$?
Comment: @AlJebr: Well, what do you mean by $R[a_1,\dots,a_n]$?  I would say that means the subalgebra of $A$ generated by $a_1,\dots,a_n$, so that would just be the definition of being finitely generated.
Comment: @EricWofsey Ok. I think I see. By $R[a_1, \dots, a_n]$ I mean polynomials over $R$ in $n$-variables evaluated at the $a_1, \dots, a_n$. So, if $A$ is finitely generated as an $R$-algebra by $a_1, \dots, a_n$, then $A = R[a_1, \dots, a_n]$ ?
Answer: I'll take a couple of examples that will make things clearer – I hpe.
Consider the fields $\mathbf Q$ and $\mathbf Q(i)\subset \mathbf C$. It is finitely generated $\mathbf Q$-algebra, as it is the same as $\mathbf Q[i]$, and we have a surjective $\mathbf Q$-algebras homorphism:
\begin{align}
\mathbf Q[X]&\longrightarrow \mathbf Q[i],\\
X&\longmapsto i.
\end{align}
Its kernel is the minimal polynomial of $i$ over $\mathbf Q$, so $\mathbf Q(i)=\mathbf Q[i]\simeq\mathbf Q[X]/(X^2+1)$. It is finitely generated as a $\mathbf Q$-vector space (of (dimension $2$) since, as you know,  all powers of $i$ are $1,i,-1$ or $-i$.
On the other hand, you may consider the $\mathbf Q$-algebra $\mathbf Q[\pi]$.  It is finitely generated and  you may define a similar surjective homomorphism:
\begin{align}
\mathbf Q[X]&\longrightarrow \mathbf Q[\pi],\\
X&\longmapsto \pi.
\end{align}
This one is  injective, since $\pi$ is not an algebraic number. As a consequence $\mathbf Q[\pi]$ is isomorphic to the polynomial ring $\mathbf Q[X]$, so that, as a $\mathbf Q$-vector space, it is infinite-dimensional.

Question about R-algebra "generated by"


Is self-selection for a treatment a problem after all?

Question: I have devised the following R code.
In this simulated dataset, we know that a certain treatment (e.g. a degree, an education) increases income by $ 1,000. Income is also caused by age.
In the first part, units are randomly assigned into treatment. Each unit has a 50% probability to receive the treatment, regardless of his/her age.
In the second part, subjects self-select into treatment: younger units have a higher probability to receive treatment than older ones. We can check that the units that receive treatment are younger: we do a boxplot, and a t-test rejects the null hypothesis that the average age is the same across the treated and non treated groups.
By running an OLS, I am able to estimate the effect of treatment on income in both cases, so also in the second case where people self-select themselves into treatment.
In the second part, can I infer from the OLS that the treatment causes an increase in income by $ 1,000?
Is self-selection a problem for that inference?
<code>N = 100000
age_coeff = 100
treated_coeff = 1000

# PART 1: RANDOM SELECTION INTO TREATMENT
df = data.frame(id=seq(from=1, to=N, by=1))
df$age = runif(N, min=0, max=100)
# size -> number of trials (zero or more)
df$treated = rbinom(N, size=1, prob=0.5)
df$treated = df$treated==1
e = rnorm(N, mean=0, sd=50)

# Check age is the same across the two groups
r <- aggregate(df$age, list(df$treated), FUN=mean)
colnames(r) <- c("Treated", "Age")
print(r)
t.test(df$age[df$treated], df$age[!df$treated])

df$income = 0 + age_coeff*df$age + treated_coeff*df$treated + e
income.lm = lm(income~age+treated, data=df)
summary(income.lm)

# PART 2: SELF-SELECTION INTO TREATMENT
library(sigmoid)
df$ptreated = logistic(50-df$age)
df$treated = rbinom(N, size=1, prob=df$ptreated)
df$treated = df$treated==1

# Check if selection into treatment is effective
boxplot(df$ptreated~df$treated)
r <- aggregate(df$ptreated, list(df$treated), FUN=mean)
colnames(r) <- c("Treated", "p(Treated)")
print(r)

# Check if age differs in the two groups
r <- aggregate(df$age, list(df$treated), FUN=mean)
colnames(r) <- c("Treated", "Age")
print(r)
boxplot(df$age~df$treated)
t.test(df$age[df$treated], df$age[!df$treated])

df$income = 0 + age_coeff*df$age + treated_coeff*df$treated + e
income.lm = lm(income~age+treated, data=df)
summary(income.lm)

# Check
df$age50 <- df$age >= 50
table(df$age50, df$treated, dnn=c("Age50", "treated"))
```
</code>
Comment: @robertspierre I cannot image scenario where you would know them "all" and there would be no measurement error at all, also regression analysis has nothing to do with causality. So are you asking about real data, or this is a completely theoretical question assuming a hypothetical scenario?
Comment: It's no problem if you don't care about drawing valid inferences about the treatment!  But in what sense is this an "econometrics estimate"?
Comment: @whuber The coefficient I obtain in the second part (where subjects self-select into treat) is correct. I'm changing the last question with the following: In the second part, can I infer that treat increases outcome by 1000?
Comment: Your code does not run: what is `age_coeff` ?
Comment: But the problem in reality with self-selection is that not only can factors you know about affect self-selection (age here), but so can factors you do not know about and cannot control for
Comment: @Henry I have modified the code to make the selection even less deterministic. `age_coeff` is the effect of age on income and is defined at the beginning of the code
Comment: @Henry If I know _all_ the factors that determine both selection into treatment and outcome (they are confounders of treatment and outcome, as `age` is in this case), and I can measure them with _no_ measurement error, and I put them into my OLS estimates as covariates, the coefficient I obtain for the treatment variable is the causal effect of the treatment on the outcome, right?
Comment: @Tim I guess my question would be: in what cases self-selection into treatment is a problem for the estimation of the causal effect?
Comment: I'm unclear if the dependent variable here is a given treatment outcome, or simply if they opted for treatment or not.
Comment: @GreggH I would be interested in the answer for both cases: when we assign treatment to younger persons, and when younger persons self-select into treatment. I have also modified the text of the question to make it clearer
Answer:
A t-test doesn't construct a certainty interval around the actual treatment difference, even the CI can't be interpreted that way. No NHST method will tell you the actual treated difference is 1000. However, both models should be powered to reject the null hypothesis that income does not differ by treatment assignment.

In the self-selection scenario, age is a confounder. But in your inference you have adjusted for age to eliminate confounding. Therefore, the linear model is expected to adjust the counfounding effects. The model is not a t-test but an adjusted linear model. Had you used the t-test the result would be biased.

In practice the issue doesn't boil down to known confounders with known functional forms, but unknown confounders with unknown functional forms. You are often presented with data that failed to collect certain values, or at the appropriate times, or with desired precision, subject to recall bias, reverse confounding, non-ignorable missingness. The methods for handling these cases when the precise mechanism is known is almost irrelevant, but understanding how to assess a data analysis and put together results from multiple studies and ascertain the impact of confounding is a subtler and more valuable art.
Comment: I don't understand (1) and (2). I used a t-test ONLY to check whether the mean age is different across the treated and non treated groups. I used the t test to check that the distribution of the confounder is different between the treated and not treated group. I did NOT use a t-test to assess the treatment difference, the difference in the outcome (which is income in the example) between treated and not treated), I used an OLS model for that. I understand (3) instead. Thanks
Comment: @robertspierre in your own words, "In the second part, can I infer that treatment increases outcome by 1000?" and to that the answer is no. For 2, admittedly I was confused what your outcome was here. In general you do not test for confounding. You can't distinguish a variable as a collider, confounder, or mediator by tests: a variable is identified as a confounder a priori - you don't test for it.
Comment: Yes the text was not very clear. The answer to my question is "no" due to reason (3)? There may be other confounders I don't observe, or reverse causality bias? I was not trying to control if the variable is a confounder. Given that I hypothesize a priori the variable is a confounder, I was trying to check whether it is randomized across treatment vs non treated group
Comment: I have modified the text of the question to make it clearer. If your (3) is the answer I will accept it


How to perform a 4 by 4 mixed ANOVA  with between- and within-subjects contrasts using R?

Question: Beginner user of R here struggling with a repeated measures ANOVA.
I have a dataset that consists of one between subjects factor with 4 levels (coded in a single variable called 'groups'), and one within subjects factor with 4 levels (coded in four separate variables 'DV1', 'DV2', 'DV3', 'DV4').
I have the following objectives:

Run an overall repeated measures ANOVA.
Compare groups using custom contrasts (as in an LMATRIX command in SPSS).
Compare different levels of DV using custom contrasts (as in an MMATRIX command in SPSS).
Do a combination of 2) and 3) simultaneously so I am comparing only certain groups at certain levels of the within-subjects factor.
Run a set of contrasts that do NOT sum to zero.

I know I can do this in SPSS without much of a problem, but I can't get a clear idea of how to do this in R. I've seen how parts of this could work in different packages, but I have not so far seen how this could work within one procedure or a set of related procedures in R.
Answer: A sketch of one solution (for another see below):

Data needs to be in the long format (i.e., on value per row) instead of in the wide format as in SPSS (i.e., one subject per row), see the reshape package, or ?reshape. That includes that there needs to be a variable indicating the subject identifier (i.e., subject id).
All factors (including the subject identifier) must be of class factor (run str on your data frame to check this). If you don't do this your results will be wrong.
If you want to obtain type-III sums of squares set the default contrasts to effect coding:
options(contrasts=c("contr.sum","contr.poly"))
Specify the desired model with lme from the nlme package (install and load the package beforehands via install.packages("nlme") and library(nlme)) using a compund symmetric correlation structure. See the answer and especially my comment to the accepted answer to this question. In your case that could be something like (if you would have provided sample data, which is strongly recommended, you would have received the correct code):
my.anova <- lme(dv ~ group*within, data = your.df, random = ~1|id, correlation = corCompSymm(form = ~1|id))
Use the generic anova function to obtain the anova table (see ?anova.lme):
anova(my.anova)
To obtain type-III sums of squares use the anova command with argument type set to "marginal" (this only works if contrasts are set to effects coding, see point 3):
anova(my.anova, type = "marginal")
The fitted object of type lme now allows diverse functions to perform contrasts. The most flexible solution (but a rather unhandy one) is the L argument in a call to anova.lme (see again ?anova.lme).
Other solutions also require a fitted lme object as an argument:
Also very flexible is the estimable function from the gmodels package. This package also offers the fit.contrasts function.
The multcomp package allows contrasts using alpha-error adjustment (but you can only perform contrasts using one of your factors), using the glht function.
A new and promising approach is the contrast package, however, so far it does not seem to privde all possible contrasts.

An alternative solution is to use standard ANOVA via the combination of afex and lsmeans as outlined in the afex-vignette.
Comment: (+1) Great and very instructive response. Waiting for the blog post...
Comment: `?anova.lme` doesn't work for me and `methods(anova)` lists it as a non-visible function.
Comment: @John Did you load `nlme` before? If not, run `library(nlme)`, then it should work. If it is still not working, `install.packages("nlme")` first.
Comment: ah... I hadn't apparently, only lme4
Comment: @ Henrick, I can go up to step 5, but I am not able to set up contrasts comparing specific between-subjects factors at specific within-subject factor levels. All of the online documentation I've seen has to do with testing differences at different levels of between-subjects factors. Do you have any example code? I can just wait for the blog post if you are going to cover it there.
Comment: @aquadhere Actually there should be no difference on whether a variable is between- or within-subjects. If your within variable is just a single factor or column in your data frame (which it should be). Then just use this one as if it would be between.
Comment: @Henrick - ah yes, I figured as much, but all the examples I've seen only test one factor - is there a way to test two factors simultaneously? That's where I get stuck.
Comment: @aquadhere that is where it gets tricky. As far as I get it, the `L` argument and `estimable()` should be able to do so. Also the `contrast` package. The best thing is, try to understand the `L` argument. Therefore have a look at `summary(my.anova)`.
Comment: +1. But "I will update this answer with a link to this post when it is done (hopefully this week)" line sounds a bit strange when it's almost 5 years later :-) Apart from that, here you essentially suggest to use a mixed model with enforced compound symmetry to simulate the RM-ANOVA behaviour. I know they give identical results when the data are balanced (same $n$ in all ANOVA cells), but do they also coincide in the unbalanced case? I am reading up on that, but I was under impression that they do NOT coincide then (I am not sure though).
Comment: @amoeba I have removed the reference. It is true, that was a little odd. I also have meanwhile found a better solution outlined in the [afex vignette](https://cran.rstudio.com/web/packages/afex/vignettes/anova_posthoc.html).
Comment: Thanks @Henrik. I would still be interested in your answer to my question about equivalence of RM-ANOVA and a mixed-model when the data are unbalanced. Or perhaps you can point me to some references where I can read up on that?
Comment: @amoeba no, they are not equivalent due to enforcing a correlation above 0, see [here](http://stats.stackexchange.com/q/14088/442). Interestingly, the same also applies to `lme4` models, see [here](http://stats.stackexchange.com/q/117660/442).
Comment: @Henrik, yes, I saw these threads and even gave bounty to Aaron's answer in the first one. This issue of positive correlation is general and relates to both balanced and unbalanced designs. I am asking about a situation in which this is not an issue (either because correlations happen to be positive or because lmer is run with enforcing compound symmetry that allows negative correlations). So, in this situation when data are balanced the results of RM-ANOVA and of mixed-model are the same (by "results" I guess I mean F- and p-values). I am wondering if this is still true for unbalanced data.
Comment: @amoeba Sorry, I cannot give you an authoritative answer on this.


SQL Server Month Totals

Question: SQL Server newbie
The following query returns SRA by Student and month only if there is a record for a student in Discipline table. I need a query to return all students and month totals even if there is no record for student in Discipline table. Any direction appreciated
SELECT  TOP 100 PERCENT MONTH(dbo.Discipline.DisciplineDate) AS [Month], dbo.Discipline.StuId, dbo.Stu.Lastname + ',' + dbo.Stu.FirstName AS Student,
    SUM(CASE WHEN Discipline.SRA = 1 THEN 1 END) AS [Acad Suspension], SUM(CASE WHEN Discipline.SRA = 2 THEN 1 END) AS Conduct,
    SUM(CASE WHEN Discipline.SRA = 3 THEN 1 END) AS Disrespect, SUM(CASE WHEN Discipline.SRA = 4 THEN 1 END) AS [S.R.A],
    SUM(CASE WHEN Discipline.SRA = 5 THEN 1 END) AS Suspension, SUM(CASE WHEN Discipline.SRA = 6 THEN 1 END) AS Tone
FROM dbo.Discipline INNER JOIN
     dbo.Stu ON dbo.Discipline.StuId = dbo.Stu.StuId
GROUP BY dbo.Discipline.StuId, dbo.Stu.Lastname, dbo.Stu.FirstName, MONTH(dbo.Discipline.DisciplineDate)
ORDER BY Student

Answer: You need to change the INNER JOIN onto dbo.Stu to a LEFT JOIN:
   SELECT MONTH(d.disciplinedate) AS [Month],
          d.StuId,
          s.Lastname + ',' + s.FirstName AS Student,
          SUM(CASE WHEN d.SRA = 1 THEN 1 END) AS [Acad Suspension],
          SUM(CASE WHEN d.SRA = 2 THEN 1 END) AS Conduct,
          SUM(CASE WHEN d.SRA = 3 THEN 1 END) AS Disrespect,
          SUM(CASE WHEN d.SRA = 4 THEN 1 END) AS [S.R.A],
          SUM(CASE WHEN d.SRA = 5 THEN 1 END) AS Suspension,
          SUM(CASE WHEN d.SRA = 6 THEN 1 END) AS Tone
     FROM dbo.Discipline d
LEFT JOIN dbo.Stu s ON s.stuid = d.stuid
 GROUP BY d.StuId, s.Lastname, s.FirstName, MONTH(d.DisciplineDate)
 ORDER BY Student

The LEFT JOIN means that whatever table you're LEFT JOINing to might not have records to support the JOIN, but you'll still get records from the base table (dbo.Discipline).
I used table aliases - d and s.  Less to type when you need to specify references.
Comment: OK  Now I am getting a record for all students. Thank you for your help and pointers. This site is great
Answer: generate a series of months, join discipline to that.

SQL Server Month Totals


How do I add  tag in a table using ajax_command_replace()?

Question: I am using ajax_command_replace() in an AJAX callback. It works fine to remove the full
 markup, but I only need to add a row to the table after a form is submitted, when data exist.
What is the correct code to achieve this?
Answer: The HTML wrapper you will be replacing sounds like it includes the whole table, so when you return the markup for the new table row, it will replace all the content in that selector/wrapper. You need to include the markup for your whole table including the newly added row, not just the new row.
Answer: ajax_command_replace() is the function to replace elements from HTML markup. The first argument it accepts is the selector for the elements to replace, which means that if that selector is also for a
 element, that element will be replaced.
To add a row into a table, you need either ajax_command_append() or ajax_command_prepend(). For example, given the following markup, ajax_command_append('#contacts', $row) would add $row at the table bottom, and ajax_command_prepend('#contacts', $row) would add $row at the top. (The example markup has been taken from HTML Tables, on the W3Schools site.)

    Company
    Contact
    Country

    Alfreds Futterkiste
    Maria Anders
    Germany

    Centro comercial Moctezuma
    Francisco Chang
    Mexico


Join dataframes from loop output using Pandas

Question: I want to create one dataframe from dataframes that are output from a for loop. Each iteration of the loop produces the variable called result.
 result = pd.concat([df1, df2], axis=1)

I want to create one dataframe that is the combination (adjacent to one another) of each result dataframe.
before the loop I created a variable combined_results
combined_results = pd.DataFrame()

And at the end of the loop tried to store each result in that variable, using append, concat etc but cant get it to work.
    combined_results = combined_results.append(result)

This appends vertically not horizontally and tried concat as per result, but no luck
Answer: You are already on the right track with the pd.concat([...], axis=1) pattern you use inside the loop. The trick is to save all your partial DataFrames into a list, and then save concatenation until the end.
I use a pattern like this:
combined_results = []
for i in range(5):
    df1 = pd.DataFrame({f'x{i}': ['a', 'b', 'c']})
    df2 = pd.DataFrame({f'y{i}': [i, i*2, i*3]})
    result = pd.concat([df1, df2], axis=1)
    combined_results.append(result)

# Use axis=1 as before to join horizontally instead of vertically.
combined_results = pd.concat(combined_results, axis=1)

combined_results
#   x0  y0 x1  y1 x2  y2 x3  y3 x4  y4
# 0  a   0  a   1  a   2  a   3  a   4
# 1  b   0  b   2  b   4  b   6  b   8
# 2  c   0  c   3  c   6  c   9  c  12

Comment: Great, thanks for the help and explanation. Just what I needed
Answer: in your loop you can do this:
df_results = []

for ... in ...:
    result = pd.concat([df1, df2], axis=1)
    df_results.append(result)

df = pd.concat(df_results)

If you show us from where you get df1 and df2, maybe we can improve this
Comment: Thanks! was correct also, but the other answer's explanation was helpful


Quadratic mean convergence of a biased coin using conditional expectation

Question: I'm a master's degree student and after a lot of research and some days trying I still can't get the answer for a question proposed by my statistics professor.
 Consider a sample of coin tosses:
$$X_1, X_2, ..., X_n | P=p \sim \text{IID Bern}(p),$$
where the coin tosses are conditionally independent Bernoulli random variables with probability given by $P$ (a head on the coin is a one and a tail is a zero).  Denote the sample mean of the coin tosses as:
$$S_n = \frac{1}{n} \sum_{i=1}^n X_i.$$
My professor claims that $\mathbb{E}(S_n) = \mathbb{E}(X_1) = \mathbb{E}(P)$, and says that it is possible to use conditional expectation to prove this.  He also asks me to prove that $S_n$ converges to $P$ in quadratic mean (not just to its expected value).  How can I prove this?
Comment: Welcome to CV! You may have more luck getting an answer if you describe the approaches you've tried and where you've gotten stuck.
Comment: @frock81: I have taken the liberty of making a major edit to your question to set the mathematics out more clearly.  Please check that this correctly captures what you were asking, and that I have not inadvertently changed your meaning.  (Incidentally, your notation is strange --- usually you would use $\bar{X}_n$ to denote a sample mean, not $S_n$.)
Answer: This is a classic case in Bayesian analysis, where you have an exchangeable sequence of observations that are conditionally independent given the underlying parameters, but they are not marginally independent when you do not condition on the parameters.  You can find simple explanations of this phenomenon, discussing the Bernoulli model as an example, in O'Neill (2009).  You can also find some more aspects of Bayesian analysis of this Bernoulli model in a related series of papers (O'Neill and Puza 2005, O'Neill 2012, and O'Neill 2015).
From the properties of the Bernoulli distribution, you have the conditional mean $\mathbb{E}(X_i|P) = P$.  Using the law of iterated expectations, you have:
$$\begin{equation} \begin{aligned}
\mathbb{E}(S_n)
= \mathbb{E} \Big( \mathbb{E}(S_n|P) \Big)
&= \mathbb{E} \Bigg( \mathbb{E} \Bigg( \frac{1}{n} \sum_{i=1}^n X_i \Bigg| P \Bigg) \Bigg) \\[6pt]
&= \mathbb{E} \Bigg( \frac{1}{n} \sum_{i=1}^n \mathbb{E} (X_i | P ) \Bigg) \\[6pt]
&= \mathbb{E} \Bigg( \frac{1}{n} \sum_{i=1}^n P \Bigg) \\[6pt]
&= \mathbb{E} ( P ). \\[6pt]
\end{aligned} \end{equation}$$
Similarly, you have:
$$\begin{equation} \begin{aligned}
\mathbb{E}(X_i)
= \mathbb{E} \Big( \mathbb{E}(X_i|P) \Big)
&= \mathbb{E}(P). \\[6pt]
\end{aligned} \end{equation}$$
This establishes the equivalence of the expected values asserted by your professor.  For the latter problem, you will need to prove that $\mathbb{E}((S_n-P)^2) \rightarrow 0$ as $n \rightarrow \infty.$  I will not give you a solution here, but as a hint, I recommend that you use the law of iterated expectation to get an expression for the expectation in the statement.  You should then be able to show that this expression converges to zero in the limit, which would establish convergence in quadratic mean.
Answer: I think your question is not quite clear.  First, it's hard for me to see how coin flips can not be independent.  Are you saying that somehow the coin flips depend on each other?  What I think that you mean is that the coin flips are not iid (they are independent but not identically distributed).  Here that would mean that the parameter P changes with each flip of the coin.
$$E[S_n] = E[E[S_n|P=p]] = E\left[E\left[\frac{1}{n} \sum_{i=1}^n X_i | P \right] \right] = E[P]$$
The first equality is the law of iterated expectations.  The second is just substituting for $S_n$.  The third is because $X_i \sim Bernoulli(P) \implies E[X_i|P=p] = p$.
For convergence in quadratic mean, you need to show that:
$0 = \lim_{n \rightarrow \infty} E[(S_n - P)^2]$
$=\lim_{n \rightarrow \infty} E[E[(S_n - P)^2|P]]$
$= \lim_{n \rightarrow \infty} E[Var(S_n | P)]$
$= \lim_{n \rightarrow \infty} E\left[Var\left(\frac{1}{n} \sum_{i=1}^n X_i | P) \right) \right]$
$= \lim_{n \rightarrow \infty} E\left[\frac{1}{n^2} \sum_{i=1}^n Var(X_i | P)\right]$
$= \lim_{n \rightarrow \infty} \frac{1}{n} E[P(1-P)]$
$= 0$
Comment: You have answered a question that was not asked.  One way for the conditions in the question to hold is to consider a jar full of coins, each with its own value of $P$.  Draw a coin out of the jar at random.  Flip it $n$ times.  The outcomes are conditionally independent of $P$, but they are not independent.


Unable to record in Jmeter

Question: I want to record a scenario for the a particular site using Jmeter. As per the standard procedure I have changed the proxy settings to localhost:8080.
When I start recording, the page of the particular site does not load.
Note: For normal browsing we use a particular proxy. Please assist on this asap.
Following is the error that I get on the browser.
java.net.ConnectException: Connection timed out: connect at
java.net.PlainSocketImpl.socketConnect(Native Method) at
java.net.PlainSocketImpl.doConnect(Unknown Source) at
java.net.PlainSocketImpl.connectToAddress(Unknown Source) at
java.net.PlainSocketImpl.connect(Unknown Source) at java.net.Socket.connect(Unknown Source)
 at java.net.Socket.connect(Unknown Source) at sun.net.NetworkClient.doConnect(Unknown Source) at
sun.net.www.http.HttpClient.openServer(Unknown Source) at
sun.net.www.http.HttpClient.openServer(Unknown Source) at sun.net.www.http.HttpClient.
(Unknown Source) at sun.net.www.http.HttpClient.New(Unknown Source) at
sun.net.www.http.HttpClient.New(Unknown Source) at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) at
sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) at
org.apache.jmeter.protocol.http.sampler.HTTPJavaImpl.sample(HTTPJavaImpl.java:485) at
org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:62) at
org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1060)
 at org.apache.jmeter.protocol.http.proxy.Proxy.run(Proxy.java:238)

Regards,
Kiran
Comment: If you are using any proxy server - Check whether you have not bypassed localhost in proxy settings. In latest version of Jmeter(2.11) you can use test script recorder template to create test plan .
Answer: you are testing an application behind a proxy server. so, you have to specify this when running JMeter. To solve your problem you must follow those steps:

Open cmd command ang go under bin folder
type jmeter.bat -H [proxyserver hostname or ip address] -P [proxyserver port]

Example : jmeter.bat -H my.proxy.server -P 8000
For More details:
If you are testing from behind a firewall/proxy server, you may need to provide JMeter with the firewall/proxy server hostname and port number. To do so, run the jmeter[.bat] file from a command line with the following parameters:
-H [proxy server hostname or ip address]
-P [proxy server port]
-N [nonproxy hosts] (e.g. *.apache.org|localhost)
-u [username for proxy authentication - if required]
-a [password for proxy authentication - if required]

Unable to record in Jmeter


Determinant of the real matrix representation of a $\mathbb{C}$-linear map

Question: If $f$ is a complex linear map on $\mathbb{C}^n$ with matrix $A=(a_{i,j})$ then, viewing $f$ as a real linear map on $\mathbb{R}^{2n}$, we find that the real $2n\times 2n$ matrix of $f$ has the form of a block matrix $(\alpha_{i,j})$ where
$$\alpha_{i,j}=\begin{pmatrix}
Re(a_{i,j}) & -Im(a_{i,j}) \\
Im(a_{i,j}) & Re(a_{i,j})
\end{pmatrix}$$
that is, the real matrix has the form
$$\mathcal{A}=\begin{pmatrix}
a & -b & c &-d & e &-f & \dots & \\
b & a & d & c & f & e & \dots & \\
g & -h & i & -j & k & -\ell & \dots & \\
h & g & j & i & \ell & k & \dots & \\
\vdots & & & & & & & \\
\end{pmatrix}$$
I was wondering if there is any way to take advantage of the symmetries of this matrix in order to compute its determinant. I know that the answer should be that $\det \mathcal{A}=|\det A|^2$ but I dont see how to obtain this by a straightforward computation. I tried doing a cofactor expansion but quickly got lost in the messiness of it. Any helpj would be greatly appreciated. Thanks!
Comment: See [*On the Relation between Real and Complex Jacobian Determinants* by Daniel J. Cross (2008)](https://www.physics.drexel.edu/~dcross/academics/papers/jacobian.pdf).
Answer: You can use the determinant formula for block matrices. If $\mathcal{A}$ is a $np \times np$-matrix made out of commuting $p\times p$-blocks, then the determinant of $\mathcal{A}$ is the same as the determinant of the $p \times p$-matrix obtained by computing the determinant of the block matrix considering its entries as the entries of an $n\times n$-matrix.
In your case we have $p=2$ and the blocks commute because complex numbers commute. Since each $2\times 2$-block of $\mathcal{A}$ corresponds to the complex number at the corresponding position in $A$, we already know the $2\times 2$-matrix of which we have to take the determinant: It is simply the $2 \times 2$-matrix representing the complex number $\det(A)$.
So if $\det(A) = a + ib$, then
$$\det(\mathcal{A}) = \det\begin{pmatrix}a & -b\\ b & a\end{pmatrix} = a^2 + b^2 = |\det(A)|^2$$
Edit:
It might be useful to have an example. For $n = p = 2$ the formula boils down to:
$$\det\begin{pmatrix} A & B \\ C & D \end{pmatrix} = \det(AD - BC).$$
Note that the term $AD - BC$ is formally the determinant of a $2 \times 2$-matrix.
Suppose that $A$ is given by
$$A = \begin{pmatrix}1 & i \\ i & i\end{pmatrix}.$$
Then
$$\mathcal{A} = \left(\begin{array}{cc|cc}
  1 & 0 & 0 & -1 \\
  0 & 1 & 1 &  0 \\ \hline
  0 & -1& 0 & -1\\
  1 & 0 & 1 & 0
\end{array}\right)$$
and by the formula above we have
$$\det(\mathcal{A}) = \det\left( \begin{pmatrix}1 & 0 \\ 0 & 1\end{pmatrix} \begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix} -
\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix}
\begin{pmatrix}0 & -1 \\ 1 & 0\end{pmatrix} \right)
= \det \begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix}$$
On a formal level this is similar to computing the determinant of $A$:
$$\det(A) = ( 1 \cdot i - i \cdot i) = 1+i.$$
Also note that in your matrix representation of complex numbers:
$$\begin{pmatrix} 1 & -1 \\ 1 & 1 \end{pmatrix} \hat{=}\; 1+i.$$
Comment: Superb. For future reference, it might me be worth pointing to a proof of the above mentioned property of the determinant, just do a google search with 'John Sylvester Determinants of Block matrices'. Also, some people might wonder why the determinant calculation with the $2 \times 2$ blocks gives the matrix representation of the determinant of complex numbers. That is because sums and products of complex numbers correspond with sums and products of their matrix representation.