What Bayesian test to conduct with one independent variable and two dependent variables?

Question: In my current study I am looking at the effects of creatine monohydrate ingestion on ground reaction force and repeated sprint times. With CM as the independent variable, and ground reaction force and repeated sprint times as the dependent variables, what Bayesian inferential test would be the best to conduct?
Answer: Bayesian inference would, in theory, involve determining the joint probability distribution (JPD) $P(R,S,C)$ that expresses the probability of a certain combination of reaction force ($R$), sprint time ($S$) and creatine monohydrate ingestion ($C$). The JPD over all variables can later be used to infer all other possible conditional probabilities using Bayes's rule. $P(R,S,C)$, can using this same rule, be factorised as $P(R|S,C)\times P(S|C)\times P(C)$. With an eye on what you probably want to infer, you just need to is estimate the probability $P(R|S,C)$ of different reaction force's ($R$) given a certain value for repeated sprint time ($S$) and CM ingestion ($C$). The same can be done for $P(S|C)$.
Estimating these probability distributions can either be done using continuous variables in which you would make certain assumptions about the functional form of your distribution and fit its parameters. Or you could bin your data in discrete intervals and count occurrences per interval.
Once you estimated $P(R|S,C)$ and $P(S|C)$, the distribution of reaction force ($R$) given a certain CM ingestion ($C=c$) (which can be used to infer it's most probable value) is expressed as :
$$ P(R|C=c) = \frac{P(R,C=c)}{P(C=c)}=\frac{\sum_S P(R,S,C=c)}{P(C=c)}=\sum_S P(R|S,C=c)\times P(S|C=c) $$
Or infer the complete distribution over both $R$ and $S$ given given a certain CM ingestion ($C=c$)
$$ P(R,S|C=c) = \frac{P(R,S,C=c)}{P(C=c)}= P(R|S,C=c)\times P(S|C=c) $$
Using these distributions, you can also determine the standard deviation, confidence intervals, etc.
Side note: You could also choose a different factorisation that might better suite your data, in the end the probabilities you infer will stay the same, these "factors" just serve as a way to make estimating your JPD practical. I did not assume any independence between $R$ and $S$, as this seems very unlikely. This would however ease the computation of $P(R|S,C)$ which would then  $=P(R|C)$.
Answer: One promising approach is to formulate the two marginal Bayesian models just as we always do, then to model the dependency of the two outcomes using a copula.  That way posterior inference can be done marginally as always but you could also make efficient joint statements regarding unions or intersections of conditions (assertions) involving the two outcomes and their association with the independent variable.  I've been looking for a good example of specifying a copula model in Stan --- I hope someone can find one and report back.
Answer: Bayes' rule states
$$
P(A|B) = \frac{P(B|A)P(A)}{P(B)},
$$
where $P(A|B)$ and $P(B|A)$ are conditional probabilities. For instance, $P(A|B)$ is the probability of $A$ occurring given that $B$ is observed.
Bayesian inferencing is a method of inference (there are others) that applies Bayes' rule in order to update our beliefs of a certain hypothesis given observed (experimental) parameters. Technically, it involves calculating the conditional posterior distribution; that is, the conditional distribution of the random variable of interest post-observation.
Often the most contentious part of the process is deciding what your priors are. In the formula above that deals with probabilities instead of distributions, $P(A)$ would be our "prior" probability; that is, the probability of $A$ occurring prior to any experimental observation.

You mention wanting to inference the effect of one variable on two outcome variables. Do you have reason to believe that the two outcome variables are independent of one another?
Answer: One of the advantages of Baysian analysis is the flexibility to tailor your analysis to knowledge you have apart from your observed data as opposed to Frequentist analysis, where many researchers live with standard answers.
Not having any more knowledge about your specific field and data sources we have to rely on standard answers. A good standard answer would be: Start with a simple linear regression model (or two models in this case) and see how that fits or if you find a way to improve the fit. Maybe quadratic terms or logarithmic terms, ... Maybe your theory gives you a good way to connect the two independent variables and you can come up with a model that comprises this connection.


Como manter o layout em modo de espera?

Question: Bom tenho dois layout teste_1 e layout teste_2 , quando viro aparelho para retrato aparece teste_1 e quando viro aparelho para paisagem aparece teste_2, tem como deixar esse layout em modo de espera (inativo) para que seu dados não seja reiniciado quando mudar a orientação.
Answer: Você pode adicionar o atributo android:configChanges="orientation" dentro da tag <activity>  no seu AndroidManifest.xml.
Essa configuração faz com que, ao invés de recriar a Activity ao mudar de orientação/configuração, sua Activity irá ter um callback (onConfigurationChanged, caso precise) no lugar.
Comment: não deu certo cai no mesmo problema da pergunta anterior, você poderia dar um exemplo.
Answer: Tente isso no AndroidManifest.xml , dentro da <activity:
android:configChanges="orientation|navigation|keyboard"

Como manter o layout em modo de espera?


Replace everything before domain javascript (Google Apps Script) regex

Question: Assuming a single subdomain, how do I replace everything in the URL before the domain and any trailing slashes?
Example strings:

https://www.google.com/

http://net.tutsplus.com/about
The result I want (from my example strings) is:

google.com

tutsplus.com/about
Currently, the regex I'm using is:

^https?:\/\/'
Which results in:

www.google.com/

net.tutsplus.com/about
This replaces everything up to the slashes in the URL, but I want to replace everything up to the first .
My current code in Apps Script is:
var body = DocumentApp.getActiveDocument().getBody();
body.replaceText('^https?:\/\/', '');

Given that I'm using Google Apps Script, it could be an issue with how replaceText() works. Thanks in advance for the help.
Comment: Try `^https?:\/\/.*?\.`to match everything up to and including the first `.`.
Comment: I would be surprised if there is not a JavaScript library for doing this.  Have you looked into this?
Comment: @sideroxylon That results in: `ww.google.com/`
Comment: @CBroe that's not at all constructive, and there's no reason for the hostility. I didn't include an exhaustive list of what I've tried for fear of cluttering the question. I've tried `^https?:\/\/.*\.$` and a number of variations.
Comment: @TimBiegeleisen A plain regex should be able to get me there. I don't want to import a library into Google Apps Script, partly because it's clunky and partly because it *shouldn't* be necessary.
Comment: You'll want to match one or more characters that aren't a dot, and then a dot. Show us something that can be considered a halfway serious attempt at doing that, then we can see where it goes from there.
Comment: @CBroe the match in a normal regex would be `^https?:\/\/\w+\.` to do the first bit. You're incorrect about ruling out the implementation of `replaceText()`. As per the Apps Script docs: "A subset of the JavaScript regular expression features are not fully supported, such as capture groups and mode modifiers." Using `\w+` will literally match the character `w` and be greedy, resulting in `google.com/` and `http://net.tutsplus.com/about`.
Comment: _"Using `\w+` will literally match the character `w`"_ – that makes no sense whatsoever. If that was really the syntax to match a specific character, then I don't see how you could match `h`, `t`, `p` and `s` upfront without using said syntax there as well.
Comment: @CBroe `\w+` should match "any alphanumeric character including the underscore" as per the MDN docs. Using `^https?:\/\/\w+` returns `.google.com/` and `http://net.tutsplus.com/about` using the code I've provided verbatim. You seem to have a high opinion of your own ability to solve this, but it were really that easy for you, you could've already done so, answered my question, and provided an explanation.
Comment: Can we assume that `body` contains both of your example strings at the same time, and you are running this `replaceText` only once? Then I'd be very interested to hear what the result is when you switch the order of those two example strings around in your body text, so that `http://net.tutsplus.com/about` comes first, and `https://www.google.com/` at a later position.
Comment: Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/102652/discussion-between-swankylegg-and-cbroe).
Answer: It looks like Google Doc's regex implementation is weak. It doesn't support capturing group, so you will run into problems with the following:

http://hoffmaninstitute.co.uk
http://google.com
http://docs.aws.amazon.com/

Assume that the text is always http://+one_sub_domain+domain+tld, you can use:
  var body = DocumentApp.getActiveDocument().getBody();
  body.replaceText('^https?://[0-9A-Za-z_]+\.', '');

Comment: thanks. This doesn't actually work for  the links in the example strings. Thanks for trying, though.
Answer: From Apps Script's .replaceText() docs:

Replaces all occurrences of a given text pattern with a given
  replacement string, using regular expressions.  A subset of the
  JavaScript regular expression features are not fully supported, such
  as capture groups and mode modifiers.

It will only accept strings as arguments. Implementing my own regex search and replace is unnecessarily complex because it necessitates converting each object type to be the appropriate Apps Script object before actually issuing a replacement.
I failed to note that subdomains should only be replaced if they're www due to some unforeseen link string formats that require a subdomain to be readable. For reference, here's a more thorough set of link formats:
https://www.google.com/
https://www.google.com
https://google.com/
https://google.com
http://www.google.com/
http://www.google.com
http://google.com
https://product.google.com/about/
https://product.google.com/about
https://product.google.com/
https://product.google.com
http://product.google.com/about/
http://product.google.com/about
http://product.google.com/
http://product.google.com

While the following is inefficient and verbose, it works:
function replaceLongUrls(element) {
    element = element || DocumentApp.getActiveDocument().getBody();

    element.replaceText('^https?:\/\/', '');
    element.replaceText('^www.', '');
    element.replaceText('/$', '');
};

Sources:

Apps Script Documentation

Google Apps Script Regex exec() returning null

replaceText() RegEx "not followed by"


Range search using solr_query on cql shell

Question: We are trying to pull data from cassandra-solr node with date ranges using solr query.
e.g.

SELECT *  FROM keyspace_name.table_name where solr_query='{"q":"<col1>:*",,"sort":"col1 asc"}';
            I could find syntax for apache solr documentation to perform range search & examples too.

In case of Datastax Solr DSE search syntax for multiple conditions/date filters is not available
https://docs.datastax.com/en/datastax_enterprise/4.6/datastax_enterprise/srch/srchCql.html
I couldn't find any documentation for Datastax Solr for following things specifically.

1)"fq" with multiple parameters   2) Syntax for date ranges.
    3)Date    formats & syntax to specify them using solr_query

Please let me know if i can refer to any source other than Datastax Solr documentation for syntax & examples for firing query with parameters
To find data between two dates.

Equivalent cql query:

select * from tableName where startdate >= '1987-19-09 11:14:00+0000' AND  startdate < '2003-10-06 08:42:00+0000' limit 500 allow filtering;

Please help.
 Thanks
Saurabh
Answer: In terms of fq with multiple parameters, the official documentation mentions:
"fq": <filter query expression(s) (string or array of strings)>,
As of DSE 4.6.10, you can do something like:
"fq":["a:foo", "b:bar", "c:baz"]
If you're working with dates, I recommend taking a look at https://cwiki.apache.org/confluence/display/solr/Working+with+Dates. (Remember that with the DSE Search JSON query format, you may have to do some JSON escaping on top of your Solr special character escaping to get the results you want.)
Answer: In Solr, you need to escape special characters with \. Special characters include +, -, &&, ||, !, (, ), ", ~, *, ?, and :.
So your cql query should be:
select * from tableName where solr_query ='{"q":"* : *","fq":{startdate:[1987-19-09T11\\:14\\:00\\+0000 TO 2003-10-06T08\\:42\\:00\\+0000], "sort":"startdate desc"} limit 500;


Is it an issue if the white and black keys are weighted differently?

Question: I am looking to upgrade my keyboard from one with unweighted (springy) keys to one with weighted keys that feels more like a real piano.
I'd like to buy the new CASIO PX-S1100. When I look at a lot of reviews of the keyboard there seems to be some contention
over the black keys being lighter than the white keys. When I played the PX-S1000 in the store I could not tell the difference between the white and black keys but
that is probably because I am a beginner and the difference may become more noticible as I get more experienced.
I am looking for some clarity on piano key weighting.
What causes the "weight" of a key in a real piano?
Is there a weight range that piano manufacturers aim to be in with their key weighting?
I understand that keys get lighter as you move to the higher keys. Why is that? Is that consistent across every piano?
Wouldn't every piano manufacturer aim to have the black keys lighter because the length of the key is shorter so it is a shorter lever that might "feel" heavier?
If I spend a lot of time practicing with the PX-S1100, when I play on a real piano will having equally weighted keys feel weird?

EDIT - Here are some of the reviews from different perspectives that I have watched regarding the PX-S1100:
Merriam Music mentions the weighting issue at 10 minutes & talks about it for the rest of the video.

Pretty much the entire ThePianoforever video is about the key weighting issue.

PianoManChuck didn't consider key weights important enough to mention is his video at all.
Comment: I'd have thought that due to the length difference (therefore leverage), the black keys *would* be weighted differently. However, only sufficient to give them the same *feel* as the white keys - otherwise there's no point.
Comment: The issue isn't bias; the issue is validity. Lots of people post reviews, but lots of people also don't know what they're talking about. I strongly suggest either posting a link or, if the reviews genuinely aren't relevant to your question, just remove the reference.
Comment: It seems like the core question is whether the reviews you read are meaningful. Link(s) to where you're getting your information would be helpful.
Comment: Thanks for the suggestion @Aaron but I believe that these reviews are biased sources and I do not think the quality my question will be improved by posting them. If I get any answers I'd rather have them stand on their own merit in isolation rather than as a response to a monetized video.
Comment: Fair enough, I'll edit some of them into my question.
Comment: That's a nice add. Can you also include the timing of where the black/white key weights are discussed? That will save people the time of watching the entire videos.
Comment: Regarding the question of what causes the "weight", you may find [What makes an acoustic piano touch 'heavy'?](https://music.stackexchange.com/q/64065/70803) helpful.
Comment: The other four questions you include all deserve their own posts. A brief search suggests they've not already been asked, but I believe they would be well received. (Also, we try to limit posts to one question per. Although these are all related, they deserve standalone answers rather than being lumped together.)
Answer: TL;DR: It's a matter of personal taste.

Preface: I've played scores of different pianos and keyboards, and it has never once dawned on me that the black and white keys were weighted differently.
A semi-scientific test: I measured the key weights of C4 (middle C), C#4, C6 (two octaves above middle C), and C#6 on two different (acoustic) pianos. I used stacks of nickels1 placed at the leading edge of each key and looked to see how many were required to depress and hold the key past its escapement.
Results (in # of nickels):

Piano
C4
C#4
C6
C#6

Baldwin 6'3" grand
19
18
18
16

Yamaha 45" vertical
20
21
16
18

Observations: The black keys are lighter than the white on the Baldwin, but heavier on the Yamaha. However, I've played both pianos for 10 years or more, am perfectly happy with both, and can play the same music comfortably on both.2
Conclusion: Key weight, and differences in key weight between white and black keys, vary from instrument to instrument (and technician to technician, presumably), and are a matter of personal preference or indifference.

Notes:
1 According to the U.S. Mint Coin Specifications, a nickel weighs 5 grams.
2 Although I've never noticed the key weights to matter, the action itself does. Pieces with rapidly repeated notes are much easier to play on the grand.

See also:
I stumbled onto the following discussion, in which someone performed a similar test.
Comment: @Tom Well, some have more, but they're really expensive.
Comment: I'm sure there is more than four keys on a piano ;).
Comment: @Tom Some have only [five keys](https://www.youtube.com/watch?v=T4iR_RABe4M).
Comment: @andrewleach You need a keyboard for every budget!
Comment: Apparently, even if they require a lot of nickels, they are fairly appreciated!
Comment: Where exactly did you place the nickels? Pulling them towards the edge will give them more leverage, effectively requiring less mass. I suspects the Yamaha is actually _equally_ weighted on black and white keys.
Comment: I placed the nickels as the forward edge of each key. So you're correct to point out that the white keys would have more leverage than the black. But the salient detail is not the actual weight of the key, but the perceived weight of making a sound; thus, the black keys being "heavier".
Answer: When I play music at the time with the black keys, I move my hand deeper on the keyboard, playing both black keys and the white keys at about the same length. I feel both black keys and white keys are heavier then white keys when I play them outside, near the edge of the keyboard. It makes sense.
When I teach my children piano, they complain that the black keys are more difficult to play. I told them to move their hand deeper on the keyboard, so that they need not to spread out their fingers more to strike the black keys, and spread their finger less to play the white keys. They did say that the keys become heavier when they play with their hand deeper on the keyboard. Later on, they accept the fact, and get use to it.
Comment: I have exactly the opposite experience to this. My keyboard's black keys are significantly heavier the deeper I play.

PS by deeper I guess you mean further away from the player
Answer: On a typical piano or organ keyboard, the fulcrum of each key lever is located significantly behind the back of the keyboard.  As a consequence, while pushing the back of the exposed part of each key will require more force than pushing the front, the difference won't be particularly huge.
The complaint in the PianoForever video is primarily that a keyboard which is designed to be as compact as possible won't be able to place the fulcrum as far back as it would be on a more conventional keyboard.  This greatly increases the amount by which the amount of force needed to actuate each key will vary with distance from the fulcrum.
If a compact keyboard were deisgned to use equal springs on the white and black keys, it would feel uneven because the back keys aren't as long as the white keys.  Casio designed their mechanism to try to compensate for this, but the amount of compensation required would vary depending upon whether someone plays nearer the tips of the keys or nearer the backs.  It sounds as though Casio may have over-compensated for key lengths, but what ratio of white key and back key strength would be optimal would be different for different performers.


specific submit handler to user form

Question: I have a submit handler to the user form.
It redirect the user to a specific page.
use Symfony\Component\HttpFoundation\RedirectResponse as RedirectResponse;

/**
 * @file
 * Primary module hooks for EHESS Propositions 3 module.
 *
 * @DCG
 * This file is no longer required in Drupal 8.
 * @see https://www.drupal.org/node/2217931
 */
function ehess_propositions_3_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'user_form') {
    foreach (array_keys($form['actions']) as $action) {
      if (isset($form['actions'][$action]['#type']) && $form['actions'][$action]['#type'] === 'submit') {
        $form['actions'][$action]['#submit'][] = '_ehess_propositions_3_user_profile_submit';
      }
    }
  }
}

function _ehess_propositions_3_user_profile_submit($form, &$form_state) {
  $response = new RedirectResponse('/admin/content/reglement/');
  $response->send();
}

But the problem is that the submit handler is fired both for user update and user deletion. That block the redirection to "confirm deletion" page of user.
Is there a specific action to updating user form only and not user delete?
Comment: You are attaching the submit handler to all submit buttons, including the cancel button. BTW don't send the response, add it to $form_state. See https://drupal.stackexchange.com/questions/5861/how-to-redirect-to-a-page-after-submitting-a-form
Comment: But nothing in the example distinguish beetwen update user and delete user. That was my question to which action attached the handler? Is there a "submit-update" action like ?
Answer: $action can be submit and delete. You don't need the foreach loop if you don't want to attach the submit handler to all submit buttons.
Add the handler to the save button directly:
function ehess_propositions_3_form_alter(&$form, &$form_state, $form_id) {
  if ($form_id == 'user_form') {
    $form['actions']['submit']['#submit'][] = '_ehess_propositions_3_user_profile_submit';
  }
}

For the correct way to redirect in a form see https://drupal.stackexchange.com/a/191594/47547
Comment: thanks really. It's obvious now!

specific submit handler to user form


Is there any time limit for players to apply for review?

Question: In Final Match of Women's Cricket World cup, Indian player got dismissed for LBW.
Then she applied for review, but umpire disapproved.
From commentator box: Umpire didn't approve because she didn't apply for review within 15 seconds.
Is there any time limit for players to apply for review?
Answer: As a matter of fact there is time limit to apply for review. Standard One-day international match playing conditions APPENDIX 7: Decision Review System (DRS) - Playing Conditions states

3 PLAYER REVIEW
3.2 The manner of requesting the Player Review
b) The total time elapsed between the ball becoming dead and the review request being made should be no more than 15 seconds. Only exception permitted is when an Umpire Review of a fair catch or bump ball. The on-field umpire should provide the player with a prompt after 10 seconds if the request has not been made at that time and the player should request the review immediately thereafter. If the umpires believe that a request has not been made within the 15 second time frame, they will decline to review the decision.

Umpire may have felt the decision was not made within 15 seconds and declined the review.


Residue Theorem

Question: I am really struggling with this integral that needs to be evaluated. I am really struggling with factoring the denominator in order to find the roots or poles. This is the main thing I am struggling with, nothing conceptually really. I have posted both the question and a screenshot of all of my work. If anybody could show me how to factor the denominator to get the poles/show me where I am going wrong in my work, I would greatly appreciate it. Thank you.
Comment: it seems you have removed the integral from the question in your last edit, but it is required to understand the question itself (even if it has already been answered, other people would find this question interesting for similar problems, so it is better to keep the question readable).
Answer: As you eventually want to factorise the denominator, then it's not a good idea to expand out the square at the start.
For $z=e^{i\theta}$
$$2-\sin\theta=\frac{4i-z+z^{-1}}{2i}=-\frac{z^2-4iz-1}{2iz}.$$
We can factorize the numerator by completing the square
$$z^2-4iz-1=(z-2i)^2+3=(z-2i-\sqrt3 i)(z-2i+\sqrt3 i)$$
so that
$$2-\sin\theta=-\frac{(z-2i-\sqrt3 i)(z-2i+\sqrt3 i)}{2iz}.$$
Now put this into your integral.
Comment: The poles will be where the denominator vanishes. Because of the squaring these poles will be double poles: twice the fun!

Residue Theorem


Desabilitar um FUNCTION em mobile

Question: Olá pessoal queria que essa minha função scroll fosse desabilitada para mobiles, ou seja só estará ativa se a largura da tela for 1200px pra cima.
$(window).scroll(function () {

// Movimento primeiro box vermelho
if ($(this).scrollTop() >= $("#box_img1").offset().top - $("#marcador_menu").offset().top) {
   $("#box_img1").stop().animate({"margin-right":"70px", "opacity":"1"},600);
   $("#box_text1").stop().animate({"margin-left":"100px", "opacity":"1"},600);
} else {
}
if ($(this).scrollTop() >= $("#box_img2").offset().top - $("#marcador_menu").offset().top) {
   $("#box_img2").stop().animate({"margin-left":"100px", "opacity":"1"},600);
   $("#box_text2").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
} else {
}
if ($(this).scrollTop() >= $("#box_img3").offset().top - $("#marcador_menu").offset().top) {
   $("#box_img3").stop().animate({"margin-right":"50px", "opacity":"1"},600);
   $("#box_text3").stop().animate({"margin-left":"100px", "opacity":"1"},600);
} else {
}
if ($(this).scrollTop() >= $("#box_img4").offset().top - $("#marcador_menu").offset().top) {
   $("#box_img4").stop().animate({"margin-left":"100px", "opacity":"1"},600);
   $("#box_text4").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
} else {
}
if ($(this).scrollTop() >= $("#box_img5").offset().top - $("#marcador_menu").offset().top) {
   $("#box_img5").stop().animate({"margin-right":"50px", "opacity":"1"},600);
   $("#box_text5").stop().animate({"margin-left":"100px", "opacity":"1"},600);
} else {
}
});

Comment: Acho que da pra usar `window.outerWidth`
Answer: SOLUÇÃO
$(window).scroll(function(){
    if ($(window).width() >= 1200){
        // do something here
    }
});

Answer: Insira seu código dentro do seguinte if:
if (window.innerWidth >= 1200) {
    //Seu code aqui =]
}

A mesma funcionalidade acima, porém dessa vez em Jquery juntamente ao seu código:
if ($(window).width() >= 1200) {
 $(window).scroll(function () {

  // Movimento primeiro box vermelho
  if ($(this).scrollTop() >= $("#box_img1").offset().top - $("#marcador_menu").offset().top) {
     $("#box_img1").stop().animate({"margin-right":"70px", "opacity":"1"},600);
     $("#box_text1").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else}
  if ($(this).scrollTop() >= $("#box_img2").offset().top - $("#marcador_menu").offset().top) {
     $("#box_img2").stop().animate({"margin-left":"100px", "opacity":"1"},600);
     $("#box_text2").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
  } else {}
  if ($(this).scrollTop() >= $("#box_img3").offset().top - $("#marcador_menu").offset().top) {
     $("#box_img3").stop().animate({"margin-right":"50px", "opacity":"1"},600);
     $("#box_text3").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else {}
  if ($(this).scrollTop() >= $("#box_img4").offset().top - $("#marcador_menu").offset().top) {
     $("#box_img4").stop().animate({"margin-left":"100px", "opacity":"1"},600);
     $("#box_text4").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
  } else {}
  if ($(this).scrollTop() >= $("#box_img5").offset().top - $("#marcador_menu").offset().top) {
     $("#box_img5").stop().animate({"margin-right":"50px", "opacity":"1"},600);
   $("#box_text5").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else {}
 });
}

Como informação adicional um outro if interessante para mobile é comparando se existe touch no dispositivo:
if(('ontouchstart' in window)){
   //O code aqui sera executado apenas em dispositivos que possuem Touch Screen
}

Answer: Você poderia fazer da seguinte forma:
$(window).resize(function() {

  if ($(window).width() > 1200) {
    $(window).scroll(myFunction);
  } else {
    $(window).unbind('scroll');
  }

});

var myFunction = function() {

  // Movimento primeiro box vermelho
  if ($(this).scrollTop() >= $("#box_img1").offset().top - $("#marcador_menu").offset().top) {
    $("#box_img1").stop().animate({"margin-right":"70px", "opacity":"1"},600);
    $("#box_text1").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else {
  }
  if ($(this).scrollTop() >= $("#box_img2").offset().top - $("#marcador_menu").offset().top) {
    $("#box_img2").stop().animate({"margin-left":"100px", "opacity":"1"},600);
    $("#box_text2").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
  } else {
  }
  if ($(this).scrollTop() >= $("#box_img3").offset().top - $("#marcador_menu").offset().top) {
    $("#box_img3").stop().animate({"margin-right":"50px", "opacity":"1"},600);
    $("#box_text3").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else {
  }
  if ($(this).scrollTop() >= $("#box_img4").offset().top - $("#marcador_menu").offset().top) {
    $("#box_img4").stop().animate({"margin-left":"100px", "opacity":"1"},600);
    $("#box_text4").stop().animate({"margin-right":"-50px", "opacity":"1"},600);
  } else {
  }
  if ($(this).scrollTop() >= $("#box_img5").offset().top - $("#marcador_menu").offset().top) {
    $("#box_img5").stop().animate({"margin-right":"50px", "opacity":"1"},600);
    $("#box_text5").stop().animate({"margin-left":"100px", "opacity":"1"},600);
  } else {
  }

});

Desabilitar um FUNCTION em mobile


Test for Python module dependencies being installed

Question: How could one test whether a set of modules is installed, given the names of the modules. E.g.
modules = set(["sys", "os", "jinja"])

for module in modules:
  # if test(module exists):
  #   do something

While it's possible to write out the tests as:
try:
  import sys
except ImportError:
  print "No sys!"

This is a bit cumbersome for what I'm doing. Is there a dynamic way to do this?
I've tried eval("import %s"%module) but that complained of a compile error.
I'm grateful for your thoughts and suggestions. Thank you.
Answer: You can use the __import__() function like this::
for module in modules:
    try:
        __import__(module)
    except ImportError:
        do_something()

You can also use imp.find_module to determine whether a module can be found without importing it::
import imp
for module in modules:
    try:
        imp.find_module(module)
    except ImportError:
        do_something()

Oh, and the reason you can't use eval() is because import is a statement, not an expression.  You can use exec if you really want to, though I wouldn't recommend it::
for module in modules:
    try:
        exec 'import ' + module
    except ImportError:
        do_something()

Answer: What's wrong with this?
modules = set(["sys", "os", "jinja"])
for m in modules:
    try:
        __import__(m)
    except ImportError:
        # do something

The above uses the __import__ function. Also, it is strictly testing whether it exists or not - its not actually saving the import anywhere, but you could easily modify it to do that.
Answer: It's worth understanding the tradeoffs between the find_module approach and the __import__ approach:

__import__('foo') will trigger any side effects of loading the module, regardless of whether you save a reference to the imported module. That might be OK, or not.
find_module('foo') won't trigger any side effects, so it should always be safe.
neither approach works well if the module is found but has other problems at load time (eg. SyntaxError, NameError, ...)
Depending on how you handle failures, simply wrapping __import__('foo') in a try/except will give you misleading results if foo is found but foo imports bar which is not found.
find_module() can't handle dotted names, so if you need to check whether 'foo.bar.bat.baz' can be imported, you have to use multiple calls to find_module and load_module as per the docs.
Answer: modules = set(["sys", "os", "jinja"])
try:
  for module in modules:
    __import__(module)
  doSomething()
except ImportError, e:
    print e


Is this snippet modification correct? (php snippet in a drupal commerce rule)

Question: I´ve read about the possibility to use this snippet to hide a specific checkout pane (in Drupalcommerce), according to the cart contents.
Could I use this somewhat modified to hide a pane when the user completing the order is registered?
<?php
function hook_commerce_checkout_pane_info_alter(&$checkout_pane) {
    foreach ($checkout_pane as $pane_name => & $pane_data) {
        if ($pane_name == 'customer_profile_shipping') {
            $order = commerce_cart_order_load($user->uid);
            if (!empty($order->data)) {
                    unset($checkout_pane['customer_profile_shipping']);
}}}}
?>

I understand that I should change this if (!empty($order->data)) { maybe to something like this? if ($user->uid > 0) {
Is that correct? I want tu hide the pofile2 pane:
So it would be like this:
<?php
function hook_commerce_checkout_pane_info_alter(&$checkout_pane) {
    foreach ($checkout_pane as $pane_name => & $pane_data) {
        if ($pane_name == 'customer_profile_shipping') {
            $order = commerce_cart_order_load($user->uid);
            if ($user->uid > 0) {
                    unset($checkout_pane['commerce_p2cp_profile']);
}}}}
?>

I´m not sure how to add that into a rule set...
Answer: You can add your PHP code in action of some rules. For this you will have to enable core PHP filter module.
Once you enable it, you go to ADD ACTION in rules and you will find a option "Execute custom PHP code". If the variable you are suppose to use is available there for using with the PHP code then it will work without any problem..
As per your requirement, if you do this by simple small module then also it will work fine as you just have to put this hook in your .module file, declare the module in .info file & enable it....I am suggesting using module as I see you are already checking your condition in code...
Comment: Yes, I´ll do that instead. Because the variable isn´t available in my rule... It will be much more easier and faster to try the module way. THANKS!!


Spammer relaying via Postfix mail server

Question: I have a mail server (cm.snowbarre.co.za) on Ubuntu 8.04 LTS which forwards all SMTP traffic to an anti-spam server cacti.snowbarre.co.za. Many times I see the headers on the anti-spam server to contain from addresses not  hosted on the mail server and I have checked and confirmed that my server is not an open relay server. How can a spammer be using my server to relay spam traffic? How can I stop this?
Open relay test:
paddington@paddington-MS-7387:~$ telnet cm 25
Trying 196.201.x.x...
Connected to cm.
Escape character is '^]'.
220 cm.snowbarre.co.za ESMTP Postfix (Ubuntu)
mail from:firstname.lastname@example.com
250 2.1.0 Ok
rcpt:firstname.lastname@example.com
221 2.7.0 Error: I can break rules, too. Goodbye.
Connection closed by foreign host.
paddington@paddington-MS-7387:~$

A typical headers is:
Received: from cm.snowbarre.co.za (cm.snowbarre.co.za[196.201.x.x])
     by cacti.snowbarre.co.za (Postfix) with ESMTPS id 00B601881AD;
     Mon, 27 Aug 2012 14:03:29 +0200 (SAST)
Received: from cm.snowbarre.co.za (localhost [127.0.0.1])
    by cm.snowbarre.co.za (Postfix) with ESMTP id 81627367E007;
    Mon, 27 Aug 2012 14:02:50 +0200 (SAST)
Received: from User (ml82.128.x.x.multilinksg.com [82.128.x.x])
    by cm.snowbarre.co.za (Postfix) with ESMTP;
    Mon, 27 Aug 2012 14:02:49 +0200 (SAST)
Reply-To: <email@example.com>
From: "Ms Nkeuri Aguiyi"<firstname.lastname@example.com>
Subject: Your Unpaid Fund.
Date: Mon, 27 Aug 2012 05:03:22 -0700
MIME-Version: 1.0
Content-Type: text/html;  charset="Windows-1251"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2600.0000
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000
X-Antivirus: avast! (VPS 120821-0, 08/21/2012), Outbound message
X-Antivirus-Status: Clean
Message-Id: <email@example.com>
To: undisclosed-recipients:;*

Comment: drcelus, most mailservers will accept the short form of the command, so `mail:` and `rcpt:`will give the same result as `mail to:`and `rcpt to:`.
Comment: Your telnet conversation is not correct. Try using `rcpt to:` instead of `rcpt` and see what happens.
Comment: What I'm missing here is info about what the mailservers do. Are they only for incoming or for outgoing, or for both with limits on who can send where? What do the logs say for this message, in both servers?
Comment: the CM server is for both incoming and out going mail (SMTP and POP), it forwards all SMTP traffic destined for a domain not residing on itself to our anti spam server (called Cacti) which relays it out. My issue is on my anti spam server I am seeing emails from CM that show headers of domains not hosted on CM. This is a big is worrying since this means someone is relaying spam via my CM server and it is serving domains it does not host. The CM server only servers SMTP for domains resident on it.
Answer: Just because there's a line in the headers which looks as if an SMTP request from ml82.128.x.x.multilinksg.com is being relayed - that's not necessarily the truth.
If you've got an open web proxy or someone has ssh access to your network then they could easily be relaying these emails from elsewhere - the only difference is that the emails are not entering your network via SMTP.
(this may not be the case here - but checking it should be at the top of your list).


Random Forest Bias in Permutation Importance.

Question: I just read on several blogs something at the form: Variable Importance using permutation will lead to a bias if the variables exhibit correlation. It is for instance stated by https://blog.methodsconsultants.com/posts/be-aware-of-bias-in-rf-variable-importance-metrics/ that

"The mean decrease in impurity and permutation importance computed from random forest models spread importance across collinear variables. For example, if you duplicate a feature and re-evaluate importance, the duplicated feature pulls down the importance of the original, so they are close to equal in importance."

In the article by Strobl et. al (2008): https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-9-307 it is argued that correlated variables will show too high variable importance, where it is stated

"We know that the original permutation importance overestimates the importance of correlated predictor variables."

Furthermore, it is described in https://scikit-learn.org/stable/modules/permutation_importance.html that

"When two features are correlated and one of the features is permuted, the model will still have access to the feature through its correlated feature. This will result in a lower importance value for both features, where they might actually be important."

The three quotes seem rather contradicting. The second quote states that correlated variable will show too high variable importance, where the third states that the variable importance will be too low. The quote agrees with this.
Do anyone know what is true? Is the variable importance overestimated or underestimated when variables are correlated?
Answer: The blog quote is identical to what was stated in the scikit learn and based on my personal experiments that are true however I also found the second quote confusing.
let me share my experiments to make that point clear.
t-test score is a distance measure feature ranking approach which is calculated for 186 features for a binary classification problem in the following figure.
the higher the value of t-score the better the feature is

looking into it we can obviously see that the best features are in the range of 45 and it neighboring while the less informative features are in the range of 90 to 100.
now let's look into the correlation between the features in the following figure.

and also the following figure is showing the classwise distribution of this binary class.

looking into the correlation figure, it is obvious that features in the range of 90 to 100 have the minimum correlation while other ranges of features that were highly informative are highly correlated.
now lest having a look into the random forest feature importance calculated based on permutation importance using scikit learn in the following figure.

now all the feature which were informative are actually downgraded due to correlation among them and the feature which were not informative but were uncorrelated are identified as more important features.
therefore we can conclude that using random forest feature selection approach for datasets with highly correlated features are not a suitable choice.


jQuery .click() not working with setInterval

Question: Hey, I have this piece of jQuery/Javascript:
$(document).ready(function() {
                var points = 0;
                var iterations = 10;
                var winWidth = $(window).width();
                var winHeight = $(window).height();

                setInterval(function() {
                    if (iterations > 0) {
                        var rndX = Math.ceil(Math.random() * winWidth);
                        var rndY = Math.ceil(Math.random() * winHeight);
                        $('div.item').remove();
                        $('body').append('

');
                        iterations--;
                    } else {
                        location.href = "http://www.google.com/";
                        $('*').remove();
                    }
                }, 1750);

                $('div.item').click(function() {
                    $(this).remove();
                    points = points + 1;
                    $('#points').val(points);
                    return false;
                });

            });

But for some reason, the $('div.item').click(function() doesn't fire up :(
Any ideas?
Answer: Instead of using "click", use "delegate":
$('body').delegate('div.item', 'click', function() {
  $(this).remove();
  points = points + 1;
  $('#points').val(points);
  return false;
});

When your interval handler code removes all the "div.item" elements from the document, that will also remove the handlers you established. By using "delegate" instead, you put just one handler on the  element, and it takes advantage of event bubbling to handle all the clicks. Those that come from elements that match the "div.item" selector will be handled with your code, just as if the handler had been directly bound to the elements.
Because the "delegate" mechanism applies the selector at the time the events actually happen, it doesn't matter whether the target element existed since the page was first received or if the element was just dynamically added (as is the case in your code).
Comment: Awesome, ty - will "tick" when SO lets me.
Answer: your divs dont exist when you try and bind your click function to the elements...
You need to bind them ahead of time (dynamically).
see .live() and .delegate()
Answer: I would suggest using JQuery's .live method for similar reasons as Pointy.
Live will bind to elements as they are created.
$('div.item').live('click', function() {
             $(this).remove();
             points = points + 1;
             $('#points').val(points);
             return false;
          });

Comment: The ".live()" function will work, but (in my opinion) "delegate()" is a better API.  When you use "live()", jQuery still has to build up the element list, even though it won't really do anything with it.  Also, with "live()" you have no choice of where the real handler is installed (always the `` element), while "delegate()" lets you put the real handler on any container element.


javax.el.PropertyNotFoundException: Property 'toArray' not found on type org.hibernate.collection.internal.PersistentSet

Question: I am using hashset.toArray()[x] to display a element in the jsf view, and this is working fine in my machine.
But when I move this on to test server, the above error is thrown.
Both machines are running tomcat 7.
what is causing this error?
Comment: version of Hibernate on local and on Test server? Also JSF version on local and on test?
Comment: It's weird, because the message is correct. A `PersistentSet` has a *method* `toArray`, not a *property*. The EL is actually trying to access the property `getToArray()` and the result is an exception. Please show us the code that's working on your machine (jsf + bean). Using it in a display tag shouldn't actually work AFAIK.
Answer: The ability to invoke arbitrary non-property-related methods in EL is introduced in EL version 2.2 which goes hand in hand with Servlet 3.0. This feature didn't exist in older versions like Servlet 2.5 / EL 2.1.
So, if you deploy your webapp to a Servlet 3.0 compatible container with a Servlet 3.0 compatible web.xml root declaration, then it will work fine. However, if you deploy your webapp to a container of an older version, or with a web.xml which dictates an older version, or have dropped arbitrary container-specific JAR files of an older version inside webapp's /WEB-INF/lib or even server's own /lib, then this feature won't work.
Provided that you're absolutely positive that the test server is running Tomcat 7 and thus not Tomcat 6 or so, then that can only mean that the web.xml was been changed to dictate an older version, or that your webapp or server's /lib is littered with arbitrary container-specific JAR files such as jsp-api.jar, el-api.jar, etc which would only conflict with container's own libraries (this is often done by ignorant starters in order to workaround compilation errors they face in their IDE; which should have been solved differently).
Comment: Thanks Balusc for detailed answer


Poisson process elapsed time distribution

Question: We know in a Poisson process with parameter $\lambda$, the time between two consecutive occurrence is exponentially distributed with parameter $\lambda$, and as a result it is memoryless.
We are interested in finding the density function for the elapsed time. Indeed we are looking at the process at a time, what is the density of the time difference of the previous occurrence and the time that we are looking at the process?
I have the following explanation, but I cannot figure out, where am I wrong.
Let $T_e$ be the elapsed time, $T_w$, the time we should wait for the next occurrence, and $T$ be the time between two occurrence. Then obviously we have
$$T_e=T-T_w\Rightarrow \mathbb{E}[T_e]=\mathbb{E}[T]-\mathbb{E}[T_w]=\frac{1}{\lambda}-\frac{1}{\lambda}=0$$
and as $T_e\geq 0$, and it has a zero mean, it means that almost everywhere its density function is zero, so
$$f_{T_e}(t)\stackrel{a.e.}{=}0$$
which doesn't make sense to be almost everywhere equal to zero. Is it wrong?
Comment: The time between the first event after $t$ and the last event before $t$ is **not** exponential with parameter $\lambda$. One can show that its expectation converges to $2\lambda$ when $t\to\infty$. This is called the inspection paradox.
Comment: I answered pretty much the same question [here](http://math.stackexchange.com/q/2135428/364279). We were to find the pdf of the time since last arrival, which is what you refer to as elapsed time. We get (almost) the same exponential distribution as for the inter-arrival times.
Comment: @Blaza Do you think that the solution (Your solution to the same question) is right? Since the expected value of that distribution is positive and as a result the expected value of the time between two ceonsecutive occurance is more than the expected value of the waiting time after looking at the process
Comment: @CLAUDE yes, as Did and Graham Kemp said in their comments/answers the time between the last occurrence and the next isn't exponentially distributed when we are looking from an explicit time point. As a side note, I think Did meant to write $2/\lambda$ instead of $2\lambda$. That also makes sense in a way that (as $t\to\infty$) we have 2 exponential variables for the elapsed and the wait time, so their sum gives the expectation $2/\lambda$.
Comment: @Blaza Yes. $ $
Answer:
Let $T_e$ be the ellapsed time, $T_w$, the time we should wait for he next occurance, and $T$ be the time between two occurance. Then obviously we have
$$T_e=T-T_w\Rightarrow \mathbb{E}[T_e]=\mathbb{E}[T]-\mathbb{E}[T_w]=\frac{1}{\lambda}-\frac{1}{\lambda}=0$$

To be clear: $T_e$ is the time since the last occurrence, $T_w$ is the time until the next occurrence, in a Poisson process, and $T$ is the time between the occurrences (thus $T=T_w+T_e$).
It is not obvious why you think the time-displacement, $T$, between the last and next occurrence should be conditionally exponentially distributed when given a fixed time-point that is between the occurrences (the time of observation).
The time between an arbitrary pair of subsequent occurrences is exponentially distributed.   However these are not arbitrary occurrences.
From the point of view of the last occurrence, $T$ is the time until the next occurrence given that it is after you arrive.  The time until you arrive is exponentially distributed, and the time after that until the next occurrence is also exponentially distributed, so their sum is Gamma distributed.
$$~~~~T\sim\Gamma(2,\lambda^{-1}) \\\therefore~~ \mathsf E(T)= 2\lambda^{-1}$$


Python check for word in list

Question: I'm writing a spell checking function and I'm using two text files: one that has misspelled text and a text file with a bunch of words from the dictionary. I have turned the text of misspelled words into a list of strings and the text file with dictionary words into a list of words. Now I need to see if the words in my misspelled list are in my list of dictionary words.
def spellCheck():
    checkFile=input('Enter file name: ')
    inFile=open(checkFile,'r')

# This separates my original text file into a list like this
# [['It','was','the','besst','of','times,'],
# ['it','was','teh','worst','of','times']]

    separate=[]
    for line in inFile:
        separate.append(line.split())

# This opens my list of words from the dictionary and
# turns it into a list of the words.

    wordFile=open('words.txt','r')
    words=wordFile.read()
    wordList=(list(words.split()))
    wordFile.close()

# I need this newList to be a list of the correctly spelled words
# in my separate[] list and if the word isn't spelled correctly
# it will go into another if statement...

    newList=[]
    for word in separate:
        if word in wordList:
            newList.append(word)
    return newList

Comment: I hope I fixed the indentation correctly - could you check? I had to juggle a few lines around...
Comment: Thank you! When I run my program I'm getting an empty list though. Do you know what I've done wrong?
Comment: Are you asking [this question again](http://stackoverflow.com/questions/17681186/python-turn-text-file-into-dictionary) but now trying to use lists instead of a `dict`?
Answer: Try this:
newList = []
for line in separate:
    for word in line:
        if word in wordList:
            newList.append(word)
return newList

The problem you had was that you were iterating over separate, which is a list of lists. There does not exist any list in your wordList, which is why that if-statement always fails. The words that you want to iterate over are in the sublists contained in separate. So, you can iterate over these words in a second for-loop. You can also use for word in itertools.chain.from_iterable(separate).
Hope this helps
Answer: First, a word about data structures. Instead of lists, you should use sets, since you (apparently) only want a copy of each word. You can create sets out of your lists:
input_words = set(word for line in separate for word in line) # since it is a list of lists
correct_words = set(word_list)

Then, it is simple as that:
new_list = input_words.intersection(correct_words)

And if you want the incorrect words, you have another one liner:
incorrect = input_words.difference(correct_words)

Note that I used names_with_underscores, instead of CamelCase, as recommended in PEP 8.
Bear in mind, however, that this is not very efficient for spell checking, since you don't examine context.

Python check for word in list


How do you fight against the entropy in the returned xml from google maps geocoding?

Question: http://maps.googleapis.com/maps/api/geocode/xml?address=55124&sensor=false
For example ^ the url above will take the supplied zip code and return approximate lat and lng coordinates and a formatted address along with "address components" containing things like city,state,country and what not.
The question is (if you play with it and enter different types of addresses) how could you write code that would discern the differences in the address components? how could i be sure that the value i'm seeking is in fact the city and not the street?
It doesn't to be reliable xml. There is entropy if i'm not mistaken. Anyone know a way to guarantee a value?
Answer: Each address component has a type like
<code><type>postal_code</type>
</code>
You can then look for the type you're interested in.
Comment: yes but for more than one address component is the type 'political' (city) from what i've observed.(county also)
Comment: If the type is political, there often is a second type field which describes it more.


Are models identified by auto.arima() parsimonious?

Question: I have been trying to learn and apply ARIMA models. I have been reading an excellent text on ARIMA by  Pankratz - Forecasting with Univariate Box - Jenkins Models: Concepts and Cases. In the text the author especially emphasizes the priciple of parsimony in choosing ARIMA models.
I started playing with auto.arima() function in R package forecast. Here is what I did, I simulated ARIMA and then applied auto.arima(). Below are 2 examples. As you can see in both example auto.arima() clearly identified a model that many would consider non-parsimonious. Especially in example 2, where auto.arima() identified ARIMA(3,0,3) when actually ARIMA(1,0,1) would be sufficient and parsimonious.
Below are my questions. I would appreciate any suggestions and recommendations.

Are there any guidance on when to use/modify the models identified using automatic algorithms such as auto.arima()?
Are there any pit falls in just using AIC (which is what I think auto.arima() uses) to identify models?
Can an automatic algorithm built that is parsimonious?

By the way I used auto.arima() just as an example. This would apply to any automatic algorithm.
Below is Example #1:
set.seed(182)
y <- arima.sim(n=500,list(ar=0.2,ma=0.6),mean = 10)

auto.arima(y)

qa <- arima(y,order=c(1,0,1))
qa

Below are the results from auto.arima(). Please note that all the coefficients are insignificant. i.e., $t$ value < 2.
ARIMA(1,0,2) with non-zero mean

Coefficients:
         ar1     ma1      ma2  intercept
      0.5395  0.2109  -0.3385    19.9850
s.e.  0.4062  0.4160   0.3049     0.0878

sigma^2 estimated as 1.076:  log likelihood=-728.14
AIC=1466.28   AICc=1466.41   BIC=1487.36

Below are the results from running regular arima() with order ARIMA(1,0,1)
Series: y
ARIMA(1,0,1) with non-zero mean

Coefficients:
         ar1     ma1  intercept
      0.2398  0.6478    20.0323
s.e.  0.0531  0.0376     0.1002

sigma^2 estimated as 1.071:  log likelihood=-727.1
AIC=1462.2   AICc=1462.28   BIC=1479.06

Example 2:
set.seed(453)
y <- arima.sim(n=500,list(ar=0.2,ma=0.6),mean = 10)

auto.arima(y)

qa <- arima(y,order=c(1,0,1))
qa

Below are the results from auto.arima():
ARIMA(3,0,3) with non-zero mean

Coefficients:
         ar1      ar2     ar3     ma1     ma2     ma3  intercept
      0.7541  -1.0606  0.2072  0.1391  0.5912  0.5491    20.0326
s.e.  0.0811   0.0666  0.0647  0.0725  0.0598  0.0636     0.0939

sigma^2 estimated as 1.027:  log likelihood=-716.84
AIC=1449.67   AICc=1449.97   BIC=1483.39

Below are the results running regular arima() with order ARIMA(1,0,1)
Series: y
ARIMA(1,0,1) with non-zero mean

Coefficients:
         ar1     ma1  intercept
      0.2398  0.6478    20.0323
s.e.  0.0531  0.0376     0.1002

sigma^2 estimated as 1.071:  log likelihood=-727.1
AIC=1462.2   AICc=1462.28   BIC=1479.06

Comment: You appear to be on to something here as the AIC procedure might be mis-identifying the simplest model. Additionally the AIC criteria premises no pulses/no level shifts/no seasonal pulses/no local time trends/constancy of parameters/constancy of error variance.
Comment: The AIC and BIC are proportional to the variance of the errors from a guessed model. This approach in my experience is just to simple due to some of the issues I have already raised and the invertibility issues raise here. There is no replacement for an intelligent identification system that constructs iterative models concluding when no uneeded parameters remain and no information/structure is evident in the residuals. In summary automatic model identification is an iterative process not a one-step process just as it is in all statistical analysis.
Comment: @Irishstat what is the sign of ma coefficient should it be interpretted as -0.1391 -0.5912 -0.5491 ?
Comment: in terms of the Pankratz Text .. yes ! . You can check the roots of the ma polynomial to see if they meet the invertibility requirements.
Comment: so the above auto.arima is not invertible i.e, -0.1391-0.5912-0.5491 is <1 therefore the model is fine.
Comment: There are multiple restrictions on the coefficients. One of them might be met BUT all have to be met. THe Box-Jenkins text and http://www.amazon.de/Time-Series-Analysis-Univariate-Multivariate/dp/0201159112 should be pursued.
Comment: auto.arima always returns an invertible model. If you think it is non-invertible, you have made an error.
Comment: @Rob I meant to say so the above auto.arima is invertible i.e, -0.1391-0.5912-0.5491 is <1 therefore the model is fine
Answer: There are a couple of issues here. Firstly, don't presume that the simulated ARIMA is truly of the order you specify; you are taking a sample from the specified model and due to randomness, the best fitting model for the particular sample drawn may not be the one from which the simulations were drawn.
I mention this because of the second and more important issue: the auto.arima() function can estimate models via a more efficient fitting algorithm, using conditional sums of squares, to avoid excessive computational time for long series or for complex seasonal models. When this estimation process is in use, auto.arima() approximates the information criteria for a model (because the log likelihood of the model has not been computed). A simple heuristic is used to determine whether the conditional sums of squares estimation is active, if the user does not indicate which approach should be used.
The behaviour is controlled via argument approximation and the simple heuristic is (length(x)>100 | frequency(x)>12), hence approximation takes a value TRUE if the length of the series is greater than $n = 100$, or there are more than 12 observations within each year. As you simulated series with $n = 500$ but did not specify a value for the approximation argument, you ran auto.arima() with approximation = TRUE. This explains the apparently  erroneous selection of a model with larger AIC, AICc, and BIC than the simpler model you fitted with arima().
For your example 1, we should have
> auto.arima(y, approximation = FALSE)
Series: y
ARIMA(0,0,1) with non-zero mean

Coefficients:
         ma1  intercept
      0.7166    19.9844
s.e.  0.0301     0.0797

sigma^2 estimated as 1.079:  log likelihood=-728.94
AIC=1463.87   AICc=1463.92   BIC=1476.52
> qa
Series: y
ARIMA(1,0,1) with non-zero mean

Coefficients:
         ar1     ma1  intercept
      0.0565  0.6890    19.9846
s.e.  0.0626  0.0456     0.0830

sigma^2 estimated as 1.078:  log likelihood=-728.53
AIC=1465.06   AICc=1465.14   BIC=1481.92

Hence auto.arima() has selected a more parsimonious model than the true model; an ARIMA(0, 0, 1) is chosen. But this is based on the information criteria and now they are in accordance; the selected model has lower AIC, AICc, and BIC, although the differences for AIC and AICc are small. At least now the selection is consistent with the norms for choosing models based on information criteria.
The reason for the MA(1) being chosen, I believe, relates to the first issue I mentioned; namely that the best fitting model to a sample drawn from a stated ARIMA(p, d, q) may not be of the same order as the true model. This is due to random sampling. Taking a longer series or a longer burn in period may help increase the chance that the true model is selected, but don't bank on it.
Regardless, the moral here is that when something looks obviously wrong, like in your question, do read the associated man page or documentation to assure yourself that you understand how the software works.
Comment: @forecaster Assume you don't know the truth, all you have is the sample to hand. In that circumstance, AIC etc are doing what you are asking them to do. As I said above, the simulated series may not be the one you asked for, or rather it may not be possible to successfully identify the true series from the small sample you took. There is nothing implied here or elsewhere in stats that you'll get the one, TRUE model. As the statistician, if you think the series is overfitted (say from prior knowledge), place a restriction on the order of the terms searched. Or use BIC as the stopping criterion.
Comment: thanks for the detailed response. I did use your approach for the second example: set.seed(453)
y <- arima.sim(n=500,list(ar=0.2,ma=0.6),mean = 10)

auto.arima(y,approximation=FALSE) and this is what i get, it is clearly overfitting the data -- ARIMA(2,0,4) with non-zero mean

Coefficients:
         ar1      ar2     ma1     ma2     ma3     ma4  intercept
      0.5369  -0.9627  0.3681  0.6799  0.7065  0.1701    20.0329
s.e.  0.0278   0.0499  0.0533  0.0630  0.0793  0.0574     0.0927

sigma^2 estimated as 1.024:  log likelihood=-716.17
AIC=1448.33   AICc=1448.63   BIC=1482.05
Comment: I have not yet read the chapter that has something called "invertiblity". Does the auto.arima in the second case violate the "invertiblity" rule in model diagnosis?, I may be off, can you please correct if I'm wrong?
Comment: I don't know sufficiently well what invertibility is to be able to answer that question. Perhaps ask this as a *new* question here?
Comment: @forecaster (Reason I say BIC, is that it places an extra penalty on complexity. In this case, the BIC of the ARIMA(1, 0, 1) has lower BIC than the model `auto.arima()` settled on, so if those were the only two candidate models, the simpler one would have been selected...)
Comment: thanks so much. This makes sense. Actually I dont mind if auto.arima returns a model of different from the simulated model. My question was around parsimonious model. Based on your explanation it is clear that BIC is the IC criteria that should be choosen to select parsimonious model.
Answer: Thanks so much@Gavin, @Irishstat and @Rob for responding to my question. It is clear that if I need a parsimonious model from an automatic algorithms like auto.arima BIC information criterion should be used as opposed to AIC especially after looking this post  and @Gavin's post above.
I also very much agree with @Irishstat that choosing a model based on IC criterion has limitations in that it does not choose a better model to fit a data with outliers and level shifts. In my opinion, outliers + level shifts + messy data = real word business data, anything else is textbook datasets. Any automatic model that doesn't consider outliers + level shifts, again in my opinion should be used with caution.
Coming to the code - auto.arima has an option to choose between the AIC or BIC. See below the code has been modified from teh above questions.
Many thanks Cross-validated community. I learn new and intresting things every day.
###############
set.seed(453)
y <- arima.sim(n=500,list(ar=0.2,ma=0.6),mean = 10)

## Adequetly describes the unknown data
fit.aic <- auto.arima(y,ic = c("aic"))
fit.aic

## Selects the model that is parsimonious
fit.bic <- auto.arima(y,ic = c("bic"))
fit.bic

BIC IC chooses an MA(2) model.
> fit.bic
Series: y
ARIMA(0,0,2) with non-zero mean

Coefficients:
         ma1     ma2  intercept
      0.9256  0.2335    20.0326
s.e.  0.0453  0.0444     0.0992

sigma^2 estimated as 1.059:  log likelihood=-724.19
AIC=1456.39   AICc=1456.47   BIC=1473.24

Answer: I took the 500 values into AUTOBOX ( a piece of commercial software that I have helped develop) and received the following advisory  based upon the Chow Test for constancy of parameters. A very basic mistake that is made in the study of a time series is to assume that the data is driven by a particular model with constant parameters.
AUTOBOX detected a break point at period 246 which might reflect a simulation that had not been "warmed up". When simulating data good practice is to delete the first "n" values and to then study the remaining ones. I took the data and segmented it into two sections ; the first 245 and the remaining 255 . Here are the two very dissimilar acf plots  .
Returning to the analysis: Here is the model that was identified for the last 246 values  and here  with the following statistics  . The Actual/Fit and Forecast is here with residual plot here  . The ACF of the residuals suggests sufficiency  . Note that the 5 pulses that were identified had a very small effect and could easily be disregarded ( in this case ! ). In summary the lesson learned here is that sometimes we have too much data and we need to consider time changing coefficients. In this case we are identifying a change in parameters which (apparently) doesn't have a major impact on the resultant model/parameters but it points out a generally needed process improvement in time series analysis. My experience with auto.arima suggests that since it explicitely doesn't treat/remedy gaussian violations it tends to over-model by leaning too much on historical values rather than extracting structure from the data. In this case since it was a tightly controlled simulation without gaussian violations it worked but I would be generally suspect of such a bandwith limited and single step approach to ARIMA model identification. Trust but Verify !
Answer: Akaike_information_criterion
says that
$\ \exp^{(AIC_{min} − AIC_i)/2}$
"can be interpreted as the relative probability that the ith model minimizes the (estimated) information loss".
IF this is so, it would help users to see these relative probabilities,
together with the AICs (?) from auto.arima( ... trace=TRUE ).
For example, the eggs data run as in
this question
gives
                                # relprob = exp( (AICmin - AIC) / 2 ) * 100
 ARIMA(0,1,0) with drift : 784.5    100
 ARIMA(0,1,1) with drift : 784.8     86
 ARIMA(1,1,0) with drift : 784.9     82
 ARIMA(0,1,0)            : 792.4      2
 ARIMA(2,1,2) with drift : Inf    0
 ARIMA(1,1,1) with drift : Inf    0

Comment: This is an answer or a question?
Comment: @Tim, I'm so tentative becuase I don't know if these relative probs are "real" or at least common.


Relation of local base with base

Question: Is it true that local base at point $x$ of a topological space is the collection of base elements which contain the point $x$ ?
Answer: Not quite, but almost. If we have a base $\mathcal{B}$ for the topological space $(X,\mathcal{T})$, then indeed, for every $x \in X$ the subset $\mathcal{B}_x = \{B \in \mathcal{B}: x \in B\}$ is a local base for the topology at $x$. The proof is immediate.
Conversely, if we have a local base $\mathcal{B}_x$ for each $x \in X$, then $\mathcal{B} = \bigcup_{x \in X} \mathcal{B}_x$ is a base for the topology. This is not hard to see, again.
But if we just have one local base at some point $x$, I don't think it needs to be of the form $\mathcal{B_x}$ for some $\mathcal{B}$ which is a base for the whole space; we can consider local bases without considering a global base at all.
Comment: About the last paragraph: the only way for this to fail is for the topology not to be $T_1$. Otherwise, we can put at every $y\neq x$ a local base consisting of open sets containing $y$ but not $x$ and apply your second paragraph. For a non-$T_1$ counterexample, consider the isolated point in Sierpiński's two-point space.
Comment: @tomasz thanks for the addition.

Relation of local base with base


Prove $\mathcal T∘\mathcal R=\mathcal T∘\mathcal S$ for any $\mathcal T$ iff $\mathcal R∘\mathcal T=\mathcal S∘\mathcal T$ for any $\mathcal T$.

Question: I am trying to understand if for two binary relations $\mathcal R$ and $\mathcal S$ the following statements are equivalent.

$\mathcal R=\mathcal S$
$\mathcal R\circ\mathcal T=\mathcal S\circ\mathcal T$ for any (possible) relation $\mathcal T$
$\mathcal R[X]=\mathcal S[X]$ for any (possible) $X$
$\mathcal R^{-1}=\mathcal S^{-1}$
$\mathcal T\circ\mathcal R=\mathcal T\circ\mathcal S$ for any (possible) relation $\mathcal T$

So if $1$ holds then trivially $2$ holds.
Now if $2$ holds then for any $X$ we can consider the identity relation $\operatorname{id}_X$ so that the equality
$$
\tag{2.1}\label{2.1}\mathcal R\circ\operatorname{id}_X=\mathcal S\circ\operatorname{id}_X
$$
holds. So by \eqref{2.1} we argue that the equality
$$
\mathcal R[X]=\mathcal R\big[\operatorname{id}_X[X]\big]=(\mathcal R\circ\operatorname{id}_X)[X]=(\mathcal S\circ\operatorname{id}_X)[X]=\mathcal S\big[\operatorname{id}_X[X]\big]=\mathcal S[X]
$$
holds.
Now if $(y,x)$ is in $\mathcal R^{-1}$ then $(x,y)$ is in $\mathcal R$ and so $y$ is in $\mathcal R[x]$; however, if by hypothesis the equality
$$
\mathcal R[x]=\mathcal S[x]
$$
holds then $y$ is in $\mathcal S[x]$ and thus $(x,y)$ is in $\mathcal S$: we conclude that $(y,x)$ is in $\mathcal S^{-1}$ which is in particular not empty. Moreover by analogous arguments we can see that if $\mathcal S^{-1}$ is not empty then even $\mathcal R^{-1}$ is not empty and in particular it contains $\mathcal S^{-1}$. So by extensionality we conclude that $\mathcal R^{-1}$ and $\mathcal S^{-1}$ are equals.
Now if $\mathcal R^{-1}$ and $\mathcal S^{-1}$ are equals then the equality
$$
\mathcal R^{-1}\circ\mathcal T^{-1}=\mathcal S^{-1}\circ\mathcal T^{-1}
$$
holds so that even the equality
$$
\mathcal T\circ\mathcal R=\big((\mathcal T\circ\mathcal R)^{-1}\big)^{-1}=(\mathcal R^{-1}\circ\mathcal T^{-1})^{-1}=(\mathcal S^{-1}\circ\mathcal T^{-1})^{-1}=\big((\mathcal T\circ\mathcal S)^{-1}\big)^{-1}=\mathcal T\circ\mathcal S
$$
holds.
Finally if 5. holds then the equality
$$
\mathcal T^{-1}\circ\mathcal R=\mathcal T^{-1}\circ\mathcal S
$$
holds so that as above it is not complicate to conclude that the equality
$$
\mathcal S^{-1}\circ\mathcal T=\mathcal R^{-1}\circ\mathcal T
$$
holds. So as in 2. we conclude that
$$
\mathcal R^{-1}[X]=\mathcal S^{-1}[X]
$$
for any set $X$. Finally as in $3$ we conclude that
$$
\mathcal R=\mathcal S
$$
So first of all I ask if actually statements 1-5 are equivalent and so I ask if I well proved it. Specifically I ask to prove it more using a more clear argument since it seems to me it would be better to prove $1\to 5\to 2\to 3\to 4\to 1$ but unfortunately I am not able to prove $5\to 2$. So couls someone help me, please?
Answer: A possible solution could be the following which use the following theorem.

Theorem 1
If $\mathcal R$ and $\mathcal S$ are two binary relations then the following statements are equivalent.

$\mathcal R\subseteq\mathcal S$;
$\mathcal R\circ\mathcal T\subseteq\mathcal S\circ\mathcal T$ for any binary relation $\mathcal T$;
$\mathcal R[X]\subseteq\mathcal S[X]$ for any binary relation $\mathcal T$;
$\mathcal R^{-1}\subseteq\mathcal S^{-1}$.

Proof.

($1\Rightarrow 2$) So if $(x,y)$ is in $\mathcal R\circ\mathcal T$ then there exists $a$ such that $(x,a)$ is in $\mathcal T$ and $(a,y)$ in $\mathcal R$ but $\mathcal R$ is contained into $\mathcal S$ and thus $(x,a)$ is in $\mathcal T$ and $(a,y)$ in $\mathcal S$: we conclude that $(x,y)$ is in $\mathcal S\circ\mathcal T$.
($2\Rightarrow 3$) So if $\mathcal R\circ\mathcal T$ is contained for any relation $\mathcal T$ into $\mathcal S\circ\mathcal T$ then $\mathcal R\circ\operatorname{id}_X$ is contained into $\mathcal S\circ\operatorname{id}_X$. Now if $y$ is in $\mathcal R[X]$ then there exists $x$ in $X$ such that $(x,y)$ is in $\mathcal R$ but $(x,x)$ is in $\operatorname{id}_X$ so that $(x,y)$ is in $\mathcal R\circ\operatorname{id}_X$ and so in $\mathcal S\circ\operatorname{id}_X$: we conclude that $(x,y)$ is in $\mathcal S$ and so $y$ is in $\mathcal S[X]$ since by hypothesis $x$ is in $X$.
($3\Rightarrow 4$) So if $(y,x)$ is in $\mathcal R^{-1}$ then $(x,y)$ is in $\mathcal R$ so that $y$ is in $\mathcal R\big[\{x\}\big]$ which by hypothesis is contained into $\mathcal S\big[\{x\}\big]$: we conclude that $y$ is in $\mathcal S\big[\{x\}\big]$ so that there exists $x_*$ in $\{x\}$ such that $(x_*,y)$ is in $\mathcal S$ but $x_*$ must be necessarly $x$ and so $(y,x)$ is in $\mathcal S^{-1}$ since $(x_*,y)=(x,y)$ is in $\mathcal S$.
($4\Rightarrow 1$) So if $(x,y)$ is in $\mathcal R$ then $(y,x)$ is in $\mathcal R^{-1}$ and so in $\mathcal S^{-1}$: we conclude that $(x,y)$ is in $\mathcal S$.

So let's we prove the following corollary

Corollary 2
If $\mathcal R$ and $\mathcal S$ are two binary relations then the inclusion
$$
\mathcal R\subseteq\mathcal S
$$
holds if and only if the inclusion
$$
\mathcal T\circ\mathcal R\subseteq\mathcal T\circ\mathcal S
$$
for any binary relation $\mathcal T$.
Proof.
If $\mathcal R$ is contained into $\mathcal S$ then $\mathcal R^{-1}$ is contained into $\mathcal S^{-1}$ so that by thm. 1 the inclusion
$$
\mathcal R^{-1}\circ\mathcal T^{-1}\subseteq\mathcal S^{-1}\circ\mathcal T^{-1}
$$
holds. Now if $(x,y)$ is in $\mathcal T\circ\mathcal R$ then $(y,x)$ is in $(\mathcal T\circ\mathcal R)^{-1}=(\mathcal R^{-1}\circ\mathcal T^{-1})$ so that it is in $(\mathcal S^{-1}\circ\mathcal T^{-1})=(\mathcal T\circ\mathcal S)^{-1}$ and so $(x,y)$ is in $\mathcal T\circ\mathcal S$.
Conversely if $\mathcal T\circ\mathcal R$ is contained into $\mathcal S\circ\mathcal R^{-1}$ then for any $\mathcal T$ then $\mathcal T^{-1}\circ\mathcal R$ is contained into $\mathcal T^{-1}\circ\mathcal R$ and so by thm. 1 the inclusion
$$
\mathcal R^{-1}\circ\mathcal T=(\mathcal T^{-1}\circ\mathcal R)^{-1}\subseteq(\mathcal T^{-1}\circ\mathcal S)^{-1}=\mathcal S^{-1}\circ\mathcal T
$$
holds: so by thm. 1 we conclude that $\mathcal R^{-1}$ is contained into $\mathcal S^{-1}$ and so $\mathcal R$ is contained into $\mathcal S$.

Now by extensionality a set $A$ is equal to a set $B$ iff $A$ is contained into $B$ and vice versa: so using thm. 1 and crl. 2 is possibile to prove the following theorem which actually is a corollary of thm. 1.

Theorem 3
If $\mathcal R$ and $\mathcal S$ are two binary relations then the following statements are equivalent:

$\mathcal R=\mathcal S$;
$\mathcal T\circ\mathcal R=\mathcal T\circ\mathcal S$;
$\mathcal R\circ\mathcal T=\mathcal S\circ\mathcal T$ for any relation $\mathcal T$
$\mathcal R[X]=\mathcal S[X]$ for any set $X$;
$\mathcal R^{-1}=\mathcal S^{-1}$.
Comment: I haven't read through your Answer for accuracy, but this looks like a reasonable way to handle the slight awkwardness that you alluded to in the Question. Personally I would probably just show separately that each of 2-5 implies 1 (since the converse is obvious) without tying any of those proofs into each other, but the inverse trick you use here is also nice :)
Comment: Thanks for your opinion! ;-)


Create a domain on server to have all users on it + shared SQL Server 2012

Question: I'm new to Active Directory Domain Services and to Windows Servers in general.
I created a domain on Windows Server 2008 R2 using AD DS and I joined all company's PCs to this domain.
Now what I want is to Install SQL Server 2012 on a server to access its instance from these same PCs (and later TFS too). What I currently found is that I can't install it on the same server since SQL Server cannot be installed on a domain controller.
So what should I do here?
1- Is there any way to install SQL Server on the same server?
2- Should I get a new server and use it for SQL and TFS?
3- If I get a new server and install SQL on it, PCs on the specified domain will still be able to access SQL on the server even if it's not in the same domain?
Sorry but I'm so confused, I'm a real beginner with these issues.
Answer:
You should not install SQL Server on your Domain Controller.
Yes
Join the new server to the same domain as the clients. Why would you do otherwise?
Comment: Yes you're right, I realized that, I think I was a bit panicked!
Thank you.
Comment: Glad to help...
Comment: I'm new to serverfault, maybe you can help me to know why my question would get -1 to not do the same error in future questions please. Thanks.
Comment: I'd only add that, although not recommended by MS, SQL Server **can** be installed on DC. I have a Windows 2012 DC running MSSQL Standard 2014 without issues (and it used to run 2012 Trial and Professional editions in the past). When you need to have an SQL Server dedicated machine, why not to have one more DC?
Comment: From [Can I install SQL Server on a domain controller?](http://blogs.technet.com/b/mdegre/archive/2011/07/01/can-i-install-sql-server-on-a-domain-controller.aspx): Security issue: You cannot run SQL Server services on a domain controller under a local service account or a network service account. Functional issue: After SQL Server is installed on a computer, you cannot change the computer from a domain controller to a domain member. The reverse applies as well. There are also a whole host of other issues that come up when running the Windows Internal Database on a DC. Also performance issues.
Answer: You might consider using Hyper-V to install these servers as virtual instances on the same hardware.  A second server as a host would easily support all this.  I would also recommend adding a second domain controller as a VM to back up the first.
Comment: So, just to be sure I got it right, you mean I use Hyper-V on my current server to install two virtual servers, one to be my domain controller and one to install SQL Server and TFS?
If so, is it better than having two different physical servers? (security wise)
Comment: Depending on the level of reliability you need to build in, you might want two physical servers. The added benefit is two domain controllers. But only you can decide the best way.


Deep Learning approaches for Record Linkage

Question: Record linkage (RL) is the task of finding records in a data set that refer to the same entity across different data sources (e.g., data files, books, websites, and databases). Record linkage is necessary when joining data sets based on entities that may or may not share a common identifier (e.g., database key, URI, National identification number), which may be due to differences in record shape, storage location, or curator style or preference.
(Source Wikipedia)
There are several approaches to solving a record linkage problem , deterministic, probabilistic , machine learning approaches etc.
I am looking for deep learning approaches for solving record linkage use cases.
Could find below work on "Entity Resolution Using Convolutional Neural Network"
https://www.sciencedirect.com/science/article/pii/S1877050916324796
Please share thoughts on how to solve a record linkage problem using deep learning.
Answer: One classic method for linking text documents uses cosine similarity on TF-IDF features. A simple way to extend this would be to use Doc2Vec or similar document embeddings instead of TF-IDF - cosine similarity of word/document embeddings captures semantic similarity (some people might point out that word embeddings aren't technically deep learning, but I think that author might find these methods useful).

Second approach is to try to learn distance function that corresponds to item dissimilarity. This is analogous to record linkage method that uses TF-IDF features (use of distance function is analogous to cosine similarity in this model).
Siamese networks can be used to learn such distance functions. They are essentially networks that given two examples return their similarity/dissimilarity. "Siamese" comes from the use of shared weights for hidden layers (they encode both inputs in the same way).
Here you can see an example talk on using Siamese Networks for similar task.
If you want to read further on Siamese Networks I encourage you to look up One Shot Learning, which is somewhat similar to record linkage.
Comment: Thank you @Jakub Bartczuk, using Siamese networks to learn the distance function is interesting.
Answer: You can get good results just by looking at vector embeddings. An example using TerminusDB and OpenAI here:
https://terminusdb.com/blog/entity-resolution-with-ai-dblp-and-acm-publication-benchmark/


EntityFieldQuery and how to use aggregate functions SUM, ARG and MAX

Question: I have entity type payment and need count all payments (calculate based on field_total), if its hard to undersand, I show it on example:
Payment A - $10
Payment B - $20
Payment C - $30
And I need query that SUM all values. In my example it will be $60.
<code>$query = new EntityFieldQuery();
$entities = $query->entityCondition('entity_type', 'node', '=')
                    ->entityCondition('bundle', 'payment')
                    ->execute();
</code>
What is best method to solve this problem? I could select all nids and load it. But this is not very good if I have thousands nodes, or Iam wrong?
Thank you for advice.
Answer: EntityFieldQuery does not support Aggregations/Expressions.
You have three options, basically:

Load all nodes, make the summary in PHP. As you already said, not a good idea if you expected thousands or even just hundreds of payments.
Write an SQL query, which should be rather simple since your field is probably only attached to the relevant bundle so you only need to query a single table and get the SUM() of the field value of all values. To be sure, you could also join the node table and add a condition on the node type. Although not nice, doing this is IMHO fine for custom code, where you know that the field will always be in the database. If you want to support different field storage implementations like MongoDB, not so much...
Lastly, a bit more work, you could maintain a global summary that you update every time a payment is created/updated by implementing a corresponding node/field hook (like hook_field_insert(), as you can see this is what taxonomy.module does...). This would probably be a good idea if you expected that you need the summary frequently. Also, while it might be tempting to use variable_get()/set to maintain a single value, that's not a good idea as it results in cache clears every time a variable is written.
Answer: To expand on Berdir's answer, EntityFieldQuery is not an SQL query builder. Sure, core contains an SQL implementation because core happens to ship with a Field SQL storage module. As such, only a very limited set of functionality is provided. In other words, how do you run AVG on the score of your YouTube videos (which could be a field just fine).


Can this expression be expressed in another way?

Question: I saw this meme, pretty daft IMO which "ironically" shows how $9+1\div10=1$ would be expressed (from what I understood) by inverting the order. I was asking asking myself, how many errors does it have? Also, why is $(x+y)^2$ produced as $x^2+y^2$ and not as $x^2+2xy+y^2$ using the binomial theorem (isn't the first wrong?).
Comment: It is only *sometimes* that wrong things are right. For example, $\frac{16}{64}=\frac{1\not\, 6}{\not\, 6\,4}=\frac14$
Comment: @HagenvonEitzen I WannaCry rather than laughing.
Answer: The $(x+y)^2$ is produced as $x^2+y^2$ for humoristic reasons, and you are correct, it should be $x^2+2xy+y^2$. This meme tries to humoristically make fun of some mathematical mistakes.
Basically all of the calculations are wrong, with each another one in more and more ridiculous way.
Answer: All of the calculations are wrong, that's the point.
Answer: They're all wrong.

$ 9 + 1 \div 10 = 9 + (1 \div 10) = 9.1 $ The mistake is grouping $(9+1) \div 10 = 1$, which ignores the order of operations.
$(x+y)^2 = x^2 + 2xy + y^2$ as you noted. This is a reference to the common mistake, Freshman's dream.
$(x + y)$ is an entire term that you can't cancel with the numerator, i.e. $\frac{x}{x+y}$ is fully simplified.
$\sin$ is a function. The letters are not variables and can't be divided out.
Answer: They are all wrong, but they are common mistakes that students make.
$9+1\div10=1$ seems to be a stupid and popular meme around the interwebs.  In my mind the $\div$ symbol makes for confusing notation and should be avoided whenever possible.  However, in this notation, division should happen before addition.
The second one.  $f(x) = x^2$ is not a linear function.... at least that would be the high-fallut'n way of saying it. There is a small (and important) subset of linear functions, but most are not.  So $\sqrt{x+y} \ne \sqrt x + \sqrt y$ and $\sin (x+y) \ne \sin x + \sin y$
Multiplication distributes over addition.  And, that is what the binomial theorem is all about.
The next one.
Division is not commutative, and it distributes one direction but not the other.
It distributes this way: $\frac {a+b}{c} = \frac ac + \frac bc$
but not this way:
$\frac {a}{b+c} \ne \frac ab + \frac ac$
And if you choose to cancel, you must cancel in every term in the numerator and in the denominator.
and the last one is just silly.


How xy different from x+y in "THE LAWS OF THOUGHT, ON WHICH ARE FOUNDED THE MATHEMATICAL THEORIES OF LOGIC AND PROBABILITIES" by Boole, George

Question: I was self studying https://archive.org/details/investigationofl00bool but could not go past the below sections.
Chapter 2:6 if x alone stands for " white things," and y for " sheep," let xy stand for " white sheep ;"  and then
Chapter 2:11 In this and in all other respects the words " and" " or" are analogous with the sign + in algebra, and their laws are identical. ....Let x represent " white things," y, "sheep" and let + stand for " and" and " or".
What is the difference between xy representing white sheep and x+y which may represent white and sheep? Am I missing something?
Comment: Boole's original work is probably not the best introduction to propositional logic for a modern reader. He was grasping towards the first algebraic treatment of truth values and didn't have the benefit of generations of teaching experience to build on for streamlining the presentation. One of the improvements since his time is that his _notation_, overloading the algebraic notations for multiplication and addition, is somewhat rarely used nowadays. It has been found to be clearer to use specific symbols $\land$ and $\lor$ (or similar) for these meanings.
Comment: Well, until they add $\wedge$ and $\vee$ keys to the standard keyboard, the operator overloading will continue to be used in computer science courses for the forseeable future.
Comment: See [The Algebra of Logic Tradition](https://plato.stanford.edu/entries/algebra-logic-tradition/) as well as [George Boole : The Laws of Thought (1854)](https://plato.stanford.edu/entries/boole/index.html#LawsThou1854).
Comment: (Though the algebraic notation is still going strong in _some_ application areas -- it doesn't seem to be uncommon to meet it in digital logic, for example).
Comment: Thanks. That cleated my confusion. I was reading Treatise on probability by Keynes and Principle of Indifference notes lead me to this book.
Answer: $xy$ means things that are white and sheep, i.e. white sheep
$x+y$ means things that are either white or sheep. This includes white sheep, but also such things as white saladshooters, and black sheep.
Comment: @sreenadha Boole didn't mean $+$ was sometimes "or" and othertimes "and", he was indicating that $+$ meant *inclusive-or* so: $x$ and/or $y$ to emphasise that *at least one but maybe both* are true.
Comment: Pardon my ignorance, what that statement meant ....+ stand for "and" or "or" ? Why are we using "or" why not "and"
Comment: @user566158 I was just reading the relevant passage ... Boole does indeed something a bit strange here; it seems like he is using the $+$ to represent either the 'or" or the 'and'.  For example, he seems to use x + y = y + x to indicate that both 'x and y = y and x', as well as 'x or y = y or x'. So that is indeed rather confusing. However, in modern notation the $+$ is definitely interpreted as just the 'or' ... Indeed, you may be better off reading some more modern texts on formal logic, which are not only more up to date, but also more readable than Boole's book.
Comment: Thanks. That cleared my confusion.
Comment: @Graham Kemp, In Boole's word, chapter 2 page 33, Let x represent " men," y, "women ;" and let + stand for " and" and " or" then we have x+y=y+z.......Let the symbol z stand for the adjective " European," then since it is, in effect, the same thing to say " European men and women," as to say " European men and European women," we have z(x+y) = zx + zy.   I tend to agree more with Henning Makholm and Bram28 than inclusive-or argument which though seems plausible creates lot more confusion with his text.
Comment: @sreenadha Agreed. In the passage 2.11 Boole is saying that when we say "men and women", we use the word "and" to indicate that the two classes are distinct.  But sometimes we use the word "or" for this as well. Boole then uses + to capture either of those uses .. (which kind of makes sense: the class people would be the 'aggregate' of men and women).  But the confusing thing is that with other uses of the word "and" we mean that something is both. That's why $xy$ would be white sheep (things that are white and a sheep).
Comment: @sreenadha To add to the confusion, Boole's use of the +, which as pointed out seems compatible with an exclusive or, is incompatible with the modern use of + in logic as an inclusive or. I am also wondering what Boole would actually say about $x+y$ if $x$ is white and $y$ is sheep .. because we're not talking about separate classes here that we aggregrate into a larger one using the +. (and maybe it is exactly that difficulty that made us change the use of + from exclusive or to inclusive or ...)
Comment: @sreenadha You know, I *really* want to reiterate my point to read a more modern treatment of logic! Boole's notation is not only confusing, but as I am reading passage, but not all that nice to work with either (he uses a - for 'exclusion' ...1-x instead of x', etc.)
Answer: $xy$ here is something that's both white and shep, while $x+y$ is something that is white, or is a sheep (or both).
Comment: @sreenadha Boole never used $+$ for just "and" alone.  He was trying to emphasise that it meant "and/or"; that is *inclusive-or*.
Comment: Why can't we use "and" for +?  ....+ stand for "and" or "or" ? Why are we using "or" why not "and"


Bootstrap/CSS: Layout issues inside 3-columns layout

Question: I'm using Bootstrap 3 and I have a full width row with 3 col-lg-4, each column with an icon, a title and a subtitle. But now I'd like to place the icon and text side by side, like this:

Initial HTML:

Title

Lorem ipsum

    </div
    …
    …

.cover-container {
   margin: 0 auto;
   width: 100%;
   padding: 0px;
}
.tile {
  text-align: center;
}

I've tried with 8 col-lg-2, but I don't like that it shows each column as a row on a smartphone.
I prefer to use 3 centered col-lg-4, but I can not place elements like in the mock-up,
can you help? Thanks…

SOLUTION:

Title

Lorem ipsum

            …
            …

.cover-tile-image {
    text-align: right;
    padding-right: 5px;
}

.cover-tile-text {
    padding-left: 5px;
}

Answer: I'm just going to point out to y'all that:
class="col-xs-4 col-sm-4 col-md-4 col-lg-4"

is extremely redundant, and equivalent to the much-simpler:
class="col-xs-4"

I encourage y'all to read Bootstrap's grid docs:

Grid classes apply to devices with screen widths greater than or equal to the breakpoint sizes, and override grid classes targeted at smaller devices. Therefore, applying any .col-md- class to an element will not only affect its styling on medium devices but also on large devices if a .col-lg- class is not present.
Answer: I'm still new in using bootstrap 3. Check the demo below I created. Hope this helps.
Fiddle
HTML

Title

Lorem ipsum

CSS
.tile, .info {
    float:left;
}

Answer: Here is the solution :) with working example.
Find here more information about the grids.
This solution is also RESPONSIVE READY.
HTML:

Title

Lorem ipsum

Title

Lorem ipsum

Title

Lorem ipsum

CSS
h3{margin:0;}

Comment: Thank you very much! I started with this solution with nested rows, added some styles to better fit my needs and combined with @cvrebert's suggestion!
Answer: HTML

Title

Lorem ipsum

Title

Lorem ipsum

Title

Lorem ipsum

CSS
#wrapper
{
    width:1000px;
    margin:0px;
    padding:0px;
}
.columns
{
    float:left;
    display:inline-block;
    width:120px;
    height:110px;
    }

Demos

Fiddle Demo
Result Demo

Output


How to configure the popular TAGS block

Question: I need this effect, but I don't know how to do that:
Comment: Welcome to Drupal Answers! I am sorry, but _I need to implement this as you see in the screenshot_ questions are expressly off-topic for us, as per Help Center.
Answer: cctags

flexible configuration and advanced features for theming clouds.
  (individual files of templates for tags, the names of the dictionary, pages clouds)
Possibility (in settings include several dictionaries (as for
  dictionaries with tree structure - in addition to enable / disable the
  desired level)) tags. Automatic generation of the required number of
  blocks and / or pages. Generation unit of Tag attached to the node.
  Generation of block-related materials for the node (via tags) Cloud
  members users (block and page) (with weights proportional to the
  number of published material)

TagCloud

Tag clouds is a light weight stable module that provides span tags,
  optimizations; It is also singular in purpose and stable.
Provides a link to the node and not the taxonomy display if the node
  count is equal to 1. This avoids taxonomy pages with only one node.
  Defaults to On, can be turned off

OR you can do it
You can do this with views:

create a view of terms
add a required relationship to "content using tags"
set use aggregation: yes
add the "term name" field a second time, set one of the to
"aggregation type: count"
add "term name" as a sort criteria, set "aggregation type: count"
Answer: There are certain modules that will do this.
TagCloud

This module Provides a TagCloud and List of Tags.
  Tag clouds is a light weight stable module that provides span tags,
  optimizations; It is also singular in purpose and stable.

Tagadelic

Tagadelic provides an API and a few simple turnkey modules, which
  allows you to easily create tagclouds, weighted lists, search-clouds
  and such.
  With the API you can build a module with a few lines of PHP, to turn
  anything that can be counted into a weighted cloud. Which can be
  presented to your users anyway and anywhere on your site.

cctags

module cctags - as an alternative to module tagadelic.
  flexible configuration and advanced features for theming clouds.
  (individual files of templates for tags, the names of the dictionary,
  pages clouds)

and with effects
Cumulus

Cumulus allows you to display your site's tags using Flash that
  rotates them beautifully in 3D. It works like a regular tag cloud, but
  is more visually exciting.
Answer: Hello You can use Tag Cloud Module for this purpose it is easy to use, following is simple method to show Popular Tags block

Unzip and copy the directory to /sites/all/modules.
/admin/modules activate TagClouds.
/admin/config/content/tagclouds set the Settings to what you want. See Module Settings bellow.
/admin/structure/block and find the block titled "Tags in [taxonomy name]" and move to where you want it displayed. See "Block Settings" bellow.

Your all set up!
Best Wishes
Rana

How to configure the popular TAGS block