DAGs with Ambiguous Temporal Ordering Between Nodes

Question: I am working on a project where I am attempting to estimate the causal impact of civil war peace agreements (treatment) on levels of violence (outcome). While developing the DAG, I came across an issue where some of the nodes cause each other (pictured below is a simplified version of the DAG designed to only include the nodes where this became a problem).

Peace agreements sometimes result in an agreement to authorize a third-party peacekeeping operation (PKO), but, in other cases, PKOs facilitate the establishment of a peace agreement. Sometimes, mediation between warring parties leads to the authorization of a PKO. In other cases, a PKO itself facilitates mediated talks between warring parties. There is no natural ordering to these phenomena. PKOs do not always precede mediation and mediation does not always precede PKOs (it depends on the context). Likewise, a similar relationship exists between peace agreements and PKOs.
One way of getting around this (I think, could be wrong) is to separate these three nodes into six nodes. Instead of PKO, Peace Agreement, and Mediation, I would have:

Mediation t-1 (Mediation preceding PKOs)
Mediation t (Mediation following PKOs)
PKO t-1 (PKOs preceding mediation or peace agreements)
PKO t (PKOs following mediation or peace agreements)
Peace Agreement t-1 (Peace Agreement preceding PKOs)
Peace Agreement t (Peace Agreement following PKOs)

Here is the DAG with the new lagged nodes incorporated:

I'm not sure that this actually solves the issue, however. After all, aren't PKO(t-1) and Mediation (t-1) & Peace Agreement (t-1) and PKO (t-1) mutually directed in the same way that their (t) counterparts are?
I was wondering if anyone could help with this issue.
Update on this question
Is another way to get around this is to assign an unobservable node to the dual-directed relationships? For example, consider the image below:

Take the PKO -> Mediation/Mediation -> PKO example. Mediation (t-1) can cause PKO (t) and PKO (t-1) can cause Mediation (t). Perhaps these can both be collapsed within the unobservable node given that the causal mechanism driving the variation in ordering is unknown?
Comment: This is definitely where I would start, but just to keep in mind that you must have longitudinal data to make this work at all. And this diagram only really allows you to handle two snapshots in time: two different times. If you want three data points, you have to add yet more nodes and arrows! A slight simplification is if you assume the Markov property: state only depends on the most recent previous state, and not two states back.
Answer: If you have bi-directional arrows it's not a DAG (i.e. not a Directed Acyclic Graph), full stop. The 'acyclic' means there are no loops in the causal model.
This is because in a DAG (see, e.g., Hernán and Robins, 2020; Pearl, 2000, etc.):

A uni-directed arrow means "the variable at the arrow's tail directly causes the variable at the arrow's head." A lack of an arrow between two variables means "no direct cause".
Unidirected arrows also define temporal ordering, as in "the variable at the arrow's tail occurred before the variable at the arrow's head occurred."

That said, there are other causal graph formalisms, including ones involving digraphs (i.e. arrows which go both directions). One such formalism is Levins' "loop analysis" (Levins, 1974, Puccia & Levins, 1986, etc.) which uses signed digraphs (i.e. two variables may be connected by no arrow, a uni-directional arrow, or a bi-directional arrow, and arrowheads have signs) to model causal systems defined by every variable being a direct or indirect cause of every variable in the system at future times, and where the sign indicates "direct causal increase" (positively signed arrowheads) or "direct causal decrease" (negatively signed arrowheads). Also temporal ordering does not exist between variables, unlike in a DAG. Rather, this signed digraph formalism "looks head-on at the arrow of time" so that all times during which the causal system exists with the specified relationships are represented.

References
Hernán, M. A., & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC.
Levins, R. (1974). The Qualitative Analysis of Partially Specified Systems. Annals of the New York Academy of Sciences, 231, 123–138.
Pearl, J. (2000). Causality: Models, Reasoning, and Inference. Cambridge University Press.
Puccia, C. J., & Levins, R. (1986). Qualitative Modeling of Complex Systems: An Introduction to Loop Analysis and Time Averaging (C. J. Puccia & R. Levins, Eds.). Harvard University Press.
Comment: Perhaps the title of my question was misleading. I do not believe that the relationship is cyclical. I acknowledge that the relationship is temporally ordered. The issue is that the causal relationship is not universally temporally ordered between the nodes. In Civil War A, Mediation -> PKO. In Civil War B, PKO -> Mediation. This is what I am struggling with. The best thing I've thought up so far is to acknowledge that background factors drive this ordering (the image from the recent edit).
Comment: That makes sense, Brian Lookabaugh!


log4net file appender not creating a file

Question: I have used log4net in the past without any trouble, however I am having some issues getting it to log to a file appender.  Very simple example.
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>

  </configSections>
  <log4net>
    <appender name="DebugFileAppender" type="log4net,Appender.FileAppender">
    <file value="Logfile.log" />
    <appendToFile value="true" />
    <layout type="log4net.Layout.PatternLayout">
      <param name="ConversionPattern" value="%5p [%d] - %m%n" />
    </layout>
    </appender>
    <root>
      <appender-ref ref="DebugFileAppender" />
    </root>
  </log4net>
</configuration>

Comment:
Answer: Does the executing account have permission to write to the folder?
Comment: I think so.  I opened the permissions on the write directory
Comment: If that does not help, try turning on the internal log4net debugging. More info here:
http://log4net.sourceforge.net/release/18.104.22.168316/doc/manual/faq.html#internalDebug
Answer: Try set write permission for the log folders for IIS_IUSRS
Answer: The comma in the type attribute should be replaced with a period: type="log4net.Appender.FileAppender"
You should be seeing an error similar to
log4net:ERROR Could not create Appender [file] of type [log4net,Appender.FileAppender]. Reported error follows.
...


Erro script Analytics Google

Question: Olá, estou tendo um erro no script gerado pelo Google do Analytics. Que está exatamente desta forma:

No console do navegador aparece o seguinte:

Uncaught SyntaxError: Unexpected token ;                (index):12

Essa linha 12 é exatamente essa:
window.dataLayer = window.dataLayer || [];

Eu não consigo entender o motivo deste erro, visto que estou copiando o que o Google me forneceu, colei no inicio da tag conforme especificado pelo Google.
Comment: Dá uma olhada direito, porque estou achando que o erro não é nessa linha, até porque não há nenhum erro nela.
Comment: Eu também acho que seja isso, só que esse script é o primeiro  carregar no head, tipo não tem nada antes que possa estar dando pau.
Answer: Tente com este Codigo:

(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-ID-ANALITCS', 'auto');
ga('send', 'pageview');
Comment: Já testei com essa tag, mas dá erro também.

Erro script Analytics Google


Finding limits with $2\log(x) + \sin(\log(x))$ - where to go with this?

Question: I'm trying to find, for the following function: $$U(x) := 2\log(x) + \sin(\log(x))\ ,\ \ x > e$$ the limit: $$\lim_{t \rightarrow \infty}\dfrac{U(t x)}{U(t)}$$ My initial idea was to expand $\sin(\log(x))$ into its polynomial form but the result is a series of logs. The condition that $x>e$ seems to suggest that these logs are all greater than $1$ for any $\alpha >0$ and therefore non-negligible. But then, I am struggling to say what that limit is above because surely each successive power in the expansion contributes even more to the limit than the previous one?
Any help is appreciated. Thank you.
Comment: Remark that $\sin$ is bounded. Moreover, write $\log(tx)=\log t+\log x$.
Comment: How about $x$ in the limit? Is it fixed?
Comment: Ah ok, that's different
Comment: Sorry I made a typo; I've relabelled things so that it makes more sense now. Yes I believe it is fixed.
Comment: Thanks for the hint. I've posted an idea below too - not sure if it's right.
Answer: For fixed $x>e$ we have
$U(tx)=2 \log(t)+2 \log(x)+ \sin(\log t+ \log x)$
Now consider
$\dfrac{U(t x)}{U(t)}=\dfrac{\frac{U(t x)}{\log t}}{\frac{U(t)}{\log t}}$
and show
$\lim_{t \rightarrow \infty}\dfrac{U(t x)}{U(t)}=2$
Edit: we have $\lim_{t \rightarrow \infty}\dfrac{U(t x)}{U(t)}=1$
Comment: Isn't the limit $1$? Don't we have: $\Bigl|\dfrac{U(tx)}{U(t)}\Bigl| \leq \dfrac{2(\log(t) + \log(x)) + 1}{2\log(t) - 1} = \dfrac{2(1 + \frac{\log(x)}{\log(t)}) + \frac{1}{\log(t)}}{2 - \frac{1}{\log(t)}}$ (and a similar expression for the other inequality) giving a limit of $\frac{2}{2} = 1$?
Comment: Yes, you are right !


What are ways to use the palindromic property of a palindromic scale?

Question: A palindromic scale, as I've seen it described on a number of music theory websites, is defined as one whose sequence of intervals is the same when ascending and descending.
For instance: D Dorian (D E F G A B C D) is palindromic because it consists of the interval pattern WHWWWHW, which is a palindrome.
Why should we care?  Is there anything that can be done with D dorian that can't be done with C Ionian because it's a palindromic scale?
Comment: As a fan of Olivier Messiaen, who used similar ideas (palindromic rhythms, modes of limited transposition), I'd say it may be useful for generating ideas in the composition process, but ultimately it's unimportant to the listener.
Answer: One possible use is that inversion around a particular pitch axis will create the same scale collection.
Let's use D Dorian, since that was in your example. If we invert D F G A C B E D around tonic (or G♯), the result is still a D-Dorian collection: D B A G E F C D. This is important because if we inverted those same scale degrees in a different collection—let's say D major (D F♯ G A C♯ B E D)—around tonic we'd be left with yet another collection: D Phrygian (D B♭ A G E♭ F C D).
In other words, palindromic collections allow for inversion around tonic while staying within the same collection, which is not true for non-palindromic collections.
Comment: @awelotta If the given melody or tone sequence is a part of a palindromic scale collection, yes.
Comment: Interesting - +1, but mainly academic? Would most of us have a clue something was written with this phenomenon in mind? Not knocking, just curious!
Comment: I really appreciate the final summary sentence. Would it be correct say that "staying within the same set" is synonymous with "staying in the same tonality?" I think "yes", but maybe I'm overlooking something.
Comment: @Tim, I agree many probably wouldn't _hear_ this, but then again it takes a lot of education to recognize all the different fugue devices, but most anyone will appreciate the overall effect even if they aren't aware of what's happening technically.
Comment: So is this basically saying that an inverted melody or tone sequence will have the same tonality/modality?


Why does Northern Ontario get their own entry in curling events such as the Brier?

Question: Was watching a few old highlights of the Brief from the 50's today, and found that Northern Ontario always had their own entry, even then. Is there a reason why even thought it's not it's own province, it would be given an entry as if it were in events such as the Brier and the Scotties?
Answer: It's history more than anything.

At the first Brier in 1927, six provinces were represented – New
  Brunswick, Northern Ontario, Nova Scotia, Ontario, Quebec and
  Saskatchewan, along with teams from Toronto and Montreal. In 1928,
  Alberta and Manitoba joined the championship. In 1932, city
  representation was dropped, but in 1936, Prince Edward Island and
  British Columbia were added, followed by Newfoundland in 1951 and
  Yukon/Northwest Territories in 1975.  Source

Northern Ontario has always felt it is "different" from Southern Ontario in a lot of ways, geographically as well as socially and politically.  There have also been movements to try to get itself separated from the rest of Ontario, but they have failed.  Overall, the people of Northern Ontario believe they have a vastly different identity than Soutnern Ontario, and they want to keep it that way.  Source
Keeping their separation in events like curling (and specifically the Brier) help keep that identity alive.
Comment: I have not found a definitive source for the answer to your question.  I think if you combine the info I provided, you can extrapolate it into something along the lines of "Northern Ontario exists because the teams felt it was different than 'just' Ontario".


Create "suggest an edit" functionality for anonymous users on nodes

Question: I'd like to provide a way for anonymous users to submit nodes, but also provide suggested edits to existing nodes. The core node revision system cannot support this.
How would one implement a revisioning system that allow anonymous users to save updates which moderators approve. I'm assuming Workbench might get me halfway there.
Edit: like you find on Foursquare and Yelp, for community suggestions on editing venues.
Comment: Are you trying to reinvent [Revision Moderation](https://www.drupal.org/project/revision_moderation), only with different wording of description>?
Comment: I think, it's the ability to create revisions which queue up. Like the edit functionality found on Foursquare or Yelp (will update question with that reference.)
Comment: So all ou want to do is to allow one role to edit, but don't allow their revisions to be automatically published? Probably module  i linked is for you
Comment: But that's unmaintained and D6, so not quite
Comment: well... worth it as idea / code source, not as ready made solution, true that. that's why i didn't post an answer.
Comment: gotcha :) yeah I was hoping to not have to do that, it's a pet project and more of a "I wonder if"
Answer: I think you may want to check out the Workbench Moderation module. Basically, you'd configure a workflow that allows anonymous users to 'edit' a node, but their edits would trigger the workflow. From an admin experience perspective, you may want to configure it to trigger an email notice whenever updates have been made, or otherwise alert content admins that suggestions have been made.
One potential gotcha is the way that Workbench handles revisions -- it doesn't fork. Meaning that if you publish a node, that becomes revision 1. If someone comes along and edits it, they suggest revision 2. If a third person comes along, they'll see that the node is on revision 1, but when they go to edit the node, they'll see it's on revision 2. Any changes would be saved as revision 3, and so on. You may need to extend the module with your own custom workflow if you're looking for forking behavior (think pull requests on GitHub).


Reducing the dimensionality of a problem

Question: My particular application needs me to build a linear model with a strong correlation structure amongst the independent variables. The dimensions of the problem are high, for instance 1million X 200. That is 1 million observations of 200 variables.
Since the problem is strongly collinear, I need to use some sort of regularization. I want to use non negative lasso ( due to the structure of the problem ). I am solving this in R and using the package penalized.
Now this is the problem I am running into, a calculation for the lambda1, the L-1 norm parameter using the optL1 function takes about 18 hours and counting. At this step I haven't even fitted for the final model. This is just the lambda1 evaluation step. This is just too long!
Unless I am missing something I need to shorten the time. One way to do it is to reduce the dimensionality of the problem. I am looking for some ideas to do that. The independent variables in my application have a relatively strong stable correlation structure. What will the implication if I just pick the topmost highly correlated variables and run the regression? Is there any other clever way to reduce the dimensionality of the problem?
Comment: How strong is a "relatively strong ... correlation structure"?  With one million observations, even correlations of 0.99 shouldn't be a problem, with the only caveat having to do with the condition number of the matrix $X^TX$.
Answer: Lasso, as far as I know, is not a tool for dealing with collinearity issues, it is a tool for variable selection that penalizes the models based on their complexity.
There are a variety of methods of dealing with collinearity. If your individual variables are not vital, you can use partial least squares regression, which is something like principal component analysis, except that it also incorporates the relationship between the linear combination of the independent variables and the dependent variable.
If your individual variables are important, you can use ridge regression, which allows some bias in order to reduce the variance that is a problem with collinear models.
If there are particular pairs of variables that are very highly correlated, you can try just picking one, either based on substantive reasons or randomly. With 200 IVs, there are rather a lot of pairwise correlations to examine (if my arithmetic is right, it's 19,900) but it's not completely intractable.
Comment: Thanks for the note. I guess ideally I also want a sparse model, and the structure has me looking for only non negative beta, hence the choice of NNLASSO. So a little more detail around the problem is the follows.
Comment: The IV's are all grouped. There is 6 such groups. Within each group, the correlations are very strong. So there are approximately 35 odd variables per group. I was just thinking (maybe naively) to pick say the top 5 variables per group that have the highest correlation with the dependent var. And then use NNLASSO on the reduced problem.
Comment: Given that the IVs are grouped, I would suggest that you use principal component analysis to get one variable for each of the six groups, and then use those 6 variables in a regression. You would not then need NNLASSO, you could easily include all 6 variables (you could still use lasso if you wanted, but it wouldn't really be needed).
Comment: Thanks for the note, can you please elaborate more on that, I apologize I am a n00b to PCA. How do you combine all the variables to do a PCA? What about the correlation between the variables that belong to different groups, in others words there might be a need for regularization after PCA too?
Comment: Since you are using R you could look at the princomp function in the stats package. You would then, I think, use the scores from each of 6 runs of the function, then combine them into a data frame with your dependent variable and then run regression.
Comment: Ok, how about sparsity, I don't necessarily want a weight on all of the variables?
Comment: I don't understand. PCA will weight the variables to create a single variable out of each of the six sets.
Comment: I guess what I mean is that I dont want a weight on all of the variables when creating the PCA. In the worst case PCA will give some non zero weight to each of the 35 variables in the group. I want to do this due to other structural reasons of the problem.
Comment: Then I don't know what to suggest


Oracle APEX 5 Select List Error ORA-01400

Question: i have a Problem with Oracle APEX 5.
I have a Popup with Textareas and with a Select List. I can write the Data of the Textareas without Problem into my Database, but i can not write the Data of my Select list into my Database. I alway get the Error ORA-01400 Server Error Insert of NULL imposible.
I also tried to write Hard data in the Table so i Changed the PLSQL to:
begin
INSERT INTO STATUS_TO_CHANGE (CHANGE_ID, CHANGE_STATUS_ID)
VALUES (:P2_CHANGE_ID, '43');
end;

and it worked without Problems :(
Maybe someone can help me :O
I use a own button to save the Textarea and a own button for the select list.
Code of my Dynamic Action:
begin
INSERT INTO STATUS_TO_CHANGE (CHANGE_ID, CHANGE_STATUS_ID)
VALUES (:P2_CHANGE_ID, :P2_STATUS_DESCRIPTION);
end;

Name of the Select List: P2_Status_Description
Select List with save Button .jpg
Error MessageData in the Table Change_Status which is shown in the LOV in the Select list
Thanks to all :)
Answer: I can't see images right now.
Anyway: this is, I believe, a "Execute PL/SQL Code" dynamic action, right? You put that BEGIN-END block into the "PL/SQL Code", but - did you put P2_CHANGE_ID and P2_STATUS_DESCRIPTION into Items to Submit? If not, do it and then try again.
Comment: OMG. Thank you it worked. I created the Dynamic Action so many times that i forgot this option. Thank you very much! :)


Imprimir texto de retorno json no meio do código - JQuery/JavaScript/JSON

Question: Tenho um gráfico que é criado conforme o ano, que é passado por AJAX e retorna um JSON contendo os valores pra criar o gráfico. Pois bem, eu envio o AJAX com o ano, retorno o JSON, porém... Dá erro, ele cria tudo errado, segue meu código.
Meu controlador onde retorno os dados e crio o JSON:
public function comparativo_json(){
    $html = '';
    $data = $this->relatoriosmodel->getComparativo($this->input->post('ano'));

    $html .= '[';
    foreach($data['mesesFuncionario'] as $func):
        $html .= "{";
            $html .= "name: '".$data['funcionarioDados'][$func['id']]->nome."',";
            unset($func['id']);
            $html .= "data: [ "; foreach($func as $f){ $html .= $f.','; } substr($f,-1); $html .= "]";
        $html .= "},";
    endforeach;
    $html .= ']';

    echo json_encode($html);
}

Meu JS:

O JSON que ele retorna, que por sinal está correto, conforme o padrão pedido pelo plugin:
[
    {
        name: 'Richard Feliciano',
        data: [ 0,0,0,0,1816,17100,2400,0,0,0,0,0,]
    },
    {
        name: 'Ewerton Melo',
        data: [ 0,0,0,0,0,12400,0,0,0,0,0,0,]
    },
]

Imagem contendo o erro:

Só preciso fazer com que o retorno data seja impresso corretamente.
Comment: A unica coisa que me ocorre é que esteja a esquecer de fazer o [JSON.parse](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/parse), porque de resto parece-me bem: __http://jsfiddle.net/yb3Nt/__ Teste assim: `series: JSON.parse(data),` Teste também `console.log(data);` ou `console.log(typeof data);` e comente aqui o que obtêm.
Comment: O problema está especificamente na impressão do `series: data,`
Comment: Fiz o `console.log(typeof data);` e ele retorna como `string`, logo abaixo fiz o parse `var dataJSON = JSON.parse(data)` e retornou erro: `Uncaught SyntaxError: Unexpected token n`
Comment: De fato deve ser a conversão para JSON dessa string
Comment: Ok, e o que obtêm se fizer `console.log(data);` ?
Comment: `[{name: 'Richard Feliciano',data: [ 0,0,0,0,1816,17100,2400,0,0,0,0,0,]},{name: 'Ewerton Melo',data: [ 0,0,0,0,0,12400,0,0,0,0,0,0,]},]`
Comment: Ele retorna o padrão do JSON certinho, mas como string. Tentei `JSON.parse(data)` ou `$.parseJSON(data)` e o erro é o mesmo apresentado no comentário acima...
Comment: E se parar pra analisar, eu dou json_encode() logo no retorno do valor.
Comment: Tentei também o `JSON.stringify(data)` que conforme consultei em alguns fóruns também serve para converter string em JSON, porém, ele mantém como string. Fiz direto só pra testar `console.log(typeof JSON.stringify(data));` mas mesmo assim continua como string.
Comment: Ou você cria um array/objeto e usa o json_encode() ou então cria uma string e não usa o encode. No caso da segunda opção (que é o que está a fazer), coloque aspas `"` à volta da chave, e valor que seja string, de cada objeto. E retire as virgulas a mais. Aí essa string vai ser aceite pelo JSON. Dê uma olhada aqui: http://jsfiddle.net/n3ARn/
Comment: @Sergio, muito obrigado pela ajuda, mas as dicas passadas abaixo resolveram o problema. Muito obrigado mesmo pela disposição em ajudar. Um abraço.
Answer: Existe um problema aqui:
foreach($data['mesesFuncionario'] as $func):
    $html .= "{";
        $html .= "name: '".$data['funcionarioDados'][$func['id']]->nome."',";
        unset($func['id']);
        $html .= "data: [ "; foreach($func as $f){ $html .= $f.','; } substr($f,-1); $html .= "]";
    $html .= "},";
endforeach;

O formato da string json não pode ter uma vírgula no fim, ela só separa os objetos. Segue o exemplo que você deu:
[{name: 'Richard Feliciano',data: [ 0,0,0,0,1816,17100,2400,0,0,0,0,0,]},{name: 'Ewerton Melo',data: [ 0,0,0,0,0,12400,0,0,0,0,0,0,]},]

Ele deveria ser:
[{name: 'Richard Feliciano',data: [ 0,0,0,0,1816,17100,2400,0,0,0,0,0,]},{name: 'Ewerton Melo',data: [ 0,0,0,0,0,12400,0,0,0,0,0,0,]}]

Repare que removi a última vírgula. Não entendo de PHP, mas o que você precisa fazer é verificar para que ele não adicione a última vírgula.
Depois de corrigir isso é só usar JSON.parse(data); que irá funcionar perfeitamente.
Comment: Fico feliz que tenha dado certo. Com strings JSON é muito importante manter o formato correto quando convertemos para um objeto em determinada linguagem. Você também pode tentar usar métodos padrão em PHP para formar strings JSON ao invés de escrever a string à mão, isso torna seu código menos suscetível a erros!
Comment: Talvez você ainda tenha problemas por não usar aspas dupla, mas confira isso... se der problema no parse, mude para aspas duplas na sua string JSON: [{name: "Richard Feliciano",data: [ 0,0,0,0,1816,17100,2400,0,0,0,0,0,]},{name: "Ewerton Melo",data: [ 0,0,0,0,0,12400,0,0,0,0,0,0,]}]
Comment: Muito obrigado era isso mesmo, usei a função do PHP `substr($html,0,-1)` para remover o último elemento da string, nos pontos onde encerrava com `,` e funcionou perfeitamente, retornou o gráfico.


Transforming old data to "match" updated data

Question: I have a couple of large data sets, one from an older machine and the other from a newer better version. I want to find a value that I could apply to the old data to basically have the new data's values. What is the best way to find this value in R?
Answer: I imagine you are hoping that it is a simple linear transformation. First, I would do a "pickup sticks" plot between the two machines. Essentially, just a line for each animal between the measurements from both machines to get an idea of the direction of the effect you should expect. The plot is usually messy but you can usually see a trend if there is one that will help to validate what you find in your statistical model. If there is a solid trend that doesn't tend to deviate more as the measured values increase or decrease (some subjects always will but in general) then you probably have a linear transformation on your hands.
If that is the case, I would recommend a linear model (the call in R is lm() for linear model). You should just have one predictor variable, "Machine" and the estimate (i.e. b) that you get will be the linear slope of one machine to the other depending on how your Machine variable is coded. Make sure you model does a good job of predicting, your R squared should be quite high (~.80 or higher; 1 means perfect prediction of every data point by your model). If it does a good job, then your error is low and your transformation should be pretty accurate. Check your residuals (i.e. errors) as well. These should have a relatively normal (Gaussian) distribution if you have a linear relationship between the 2 machines.
However, you also want to link the data of one subject's sample to the other sample from the same subject. Now you technically have a repeated measures analysis. Machine is your repeated measure. A Repeated Measures ANOVA could do this pretty easily, but if you have missing data for subjects on one machine and not the other, or you have some variability that makes you think "this looks pretty linear, but there are definitely a concerning chunk that deviate from that" you will want a method which can account for these issues. A Linear Multilevel Model (aka LMER, or LMM) is the way to go. It is just like an LM with a few tweaks. You still get an estimate just like in an LM that is interpreted the same way but you also can specify a random (as opposed to a fixed/main effect). The syntax for this is below:
<code>library(lme4)

model <- lmer(Y ~ Machine + (1|SubjectID), data=yourdata)
summary(model)
</code>
Back to the estimate. The estimate for Machine is the slope of the "transformation" line in this case between OLD and NEW. You should be able to multiply OLD by the estimate to get a value pretty close to NEW. The intercept estimate will not matter. If you need some help with the syntax let me know.
Comment: The data is measurements taken from semen collections such as volatility and size. The samples were split between the two machines for each animal so it is as close to just differences in the machines as we can biologically get. The goal of this is to find a general formula that we can use to make old data, not necessarily this data, more accurate.
Comment: @CashCab1221 Thanks that really helps! I have edited the answer above to help. If you could post the pickup sticks plot I mentioned, it may help more.
Comment: Thank you, I'm already pretty proficient in R and had basically already done this but didn't know if there was a different better way.
Comment: @CashCab1221 There may be. If you have missing data for some subjects on one machine or the other, some decent variability in that pickup sticks plot, or want to specify that one individual subjects samples should be more like one another than they are like other individuals' samples, you may instead want to use a linear mixed effects model from the lme4 library. It generalizes better out of the original sample than LM. You just need to specify the random effect of "subject" (the intercept linking the subjects' samples together). This would be the most accurate way I know. Edited above to suit


Programmatically set date fields in Drupal 7: Date, Date (ISO Format) and Date (Unix Timestamp)

Question: Using the Form API and hook_form_alter(), how can I manually set (force) the values for each of Drupal 7's date field types using a valid Unix Timestamp?
When the form is finally submit the field_date types are always empty arrays.  Below is the broken code I'm working from.
hook_form_alter(&$form, &$form_state, $form_id) {

    // http://drupal.org/node/1455576 - Date
    $form['field_date'][LANGUAGE_NONE][0]['value'] = time();

    // http://drupal.org/node/1455576 - Date (ISO Format)
    $form['field_date_iso_format'][LANGUAGE_NONE][0]['value'] = time();

    // http://drupal.org/node/1455576 - Date (Unix Timestamp)
    $form['field_date_unix_timestamp'][LANGUAGE_NONE][0]['value'] =time();
}

I should mention the fields widget is set to text and it's format is 'Y-m-d H:i:s'
I have also tried date('Y-m-d H:i:s', time()) instead of time().
Example dpm() output.

I should also note that as a workaround I can set the field if I modify it in hook_node_presave().  To do this I set $node->field_date['und'][0]['value'] = date('Y-m-d H:i:s', $my_unix_timestamp); I would still like to understand how to do this using hook_form_alter instead.
Comment: In what way is this code broken ?
Comment: both date('Y-m-d', strtotime('31 May 2011')) and time() are always empty.  I'm starting to think I'm using the wrong array for dates.  $form['date_field']['und'][0]['value'] may not be correct? Back to the docs it is.
Comment: When the form is finally submit the field_date types are always empty arrays.
Comment: Did you try to set `$form['field_date'][LANGUAGE_NONE][0]['#default_value']['value']` ?
Comment: Could you not try remove any custom code and submit the form from the ui to check the format saved natively from the field?
Comment: The UI works as it should and saves as to $node_object->date_field['und'][0]['value'] = '2012-03-30 08:17:55';  Going to try hook_node_presave instead of hook_form_alter to make the changes.
Comment: Just for reference, when using hook_node_presave() to modify the $node, I can modify the field using $node->field_date['und'][0]['value'].  I have yet to find a hook_form_alter solution to converting a timestamp into a format that can be saved.
Answer: Here is a code that treat site timezone right for me:
 $node->field_date[LANGUAGE_NONE][0] = array(
    'value' => format_date(strtotime('now'), 'custom', 'Y-m-d H:i:s', 'UTC'),
    'timezone' => 'UTC',
    'timezone_db' => 'UTC',
  );

Comment: 5 years later this is still a good answer. Wouldn't it make sense though to use `'timezone' => variable_get('date_default_timezone', 'UTC'),` ?
Answer: Each date field-type is expecting a differently formatted timestamp (based on how it's stored in the DataBase):
// For datetime
$node->field_datetest[$node->language][0]['value'] = "2011-05-25 10:35:58";

// For date
$node->field_datetest[$node->language][0]['value'] = "2011-05-25T10:35:58";

// For datestamp
$node->field_datetest[$node->language][0]['value'] = strtotime("2011-05-25 10:35:58");

Note that you don't need to specify a complete date; for datetime and date you can just pad with zeros, e.g. "2011-05-00 00:00:00" (datetime), "2011-00-00T00:00:00" (date), etc. For datestamp you could just do e.g. strtotime("2011-05-25").
Important: Also note that while the exact value you specify will be stored in the database, the actual time displayed on the site might be different depending on timezone settings. When you create a new datetime/date/datestamp field, you get to choose between five different timezone handling methods. The default one is "site's time zone":
When entering data into the field, the data entered is assumed to be in the site's time zone. When the data is saved to the database, it is converted to UTC.
  However, if you set a date field programmatically like in the above example then no conversion takes place, so make sure you account for the field's timezone settings. Or in other words, if you use "site's time zone", make sure the time is in UTC.

This information was referenced from the section labeled Date field (datetime, date, datestamp) on fooninja's blog.
This is also a great general resource for programmatic node creation in Drupal 7.
Comment: You need quotes around the value keys
Comment: Thanks masterchief! Updated to reflect your improvement.
Answer: This worked for me.
$node->field_date[LANGUAGE_NONE][0] = array(
      'value' => date('Y-m-d H:i:s', strtotime('now')),
      'timezone' => 'UTC',
      'timezone_db' => 'UTC',
);

Answer: The time() value will not work at this level.
You have to use something like :
$mydate = date('Y-m-d', strtotime('31 May 2011'));

Comment: This works for me at hook_node_presave, but not in hook_form_alter.
Comment: Did you try to set `$form['field_date'][LANGUAGE_NONE][0]['#default_value']['value']` as mentioned above ?
Comment: Found it node presave at $node->field_date[LANGUAGE_NONE][0]
Comment: Not working with hook_form_alter... any help?
Answer: For Date stuff we could write a js for getting the current date...
jQuery(document).ready(function() {

    var fullDate = new Date()
    var HR_fullDate = fullDate.toDateString().toString().substring(4,15);
    jQuery('strong.date').contents().replaceWith(HR_fullDate);

});

Answer: If you're using field type Date (ISO format) then use:
$node->field_datetest[$node->language][0][value] = "2011-05-25T10:35";

Answer: Try this:
$now = date_now('UTC');
$now = date_format($now, "Y-m-d H:i:s");

Answer: For those dealing with $form_state and popup widget:
// The date format in this sample is dd/mm/yyyy
$form_state['values']['field_my_date_field']['und'][0]['value']['date'] = '14/05/2014';

Answer: To set a value to a date field follow below codes.
Using hook_form_alter

$form["field_date"]["und"][0]['#default_value']['value'] = $new_date;

Using hook_node_submit or hook_node_presave (Saving the value to the database)
 function MY_MODULE_node_presave($node)
  {
      $node->field_date['und'][0]['value'] =  '2014-09-23 00:00:00';
  }

Or if you need to save value of one custom field(defined in hook_form_alter) to another, see below codes.
function MY_MODULE_node_submit($node, $form, &$form_state) {
  if($node->type == 'my_node_type') {
    $test_date =  $form['test_date ']['#value'];
    //Ensure that $test_date format is '2014-09-23 00:00:00'
    $node->field_date['und'][0]['value'] =  $test_date ;
   }
}

Answer: The accepted answer was giving me a PDO Exception. Only a simple time() call worked for a "datestamp" field!
        $entity->field_upload_time["und"][0]["value"] = time();

Comment: This was the only thing that worked for me as well.
Answer: used time() and worked OK....
$fc_item_wrapper->field_data_pedido->set(time());


How to display multi page TIFF with PyQt

Question: I need to display a multi page tiff file with PyQt
from the search in the internet they say that Qt doesn't support
multi page TIFF.
My code only shows the first page:
<code>#!/usr/bin/python
# -*- coding: utf-8 -*-
import os
import sys
from PyQt4.QtGui import *

class Dialog(QDialog):
    def __init__(self,  parent = None):
        super(Dialog,  self).__init__(parent)
        self.resize(1280, 1024)
        masterLayout = QHBoxLayout(self)
        mainLayout = QVBoxLayout()
        self.pictureLabel = QLabel()
        mainLayout.addWidget(self.pictureLabel)
        self.status_Label = QLabel('100')
        masterLayout.addLayout(mainLayout)
        self.img_refresh()

    def img_refresh(self):
        imagem = QPixmap('image.tif')
        myScaledPixmap = imagem.scaled(1280,1024)
        self.pictureLabel.setPixmap(myScaledPixmap)
        self.setWindowTitle('test')

def main():
    app = QApplication(sys.argv)
    form = Dialog()
    form.show()
    app.exec_()

if __name__ == '__main__':
    main()
</code>
Comment: Using [pillow](https://pypi.python.org/pypi/Pillow) is also an alternative.
Answer: You can use the tifffile package which also supports multipage tiffs. They can be converted to numpy arrays which can be converted to QImages with qimage2ndarray and these can be converted to QPixmaps.
Information about the tifffile package:
https://pypi.python.org/pypi/tifffile
Documentation of the qimage2ndarray package:
http://hmeine.github.io/qimage2ndarray/
Answer: You can use MultiPageTIFFViewerPyQt https://github.com/marcel-goldschen-ohm/MultiPageTIFFViewerPyQt/tree/master which is pretty well written.


Sublime 3 JS Snippets to Typescript

Question: I have a lot of useful snippets on JS.
Like cl for console.log(); or
fn for
function methodName (arguments) {
    // body...
}

However I can't use them on *.ts files.
How can manage the snippets and complation for js to ts also.
Thanks
Answer: You have to add a typescript source to your snippets scope.
<scope>source.js, source.ts</scope>

There is a exemple of snippet that you can find in the TypeScript package:
<snippet>
    <content><![CDATA[
class ${2:ClassName}${3: extends ${4:AnotherClass}} {
    $5
    ${6:constructor}(${7:argument}) {
        ${0:// code...}
    }
}
]]></content>
    <tabTrigger>class</tabTrigger>
    <scope>source.ts</scope>
    <description>class …</description>
</snippet>

Comment: thnx for reply, where can i find these files. `*.sublime-package` files are binary.
Comment: @engincancan Just FYI the *.sublime-package are actually zip files. You can use 7zip or any other tools to view and unzip them

Sublime 3 JS Snippets to Typescript


Following Drupal 8 Commerce custom offer tutorial, but getting "name already in use" error

Question: I am following the tutorial for creating a custom offer at the official Drupal 8 Commerce documentation.  But I'm getting an error that I pasted below when I click on any of the conditions, and none of the forms are appearing for any of the commerce built-in offers.  I followed the tutorial exactly, except for extending the OrderPromotionOfferBase class instead of the OrderFixedAmountOff class.  I pasted the contents of the OrderItemFixedAmountTarget class I defined for the offer below the error.

Error
AjaxError:
An AJAX HTTP error occurred.
HTTP Result Code: 200
Debugging information follows.
Path: /promotion/1/edit?destination=/admin/commerce/promotions&ajax_form=1
StatusText: OK
ResponseText:
( ! ) Fatal error: Cannot declare class Drupal\my_module\Plugin\Commerce\PromotionOffer\OrderItemFixedAmountTarget, because the name is already in use in /home/usertest/workspace/drupalsite/web/modules/my_module/src/Plugin/Commerce/PromotionOffer/OrderItemFixedAmountTarget.php on line 18
Call Stack
#TimeMemoryFunctionLocation
10.0000438056{main}(  ).../index.php:0
20.0006554920Drupal\Core\DrupalKernel->handle(  ).../index.php:19
30.00441717448Stack\StackedHttpKernel->handle(  ).../DrupalKernel.php:693
40.00441717448Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(  ).../StackedHttpKernel.php:23
50.00441718144Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(  ).../NegotiationMiddleware.php:52
60.00441718144Drupal\page_cache\StackMiddleware\PageCache->handle(  ).../ReverseProxyMiddleware.php:47
70.00441718144Drupal\page_cache\StackMiddleware\PageCache->pass(  ).../PageCache.php:85
80.00441718144Drupal\Core\StackMiddleware\KernelPreHandle->handle(  ).../PageCache.php:106
90.00562139576Drupal\Core\StackMiddleware\Session->handle(  ).../KernelPreHandle.php:47
100.00642260136Symfony\Component\HttpKernel\HttpKernel->handle(  ).../Session.php:57
110.00652260552Symfony\Component\HttpKernel\HttpKernel->handleRaw(  ).../HttpKernel.php:68
120.02877724544Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}(  ).../HttpKernel.php:151
130.02877724544Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(  ).../EarlyRenderingControllerWrapperSubscriber.php:97
140.02877727080Drupal\Core\Render\Renderer->executeInRenderContext(  ).../EarlyRenderingControllerWrapperSubscriber.php:124
150.02877727432Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}(  ).../Renderer.php:582
160.02877727432call_user_func_array:{/home/usertest/workspace/drupalsite/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php:123}
(  ).../EarlyRenderingControllerWrapperSubscriber.php:123
170.02877727840Drupal\Core\Entity\HtmlEntityFormController->getContentResult(  ).../EarlyRenderingControllerWrapperSubscriber.php:123
180.02917849224Drupal\Core\Form\FormBuilder->buildForm(  ).../FormController.php:93
190.043110154816Drupal\Core\Form\FormBuilder->processForm(  ).../FormBuilder.php:319
200.087912157904Drupal\Core\Form\FormBuilder->rebuildForm(  ).../FormBuilder.php:626
210.087912157904Drupal\Core\Form\FormBuilder->retrieveForm(  ).../FormBuilder.php:369
220.087912160160call_user_func_array:{/home/usertest/workspace/drupalsite/web/core/lib/Drupal/Core/Form/FormBuilder.php:519}
(  ).../FormBuilder.php:519
230.087912160568Drupal\commerce_promotion\Form\PromotionForm->buildForm(  ).../FormBuilder.php:519
240.088212170728Drupal\commerce_promotion\Form\PromotionForm->buildForm(  ).../PromotionForm.php:31
250.088312171952Drupal\commerce_promotion\Form\PromotionForm->form(  ).../EntityForm.php:117
260.088312171952Drupal\commerce_promotion\Form\PromotionForm->form(  ).../PromotionForm.php:38
270.088312172728Drupal\Core\Entity\Entity\EntityFormDisplay->buildForm(  ).../ContentEntityForm.php:125
280.089312198408Drupal\commerce\Plugin\Field\FieldWidget\PluginSelectWidget->form(  ).../EntityFormDisplay.php:177
290.089312198408Drupal\commerce\Plugin\Field\FieldWidget\PluginSelectWidget->formMultipleElements(  ).../WidgetBase.php:104
300.089312198784Drupal\commerce\Plugin\Field\FieldWidget\PluginSelectWidget->formSingleElement(  ).../WidgetBase.php:202
310.089312199160Drupal\commerce\Plugin\Field\FieldWidget\PluginSelectWidget->formElement(  ).../WidgetBase.php:335
320.089312201184Drupal\commerce\Plugin\Field\FieldWidget\PluginSelectWidget->supportsConfiguration(  ).../PluginSelectWidget.php:116
330.089312201584is_subclass_of
(  ).../PluginSelectWidget.php:149
340.089312201712spl_autoload_call
(  ).../PluginSelectWidget.php:149
350.089312201840Composer\Autoload\ClassLoader->loadClass(  ).../PluginSelectWidget.php:149
360.089412202032Composer\Autoload\includeFile(  ).../ClassLoader.php:322
370.089412202272include( '/home/usertest/workspace/drupalsite/web/modules/my_module/src/Plugin/Commerce/PromotionOffer/OrderItemFixedAmountTarget.php' ).../ClassLoader.php:444

OrderItemFixedAmountTarget.php

namespace Drupal\my_module\Plugin\Commerce\PromotionOffer;

use Drupal\commerce_promotion\Plugin\Commerce\PromotionOffer\OrderPromotionOfferBase;
use Drupal\commerce_promotion\Plugin\Commerce\PromotionOffer\FixedAmountOffTrait;
use Drupal\commerce_order\Adjustment;
use Drupal\commerce_promotion\Entity\PromotionInterface;
use Drupal\Core\Entity\EntityInterface;

/**
 * @CommercePromotionOffer(
 *   id = "my_module_fixed_amount_target",
 *   label = @Translation("Discount each matching product to fixed amount"),
 *   entity_type = "commerce_order_item",
 * )
 */
class OrderItemFixedAmountTarget extends OrderPromotionOfferBase {

  use FixedAmountOffTrait;

  /**
   * {@inheritdoc}
   */
  public function apply(EntityInterface $entity, PromotionInterface $promotion) {
    $this->assertEntity($entity);
    /** @var \Drupal\commerce_order\Entity\OrderInterface $order */
    $order = $entity;
    $subtotal_price = $order->getSubTotalPrice();
    $amount = $this->getAmount();
    if ($subtotal_price->getCurrencyCode() != $amount->getCurrencyCode()) {
      return;
    }
    // The promotion amount can't be larger than the subtotal, to avoid
    // potentially having a negative order total.
    if ($amount->greaterThan($subtotal_price)) {
      $amount = $subtotal_price;
    }
    // Split the amount between order items.
    $amounts = $this->splitter->split($order, $amount);

    foreach ($order->getItems() as $order_item) {
      if (isset($amounts[$order_item->id()])) {
        $order_item->addAdjustment(new Adjustment([
          'type' => 'promotion',
          // @todo Change to label from UI when added in #2770731.
          'label' => t('Discount'),
          'amount' => $amounts[$order_item->id()]->multiply('-1'),
          'source_id' => $promotion->id(),
        ]));
      }
    }
  }
}

my_module.info.yml:
name: 'Custom Promotion Conditions'
type: module
description: 'Provides Custom Offers and Conditions'
core: 8.x
package: 'Custom'
dependencies:
  - commerce:commerce
  - commerce:commerce_order
  - drupal:options

my_module.module:
<?php

/**
 * @file
 * Contains my_module_custom_promotion.module.
 */

use Drupal\Core\Routing\RouteMatchInterface;

/**
 * Implements hook_help().
 */
function my_module_custom_promotion_help($route_name, RouteMatchInterface $route_match) {
  switch ($route_name) {
    // Main module help for the my_module_custom_promotion module.
    case 'help.page.my_module_custom_promotion':
      $output = '';
      $output .= '
' . t('About') . '
';
      $output .= '
' . t('My Awesome Module') . '
';
      return $output;

    default:
  }
}

/**
 * Implements hook_theme().
 */
function my_module_custom_promotion_theme() {
  return [
    'my_module_custom_promotion' => [
      'render element' => 'children',
    ],
  ];
}

I've tried my best but I cannot figure out what I'm doing wrong and causing this "name already in use" error.  I've searched my entire project just in case, but the text "OrderItemFixedAmountTarget" is only used in this one file.
Comment: Perhaps you have used it on a previous unsuccessful attempt and the system still thinks its there.  Perhaps a workaround would be to move my_module folder temporarily out of the drupal installation, clear cache and and bring it back its place.  I would also try `drush entup` might help.
Comment: The same error happens if I change the name. If I change it to "MyNewModuleOrderItemFixedAmountTarget", it gives the error:
`Fatal error: Cannot declare class Drupal\my_module\Plugin\Commerce\PromotionOffer\MyNewModuleOrderItemFixedAmountTarget, because the name is already in use in...`
Comment: One thing I did notice, is that if I move OrderItemFixedAmountTarget.php to the `promotion/src/Plugin/Commerce/PromotionOffer/` alongside all the default offers, it does give a different error.  I'm not sure, it may be how I am setting up my module that is the problem.  I generated the module using Drupal CLI `./bin/drupal gm` command.
Answer: Giorgosk's answer didn't fix the problem but it did lead me to notice that the name of my .yml and .module files didn't match the short name I had given the module, and that was the reason for the bizarre error messages.  Thanks!


Acesso a base MySql pelo connector/net

Question: Estou desenvolvendo um sistema que irá acessar o banco de dados MySql apenas para consultar dados. Fiz a referência do mysql.data.dll e estou utilizando o acesso através do MySqlConnection.
Acontece que o sistema está retornando o seguinte erro:

Access Denied for user 'meuusuario....'

O interessante é que eu consigo acessar a mesma base com o mesmo usuário através do SqlYog, porém não consigo através do .net.
Seria alguma configuração no usuário dentro do MySql para aceitar o acesso ?
Comment: Disponha a sua classe de conexão ou rotina @Silva
Comment: Obrigado pela resposta @Harry Potter. Sou novo por aqui...Como faço para incluir minha classe sem exceder o tamanho do comentário ?
Comment: Pode editar a sua questão colocar no ponto ideal e apertar no {} do editor
Comment: Esse usuário tem senha?
Comment: Se o banco de dados estiver executando em um ip diferente da sua aplicação verifique se você consegue acesso externo ao seu banco de dados e se o seu usuário tem permissão para acessar a base dados. E verifique o log do MySQL. Existe uma pergunta sobre problemas de acesso ao MySQL em http://pt.stackoverflow.com/questions/17597/acesso-remoto-mysql-servidor-amazon/22964#22964, você pode encontrar procedimentos de como configurar o MySQL para acesso externo (se esse for o caso).
Answer: Você já verificou se o MySQL está aceitando conexões de outros hosts? Existe uma configuração no MySQL que permite que ele seja acessado somente da máquina que está instalado, ou de outros hosts.
É importante verificar na configuração (my.ini ou my.cnf, dependendo se sua instalação é windows ou unix) e ver se na seção [mysqld] as configurações estão corretas.
Alguns parâmetros que podem ser interessantes aqui são: bind-address (endereços que a conexão é permitida) e enable-named-pipe.

Acesso a base MySql pelo connector/net


Sufficient statistic, specifics/intuition problems

Question: I'm teaching myself some statistics for fun and I have some confusion regarding sufficient statistics. I'll write out my confusions in list format:

If a distribution has $n$ parameters then will it have $n$ sufficient statistics?

Is there any sort of direct correspondence between the sufficient statistics and the parameters? Or do the sufficient statistics just serve as a pool of "information" so that we can recreate the setting so we can calculate the same estimates for the parameters of the underlying distribution.

Do all distributions have sufficient statistics? ie. can the factorization theorem ever fail?

Using our sample of data, we assume a distribution that the data is most likely to be from and then can calculate estimates (e.g. the MLE) for the parameters for the distribution. Sufficient statistics are a way to be able to calculate the same estimates for the parameters without having to rely on the data itself, right?

Will all sets of sufficient statistics have a minimal sufficient statistic?

This is the material which I am using to try to understand the topic matter here.
From what I understand we have a factorization theorem which separates the joint distribution into two functions, but I do not understand how we are able to extract the sufficient statistic after factorizing the distribution into our functions.

The Poisson question given in this example had a clear factorization, but then it was stated that the sufficient statistics were the sample mean and the sample sum. How did we know that those were the sufficient statistics just by looking at the form of the first equation?

How is it possible to conduct the same MLE estimates using sufficient statistics if the second equation of the factorization result will sometimes depend on the data values $X_i$ themselves? For instance in the Poisson case the second function depended on the inverse of the product of the factorials of the data, and we would no longer have the data!

Why would the sample size $n$ not be a sufficient statistic, in relation to the Poisson example on the webpage? We would require $n$ to reconstruct certain parts of the first function so why is it not a sufficient statistic as well?
Comment: Just a quick clarifying question - which "angle" are you coming at sufficiency from?  Maximum likelihood? Bayesian? Maximum entropy? Sampling theory? Something else?
Comment: I was coming from a standpoint of MLE, sorry if my posting wasn't the greatest, it's my first post on this forum!
Answer: You'd probably benefit from reading about sufficiency in any textbook on theoretical statistics, where most of these questions will be covered in detail. Briefly ...

Not necessarily. Those are special cases: of distributions where the support (the range of values the data can take) doesn't depend on the unknown parameter(s), only those in the exponential family have a sufficent statistic of the same dimensionality as the number of parameters. So for estimating the shape & scale of a Weibull distribution or the location & scale of a logistic distribution from independent observations, the order statistic (the whole set of observations disregarding their sequence) is minimal sufficient—you can't reduce it further without losing information about the parameters. Where the support does depend on the unknown parameter(s) it varies: for a uniform distribution on $(0,\theta)$, the sample maximum is sufficient for $\theta$; for a uniform distribution on $(\theta-1,\theta+1)$ the sample minimum and maximum are together sufficient.
I don't know what you mean by "direct correspondence"; the alternative you give seems a fair way to describe sufficient statistics.
Yes: trivially the data as a whole are sufficient. (If you hear someone say there's no sufficient statistic they mean there's no low-dimensional one.)
Yes, that's the idea. (What's left—the distribution of the data conditional on the sufficient statistic—can be used for checking the distributional assumption independently of the unknown parameter(s).)
Apparently not, though I gather the counter-examples are not distributions you're likely to want to use in practice. [It'd be nice if anyone could explain this without getting too heavily into measure theory.]

In response to the further questions ...

The first factor, $ \mathrm{e}^{-n\lambda}\cdot\lambda^{\sum{x_i}}$, depends on $\lambda$ only through $\sum x_i$. So any one-to-one function of $\sum x_i$ is sufficient: $\sum x_i$, $\sum x_i/n$, $(\sum x_i)^2$†, & so on.
The second factor, $\tfrac{1}{x_1! x_2! \ldots x_n!}$, doesn't depend on $\lambda$ & so won't affect the value of $\lambda$ at which $f(x;\lambda)$ is a maximum. Derive the MLE & see for yourself.
The sample size $n$ is a known constant rather than a realized value of a random variable‡, so isn't considered part of the sufficient statistic; the same goes for known parameters other than the ones you want to infer things about.

† In this case squaring is one-to-one because $\sum x_i$ is always positive.
‡ When $n$ is a realized value of the random variable $N$, then it will be part of the sufficient statistic, $(\sum x_i,n)$. Say you choose a sample size of 10 or 100 by tossing a coin: $n$ tells you nothing about the value of $\theta$ but does affect how precisely you can estimate it; in this case it's called an ancillary complement to $\sum x_i$ & inference can proceed by conditioning on its realized value—in effect ignoring that it might have come out different.
Comment: I'd love to see the counterexamples to 5. I tried to prove the opposite for a while with Zorn's Lemma but it breaks down at one point. But from what I've gathered the counterexample should be really whacky. Do you have any reference point where I could find it? I don't mind it being heavy on measure theory.
Comment: @sjm.majewski: Lehmann gives Pitcher (1957), "Sets of measures not admitting necessary and sufficient statistics or subfields", *Ann. Math. Statist.*, **28**, p267-268; and Landers & Rogge (1973). "On sufficiency and invariance", *Ann. Statist.*, **1**, p543-544.


Compile Error with ios_base::sync_with_stdio(false);

Question: I wrote this code for Sieve Of Eratosthenes but it is giving compile time error on ios_base::sync_with_stdio(false);.
The code is executing when I remove that line.
#include<bits/stdc++.h>
using namespace std;

ios_base::sync_with_stdio(false);
cin.tie(0);
cout.tie(0);

void sieveOfEratosthenes(int n) {
  bool prime[n + 1];
  memset(prime, true, sizeof(prime));
  for (int p=2; n>=p*p; p++) {
    if (prime[p]) {
      for (int i = p*p; i<=n; i+=p)
        prime[i] = false;
    }
  }

  for(int p=2; p<=n; p++)
    cout <<p << " ";
}

int main() {
  cout << "Hello World!\n";
  int n;
  cout << "Please enter an integer n\n";
  cin >> n;
  sieveOfEratosthenes(n);
}

Comment: You have fallen into the [cargo cult](https://en.wikipedia.org/wiki/Cargo_cult_programming) of so-called "competitive" programming: Just copy from someone else without bothering to understand the code you're copying. You usually don't need that *boiler-plate* code presented by such sites. Also read [Why should I not #include ?](https://stackoverflow.com/questions/31816095/why-should-i-not-include-bits-stdc-h) and [Why is "using namespace std;" considered bad practice?](https://stackoverflow.com/questions/1452721/why-is-using-namespace-std-considered-bad-practice)
Comment: Not to mention that they don't actually each you C++ or good programming habits. Don't use such sites to learn programming or C++. Get [some good books](https://stackoverflow.com/questions/388242/the-definitive-c-book-guide-and-list) and take classes.
Comment: bool prime[n+1] is not valid c++ btw, it's an extension
Comment: Put it in your `main` function.
Comment: Try to move `ios_base::sync_with_stdio(false);` into `main()`
Comment: That `memset` is also highly suspect. It makes implicit assumptions about the size and representation of `bool` objects. Use [`std::fill`](https://en.cppreference.com/w/cpp/algorithm/fill) instead.
Answer: Moving
ios_base::sync_with_stdio(false);
 cin.tie(0);
 cout.tie(0);

into main function worked :)


PDFtron "Value does not fall within the expected range." exception thrown when loading pdf

Question: I get the exception  {"Value does not fall within the expected range."} when I try to add a new pdf document into the view. And in the view, there are two pdf documents to load. One is loaded and the other throws this exception when loading.
Here is the c# code.
                       (this.DataContext as ViewerPageViewModel).Activate();
                       (this.DataContext as ViewerPageViewModel).OpenDoc(pdfDoc);
                       PDFViewCtrlBorder.Child = (this.DataContext as ViewerPageViewModel).PDFViewCtrl;

Here is the XAML code.
                                <Grid
                                x:Name="PaperGrid"
                                Grid.Row="0"
                                Grid.RowSpan="2"
                                Background="{StaticResource WhiteBackgroundBrush}"
                                Visibility="Collapsed">
                                <Border x:Name="PDFViewCtrlBorder" >
                                    <Grid>
                                        <Border Child="{Binding}" />
                                    </Grid>
                                </Border>
                                </Grid>

Answer: This is quite a generic exception when setting an UIElement property with an invalid value.
Could you please share more information regarding the exception and perhaps a sample project so I can better understand it?
Also, if you are trying to open 2 PDFs on the same View, I would make sure to proper use binding to both PDFViewCtrl properties in your ViewModel and not try to set them on Load event or code-behind.
<Border x:Name="PDFViewCtrlBorder" >
     <Grid>
           <Border Child="{Binding PDFViewCtrl}" />
     </Grid>
</Border>

You can also take a look at our UWP samples repository which has a PDFCompare sample project which loads 2 PDFs on the view
https://github.com/PDFTron/pdftron-uwp-samples
You can also contact us on https://www.pdftron.com/company/contact-us/. From there we can provide you with one-on-one support and help you with a solution for your use case.
Kind regards,
Robson


drupal 8.1 blocks config choices different from D8

Question: I did a migration in Nov/2015 of my D6 to D8 and the blocks look like this.

Then I repeated the migration of my D6 to D8 on May/2016 and my blocks
look like this.

They are different.
Where is the documentation for the configuration of the block visibility
Thanks,
Comment: It seems you have more modules installed, in the second screenshot. Are the blocks created in the same way?
Comment: The second screen shots does not have more blocks.  The second migration was created by downloading drupal8.1.0 and then I added all of the modules from my first migration then I did a migration from D6 to D8 and all of a sudden I get all the extra choices in the block.  This morning
Comment: This morning I did a different approach to update my migration from D8.0 to D8.1.  I copied all of files from D8.1 into my migrated D8.0 and after restarting the migrated site running D8.1 I also can see all of these extra choices.  Somehow all of these extra choices show up but I can not find documentation about it or why they are are there?
Answer: This is not a difference between 8.0 and 8.1, it happens in both versions.
It is a core bug, triggered by ctools. See https://www.drupal.org/node/2632434
Comment: Berdir - Thanks the info.  Is good to know that it is not something I did :-)


Azure Application Gateway Blocking Client

Question: My Azure Network connected with Office Datacenter. We are using Application Gateway with Private IP.
We were using Application Gateway WAF Tier Firewall Enabled but detection mode.
Under that situation, some particular user got blocked. Then we disabled Firewall mode and the issue solved.
Later that situation again arises, even firewall mode was disabled. I turn on the firewall mode and disable again, and that resolves the issue.
In order to get rid of that situation, i migrate Application Gateway to Standard Tier. After 3 days later we again faced that issue. Then i again migrate to WAF Tier, Enable Firewall under Detection mode then disable Firewall but keep WAF Tier, and That resolve the issue.
It doesn't block any particular page, it blocks the full domain even a simple HTML page not able to access.
Answer: Do you have proper exceptions set for the ports required by traffic manager?

Network Security Groups (NSGs) are supported on the application gateway subnet with the following restrictions:
Exceptions must be put in for incoming traffic on ports 65503-65534 for the Application Gateway v1 SKU and ports 65200 - 65535 for the v2 SKU. This port-range is required for Azure infrastructure communication.

I would recommend verifying the Firewall guidance from Microsoft here.
Comment: My site is working smoothly, but sometimes i am facing that issue. If it is port related then it must not work anyhow.


Create Png by sending json to php using ajax

Question: I am trying to make an app where users can crop the image then save then can download it, but after cropping when i click on save button the png image created is of 0kb...
what could be the error?
this is my js file
function getIndex(e) {
    var t = c.getObjects();
    for (var n in t) {
        if (t[n] == e) {
            return n
        }
    }
}

function loadingShow(e) {
    $("#overlay_loading").show();
    $("#overlay_loading #load_message").html(e);
    $("body").css("overflow", "hidden")
}

function loadingHide() {
    $("#overlay_loading").hide();
    $("body").css("overflow", "inherit")
}

function getData() {
    var e = $("body").data("dats")
}

function reloadThumbs(e) {
    if (!cropping) {
        var t = $($("#object_layers li").get().reverse());
        if (e) {
            var n = [];
            var r = getIndex(e);
            n[r] = e
        } else {
            var n = c.getObjects();
            var i = "";
            if (n.length !== t.length) {
                $("#object_layers").empty();
                for (var s = 0; s < n.length; s++) {
                    var o = $('<image width="120" height="75" />').attr("src", i);
                    $("
").prependTo("#object_layers").append(o)
                }
                t = $($("#object_layers li").get().reverse())
            }
        }
        for (s in n) {
            if (n[s] !== undefined) {
                var u = fabric.document.createElement("canvas");
                if (!u.getContext && typeof G_vmlCanvasManager != "undefined") {
                    G_vmlCanvasManager.initElement(u)
                }
                u.width = $("#cc").width();
                u.height = $("#cc").height();
                fabric.util.wrapElement(u, "div");
                var a = new fabric.Canvas(u);
                a.backgroundColor = "transparent";
                var f = n[s].isActive();
                a.add(n[s]);
                a.renderAll();
                var i = a.toDataURLWithMultiplier("png", .35101058);
                if (f) {}
                $(t.get(s)).find("img").attr("src", i);
                $("#object_layers li").click(function () {
                    $("#object_layers li").removeClass("layer_selected");
                    $(this).addClass("layer_selected")
                })
            }
        }
        reloadData()
    }
}

function reloadLayers() {
    $($("#object_layers li").get().reverse()).each(function () {
        if ($(this).data("object") !== undefined) c.bringToFront($(this).data("object"))
    });
    reloadData()
}

function reloadData() {
    var e = c.getObjects();
    var t = $($("#object_layers li").get().reverse());
    for (var n in e) {
        if (e[n] !== undefined) {
            $(t.get(n)).data("object", e[n])
        }
    }
    c.selection = false
}

function cropStart() {
    cropping = true;
    c.forEachObject(function (e) {
        e.selectable = false
    });
    var e = actObj;
    var t = {
        left: 450,
        top: 150,
        width: 300,
        height: 200
    };
    console.log(t);
    cropObject = new Crop({
        left: t.left,
        top: t.top,
        fill: "rgba(255,255,255,0)",
        width: t.width,
        height: t.height
    });
    c.add(cropObject);
    c.deactivateAll();
    cropObject.selectable = true;
    c.setActiveObject(cropObject);
    c.bringToFront(cropObject);
    c.renderAll();
    $("#crop_control").show()
}

function crop() {
    var e = cropObject.width * cropObject.scaleX;
    var t = cropObject.height * cropObject.scaleY;
    var n = cropObject.left - e / 2;
    var r = cropObject.top - t / 2;
    var i = actObj;
    i.clone(function (s) {
        var o = fabric.document.createElement("canvas");
        if (!o.getContext && typeof G_vmlCanvasManager != "undefined") {
            G_vmlCanvasManager.initElement(o)
        }
        o.width = e;
        o.height = t;
        fabric.util.wrapElement(o, "div");
        var u = new fabric.Canvas(o);
        u.backgroundColor = "transparent";
        s.setOpacity(1);
        u.add(s);
        s.left -= n;
        s.top -= r;
        u.renderAll();
        var a = u.toDataURL();
        var f = $("").get(0);
        i.scaleX = 1;
        i.scaleY = 1;
        i.setElement(f);
        i.width = e;
        i.height = t;
        i.setAngle(0);
        c.setActiveObject(i);
        c.renderAll();
        setTimeout(function () {
            c.renderAll();
            reloadThumbs(i)
        }, 100);
        reloadThumbs()
    });
    cropFinish()
}

function cropFinish() {
    $("#crop_control").hide();
    $(".ct").removeAttr("disabled");
    $("#object_layers").sortable("enable");
    cropping = false;
    c.remove(cropObject);
    cropObject = null
}
var cropping = false;
var c = new fabric.Canvas("cc");
var start, set = "personal";
c.setOverlayImage("img/foreground-personal.png", c.renderAll.bind(c));
c.backgroundColor = "rgba(59,89,152,1)";
$(function () {
    if (window.File && window.FileReader && window.FileList && window.Blob) {} else {}
    $("#fileupload").fileupload({
        add: function (e, t) {
            if (typeof FileReader !== "undefined") {
                $(t.files).each(function (e) {
                    var t = this;
                    var n = new FileReader;
                    n.onload = function (e) {
                        var t = e.target.result;
                        fabric.Image.fromURL(t, function (e) {
                            if (e.getWidth() > 800) {
                                e.scaleToWidth(800)
                            }
                            if (e.getHeight() > 400) {
                                e.scaleToHeight(400)
                            }
                            c.add(e);
                            e.setActive(true);
                            c.centerObjectH(e).centerObjectV(e);
                            e.setCoords();
                            c.renderAll();
                            var n = $('<image width="120" height="55" />').attr("src", t);
                            $("
").prependTo("#object_layers").append(n);
                            reloadThumbs()
                        })
                    };
                    n.readAsDataURL(t)
                })
            } else {
                t.submit()
            }
        },
        dataType: "json",
        done: function (e, t) {
            $.each(t.result, function (e, t) {
                $("
").text(t.name).appendTo(document.body);
                fabric.Image.fromURL(t.url, function (e) {
                    var t = e.set({
                        left: 110,
                        top: 75
                    }).scale(.7);
                    c.add(t).renderAll()
                })
            })
        }
    });
    $(document).keydown(function (e) {
        if (e.which == 46 && !cropping) {
            var t = c.getActiveObject();
            if (t) {
                c.remove(t);
                reloadThumbs()
            }
        }
    });
    $("#save").click(function () {
        var e = JSON.stringify(c);
        loadingShow("Saving... Please Wait...");
        $.ajax({
            type: "POST",
            url: "ajax.php",
            data: {
                d: e
            }
        }).done(function (e) {
            loadingHide();
            alert("Data Saved!")
        })
    })
});
c.observe("object:modified", function (e) {
    reloadThumbs(e.target)
});
c.observe("object:selected", function (e) {
    var t = e.target;
    var n = getIndex(t);
    if (t.type == "image") {
        $(".ot").attr("disabled", "disabled").val("");
        $(".oi").removeAttr("disabled")
    } else if (t.type == "text") {
        $(".oi").attr("disabled", "disabled");
        $(".ot").removeAttr("disabled");
        var r = t.toObject();
        var i = t.getFill();
        if (start && r.text == "Click here to start") {
            c.clear();
            reloadThumbs();
            start = false
        }
        $("#text_text").val(r.text);
        $("#text_color").val(i.toUpperCase());
        $("#text_font").val(r.fontFamily)
    }
});
c.observe("selection:cleared", function (e) {
    $(".ot").val("").attr("disabled", "disabled");
    $(".oi").val("").attr("disabled", "disabled")
});
var cropObject = null;
var actObj;
$("#image_crop").click(function () {
    $(this).attr("disabled", "disabled");
    $(".ct").attr("disabled", "disabled");
    $("#object_layers").sortable("disable");
    actObj = c.getActiveObject();
    cropStart();
    return false
});
$("#crop_ok").click(function () {
    crop();
    c.forEachObject(function (e) {
        e.selectable = true
    })
});
$("#crop_cancel").click(function () {
    cropping = false;
    cropFinish();
    c.setActiveObject(actObj);
    c.forEachObject(function (e) {
        e.selectable = true
    });
    return false
})

this is the save code in js using ajax
$("#save").click(function () {
            var e = JSON.stringify(c);
            loadingShow("Saving... Please Wait...");
            $.ajax({
                type: "POST",
                url: "ajax.php",
                data: {
                    d: e
                }
            }).done(function (e) {
                loadingHide();
                alert("Data Saved!")
            })
        })
    });

this my  php file
echo $_POST['e'];
echo json_decode($_POST['e']);
echo var_dump(json_decode($_POST['e']));
$name=time();
file_put_contents("uploads/" . $name . ".png", $data);

Comment: Did you research this at all, or did you find a problem with your code and bring it here expecting us to fix it for you and send you on your happy merry little way? Please present what you tried so we can elaborate as to why it might not be working.
Comment: What function is called when you click save?...What is your HTML like...? How can we help if we don't even know what is being called on save?
Comment: i have tried to crop the image in the canvas and to save it.. but the image generated when saved in server is 0kb
Comment: Where is $_POST['e'] being set?
Comment: $_POST['e'] is set in ajax.php
Comment: Lol, dude. Can you add all of the relevant code so we can determine what's wrong?
Comment: i added all the codes
Answer: Your posted data has a key d but you are trying to read from e which doesn't have a value.
Even if that wasn't the case, then I'm pretty sure that no valid JSON data will decode to be a PNG binary anyway.
Comment: when i read from 'd' the image is not getting generated. its is keep on loading like "saving please wait"
Comment: then what will be valid json data?


Shortest Confidence Interval of Unimodal Distribution

Question: Currently I am taking a statistical course. The lecturer proposed a proposition about the shortest confidence interval of unimodal distribution:
assume distribution $f$ is unimodal, if $\int_{a^*}^{b^*}f(y) dy = \alpha_0$ in $[a^*, b^*]$ and $f(a^*) = f(b^*)$. Then $\forall [a, b]$ with  $\int_{a}^{b}f(y) dy = \alpha_0$ we have $b^* -a^* \leq b-a$.
The lecturer says that the proof is very easy, although I dont have any idea about it.
I think it is intuitive for the symmetrical unimodal case, but I dont know how to generalize it to all the unimodal distribution.
Could anyone help me? Thanks in advance!
P.S. I dont have background of measure theory.
Answer: Suppose the pair $(a, b)$, with $b > a$, satisfy the equation (*)
$$ F(b) - F(a) = \alpha_0 \tag{*} $$
where $F$ is the CDF. Note that $a, b$ are implicitly related due to this constraint. We want to find out which pair of $(a, b)$ will minimize the distance $b - a$
Differentiate both sides with respect to $a$, we have
$$ f(b) \frac {\partial b} {\partial a} - f(a) = 0 \Rightarrow
 \frac {\partial b} {\partial a} = \frac {f(a)} {f(b)}$$
Therefore,
$$ \frac {\partial} {\partial a} (b - a) = \frac {f(a)} {f(b)} - 1
\begin{cases} < 0 & \text{when}& f(a) < f(b) \\
= 0 & \text{when}& f(a) = f(b) \\
> 0 & \text{when}& f(a) > f(b) \\
\end{cases} \tag{**}$$
Denote $m$ be the mode. Then we have $f(y)$ is strictly increasing when $y < m$ and strictly decreasing when $y > m$. That is, when $a < b < m$, we have $f(a) < f(b) < f(m)$ and when $m < a < b$, we have $f(m) > f(a) > f(b)$. So if $f(a^*) = f(b^*)$, we have $a^* < m < b^*$.
On the other hand, as $(a^*, b^*)$ also satisfy $(*)$, so for any pair of $(a, b)$ satisfying $(*)$, we have
$$ F(b) - F(a) = \alpha_0 = F(b^*) - F(a^*) \Rightarrow F(b^*) - F(b) = F(a^*) - F(a) $$
Therefore $F(b^*) - F(b)$ and $F(a^*) - F(a)$ must of the same sign. As the CDF $F$ is monotonic, we have $\{a < a^* \text{ and } b < b^*\}$, or $\{a > a^* \text{ and } b > b^*\}$
So if $a < a^*$, we have $b < b^*$, i.e. $b < m$ or $m \leq b < b^*$. If $m \leq b < b^*$, we have
$$f(b) > f(b^*) = f(a^*) > f(a) $$
where the inequality is due to $f$ is unimodal. And if $m > b > a$, we have $f(b) > f(a)$ as $f$ is unimodal. So in any case, $f(b) > f(a)$ when $a < a^*$. Similarly, we can show that $f(a) > f(b)$ when $a > a^*$. So the equation $(**)$ can be translated as
$$ \frac {\partial} {\partial a} (b - a) = \frac {f(a)} {f(b)} - 1
\begin{cases} < 0 & \text{when}& a < a^* \\
= 0 & \text{when}& a = a^* \\
> 0 & \text{when}& a > a^* \\
\end{cases} \tag{**}$$
which concludes that $(a^*, b^*)$ is the unique minimizer of the distance $b - a$


How to create set of images (~1000) from single image , by creating different variations?

Question: I am new to Image Processing. I have certain set of unique images. I need to create different variation of same image so that I can use that set of variations to train neural network. I need to do this as I dont have enough number of images for each category. Does any one have any clue how can I start ?
I have some ideas like, image dimensions, image angle can be changed to create variation but I am not sure if that will really help neural network.
Thanks
Answer: You can try Image Augmentation Technique which involves Flipping, Rotating and Scaling of an image.
If you are using Keras, You can use ImageDataGenerator.
<code>keras.preprocessing.image.ImageDataGenerator
</code>


Diophantine equation with power

Question: Find all the integer solutions of:
$2^{n+1} + 41 = m^2$
I am stuck, and I am not sure if I am going on the right path..
adding 8 to both sides:
$2^{n+1} + 41 + 8 = m^2 + 8$
$2^{n+1} + 49 = m^2 + 8$
$2^{n+1} - 8 = m^2 -49$
$2(2^n - 4) = (m+7)(m-7)$
I am not sure on how to keep going, how do I manage to solve this question?
Thank you!
Comment: You should note that there are a couple of small solutions, so there's no point looking for some simple congruence contradiction.
Comment: Looking at the equation $\bmod 8$ provides one solution. $m$ is plainly odd, and all odd squares are $\equiv 1 \bmod 8$. That means $2^{n+1}\ge 8$, because $41\equiv 1 \bmod 8$. One solution is then seen to be $m=7$. I haven't pursued whether there are other solutions.
Comment: I still don't see how to answer this question I got stuck at the middle
Comment: Another couple of small solutions (m,n): (-13,6) and (13, 6).
Comment: @AugustoSanti yes, but how do you prove it, I can also write a program that does this..
Comment: You should notice that $\,n\,$ is a solution if and only if $\,n \equiv 2 \mod 4$.
Comment: @lulu I tried searching for answers to the equation not to find congruence contradiction, can you please help me solve this? I am lost.. Thank you!!
Comment: The relevant reference appears to be a paper by Apery that I have not found online.  The reference is R. Apéry, Sur une équation diophantienne, C. R. Acad. Sci. Paris Sér. A 251 (1960), 1263-1264 and 1451-1452.
Answer: Some insight can be gained by looking at the equation $\bmod 40$. Note that for $n\ge 1$, the residues of $2^{n+1} \bmod 40$ are $\{4,8,16,32,24\}$ with $4$ occurring only once and the other values cycling. Since $4$ does not yield a solution ($4+41\ne m^2$), we can ignore its single occurrence. Of course,  $41\equiv 1 \bmod 40$, so we are looking for solutions to $m^2\equiv \{9,17,33,25\} \bmod 40$. No integer squared ends in the digits $3,7$, so we can further eliminate $17,33$ from consideration, leaving us to examine $m^2\equiv \{9,25\} \bmod 40$. From this, we conclude that $m$ ends in one of the digits $\{3,7,5\}$.
Case 1: $m^2\equiv 25 \bmod 40$. Then $2^{n+1}$ must end in the digits $84$. This occurs for $n+1=20k+14$. $n+1=14,34$ do not yield solutions, and for $n+1=54,74,\dots$ I am not able to calculate, so there might be solutions, but I can't say definitely yes or no.
Case 2: $m^2\equiv 9 \bmod 40$. This implies that $2^{n+1}$ ends in the digit $8$, which occurs when $n+1=4k+3$. The readily found solutions for $n+1=3,7$ are $m=7,13$. Up to $n+1=47$, no further solutions are found.
Added by edit: OP correctly points out in a comment that $n$ must be even. If $n$ is odd, then $n+1$ is even, $2^{n+1}$ is a square, and by rearranging terms, $41$ can be expressed as the difference between two squares. Since $41$ is prime, it can only be expressed as the difference between two squares in one way: $41^2-40^2=(21+20)(21-20)$. In this complexion, $m=21$, but $20$ is not a power of $2$, so $n$ cannot be odd, hence $n+1$ cannot be even, which is required in Case 1 where $n+1=20k+14$. Thus there are no Case 1 solutions, as OP states.
Comment: but n is always even so it is not possible that m^2 = 25 mod 40 that case 1 is not possible
Comment: So if only case 2 is valid here, then you solved the whole question?  what about where m = 10K + 7 or m= 10K + 3 ??
Comment: @Remember1312 I pondered looking at those cases, which in essence ask what values $K$ might take, but I made no progress. If there are solutions of those forms however, then there will be values of the exponent $n+1$ that correspond to those solutions, and what I found was that up to $47$ (as high as I could conveniently calculate), no such exponents exits. FWIW, there are many fewer cases to look at when focusing on acceptable values of $n+1$ than on acceptable values of $K$.

Diophantine equation with power


Using variables on multiple PHP pages for mySQL

Question: I have created a member login page and now I am working on a restricted 'Member's Only Section'. I am fairly new to mySQL and I am also practically trying to teach myself. My question is related to authorizing someone who has just logged in, to allow them to go to that restricted section. Vice versa, if they are not logged in, they should not be able to access it or find an error. Below is the code that I have from my login page and also the code for the restricted section...
echo "Welcome."; //Successful
echo "
";
echo " Click here  to continue to the Member Page."; // creates a link to go to.
$sql = " INSERT INTO Login (loginName,loginTime)
VALUES ('$username', NOW() ) "; // creates the login time.
$result = $mysqli->query($sql) or die ($mysqli->error); // shoots an error if i did something wrong.
$_SESSION['logname'] = $userlogin;
$_SESSION['auth']="yes";

Below is the code for the information section:
if ($_SESSION['auth'] != "yes")
{
    header("Location: membership.php");
    exit();
}

$mysqli = new mysqli(HOST, USER, PASSWORD, DATABASE)
or die("Failed to connect");
$sql = "SELECT firstName,lastName FROM Member
WHERE loginName='{$_SESSION['logname']}' ";
$result = $mysqli->query($sql) or die($mysqli->error);

My main issue is that I can access this page whether I am logged in or not... is the variable $_SESSION['auth'] not a global variable?
Comment: Do you have `session_start()` at the beginning of both scripts?
Comment: It looks like you're using "smart quotes" in your scripts. I don't think PHP allows that, you have to use ASCII single quotes.
Comment: Yes, I do. I placed it above my variables for $mysqli = new mysqli(HOST, USER, PASSWORD, DATABASE), so I did not want to add that in my code. But it is there.
Comment: What does `var_dump($_SESSION)` show when someone goes to the second page when not logged in?
Comment: How is $username generated, may be better to prepared statements.
Comment: I'll try var_dump at the end of the second code. The $username is generated when a person places their data into the form. The first section of the code works. I am having trouble using the $_SESSION['auth'] variable.
Comment: array(0) { } is the result of var_dump($_SESSION)
Comment: I'd like to note that you should probably use [PDO](http://wiki.hashphp.org/PDO_Tutorial_for_MySQL_Developers), as your current code allows for SQL injection. Using `mysqli_real_escape` is also a solution, but I'd recommend learning PDO instead.
Answer: you are using smart quotes, the result will be unexpected, and not 'yes' :
Following example:
session_start();
$_SESSION['auth']="yes";
echo "var=".$_SESSION['auth'];

prints var=â€yesâ€
Replace the single and double smart quotes with dump quotes
$_SESSION['logname'] = $userlogin;
$_SESSION['auth']="yes";

and also when you check:
if ($_SESSION['auth'] != "yes")

Comment: So, just change $_SESSION['auth']="yes"; to $_SESSION['auth']='yes';
Comment: you have to be more specific than that. add echos to see the variable values


How is counterpoint different from harmony?

Question: Harmony is a kind of second sound you hear. Counterpoint is also the second sound you hear.
Aside from technical differentiation, how can you by ear differentiate these two concepts. Is it possible to judge something as harmony by ear, and determine that it was a counterpoint by theory?
Comment: Harmony is not a "second sound you hear". It is the context created by the combination of all the sounds you hear. Even music with a single melodic line---e.g. unaccompanied violin or solo singer---has harmony despite its not even having a second sound. Do not confuse "harmony" with "accompaniment".
Comment: Well, in general, you're correct, but there is a specialized use of the term: "singing harmony" means singing a secondary voice in harmony with the main voice.
Answer: The basis of counterpoint (point against point) is melody.  Harmony is evident in counterpoint which, I suppose, is what is causing the confusion.  A theory professor once told me that Harmony is a byproduct of the rules of counterpoint being used properly.  Counterpoint changed from renaissance to baroque in some significant ways.  Renaissance counterpoint is often referred to as melodic counterpoint while Baroque is referred to as Harmonic counterpoint.
Composers who used counterpoint (renaissance period composers) did not think about the harmony in terms of chords, our modern sense of harmony.  The melodic lines that were created using these counterpoint rules resulted in something that we can perceive as chords.  However their focus was solely on melodies and the way they interacted. Baroque counterpoint is structurally based around chords but the focus is still on melodic interaction.
One trick that is very commonly used in counterpoint is imitative counterpoint. In this trick a melody is introduced in one voice.  After the melody is completed, the same melody (often beginning at the interval of a fifth above the original melody) is introduced in a different voice while the first voice moves on to a secondary or counter-melody.
Listen for these types of thematic devices.  The melodies will be very pronounced.  Instead of hearing chords being sustained under a melodic line you will hear multiple melodies in multiple voices that come together to create a unique kind of harmony.
It is a very subtle composition technique.  I hope this helps.
Comment: I liked your answer a lot. Can you please point some good reading material about composing counterpoint in the renaissance form (ie, based on melody rather than harmony)?
Comment: @AlejandroGarcíaIglesias The standard text for 16th century counterpoint is [Johann Fux's Study of Counterpoint](http://www.amazon.com/Study-Counterpoint-Johann-Joseph-Parnassum/dp/0393002772/ref=sr_1_1?ie=UTF8&qid=1338948412&sr=8-1)
Comment: I think the "trick" you describe is simply canon, no?  And that can be any interval of course, even inverse ("upside down" if you like).
Comment: I'm no specialist, but it seems to me the chords were there, especially in popular songs. The art of counterpoint is to play around the chords so they don't need to be made explicit, but they're there implicitly, and I'll bet that simple folk musicians, when playing the same songs, would be strumming them.
Answer: I think the difference you would hear would be the difference in the direction and rhythm of the lines.  Counterpoint would fill in the melodic "gaps" rhythmically and harmonically.  Basic harmony often lines up with the melody.  Counterpoint frequently goes opposite the melody, thus its name.
Listen to some Baroque music, where counterpoint was used heavily (Bach is great for this).  Later periods of music still used it, but didn't rely on it so much as in the Baroque.
Comment: Counterpoint is one of the reasons I have always enjoyed playing Bach.  The 2nd part is not secondary and just harmony with the 1st part.  It stands on its own and often crosses the 1st part, making is fun to play.
Answer: Harmony refers only to the relative pitches of the different voices you hear sounding together in a nice way; it says nothing about the timing of the notes in the different voices.
Counterpoint refers to different voices forming their own separate melodies: (many of) their notes are produced at different times and with different durations.  Counterpoint will (in all cases that I've heard) create harmony, but a lot of music with harmony only has a single voice with a melody, in which case there is no counterpoint.  Polyphony is the extreme form of counterpoint where all voices sing separate melodies.  In extreme cases, the melodies are actually completely different songs (e.g. In feuers hitz and O rosa bella, from the Glogauer songbook), but even then, they are usually combined in such a way that the result is in harmony.
Of course, there are all kinds of mixtures: for instance, a canon, in which all voices have the same melody, only shifted in time.
Answer: This question sounds hard, but it's actually very easy to answer :-)
Harmony is vertical based and counterpoint is horizontal based.
In the old days, when mr. Bach was doing his thing, harmony as we know it didn't really exsists in the same way. Ofcourse, multiple-sounding-notes are creating a harmony. But it wasn't a harmony like we hear now in pop and jazz (etc...)
So: if you have a jazz tune in C and the melody goes: E-F-G and the bass player plays C-D-E then there is a big change the piano/guitar player plays: Cmaj7, Dmin7, Emin7.
This is vertical.
If Bach wrote: E-F-G and wrote in the bass C-D-E he didn't mean Cmaj7, Dmin7, Emin7 he just meant them to go together.
This is horizontal
Then there is something else, counterpoint is a set of rules most of them influenced by church. Google for consonant, dissonant etc. Nowadays you can do whatever you want (thanks to enlightenment! Jeej!) but back in the days, you couldn't.
So, why studie counterpoint and harmony?
Well, harmony is obvious; music of today works on 'harmony'
Counterpoint is nice to studie so you can learn to work in a framework. Maybe you're even able to create your own! Like the 12 tone system Schoenberg created (http://en.wikipedia.org/wiki/Twelve-tone_technique)
* - if you do this, please mention it at music.stackexchange ;-)
Then I would like to end with two youtube clips.
1) It's a fuge (a fuge is a way of composing, also rules. But used in the language of counterpoint)
https://www.youtube.com/watch?v=95gLT7NzHAM
2) Enjoy the lines played by Glenn Gould, you can hear them move together!
https://www.youtube.com/watch?v=N2YMSt3yfko
Comment: I have to take issue with the implication that Bach wasn't aware of or intentional about his use of harmony, as "he just meant them to go together" implies. He meant them to go together because he deeply understood the harmonic consequences of superimposing those lines. His use of harmony was no accident; rather it was the product of someone who completely understood what he was doing.
Comment: I think @AlexBasson is correct.  In fact, the Cmaj7, Dmin7, Emin7 is not more vertical than the Bach example either.  It's just written that way because that's an easy way to communicate to the player (without writing out the complete voicing).  But in both cases the middle chord is just a passing chord and doesn't really have a harmonic function (so you could call it "horizontal").  There isn't really much difference between (classical) jazz and Bach except that the rules are less strict.  But it's functional harmony in both cases.
Comment: Also, harmony isn't just vertical, it's horizontal as well: a certain chord, used in certain situation, "leads" to another.
Answer: The voices who are bored are harmonizing. If everyone in the group is interested in their line, it's counterpoint.
Comment: Don't blame the performers, blame the arranger ...
Answer: Think of counterpoint as a group of independent melodies that overlap and happen simultaneously, yet are designed to create a consonant effect.
Think of harmony as what happens when you strum a series of chords in a progression on the guitar.
Most music nowadays has one melody and chords underneath it to support it. But in earlier times counterpoint was more prevalent. Counterpoint tends to be a great deal more intricate and difficult to compose effectively, by the way.
Answer: Counterpoint is the simultaneous combination of relatively independent musical lines. Hey, I think I just remembered the definition word-for-word from our AP Theory book! Haha. Well, lines of harmony usually go with the melody in the same rhythm, and somewhat the same direction. Contrapuntal lines are almost completely different, but sound good together.
Answer: Short answer:
In counterpoint, the harmony is created through melodies played simultaneously by different voices.
Long answer:
In a modern song, you'll probably find a single melody line played over a guitar strumming the appropriate chords for the harmony. When this is the case, the harmony and melody are broken into two separate functions. One person is responsible for the melody, one person is responsible for the harmony. Counterpoint, especially (bach) fugues, have the potential for incredibly complicated harmonic structure. These pieces typically have 3-4 voices (even 6 in one example). The purpose of each voice is to play a melody, something that can stand alone and still sound relatively interesting. But these voices must be separated by appropriate intervals and subsequent voices must be delayed the proper amount in order that the separate voices, upon singing their own melodic lines, generate a cohesive harmonic progression.
My advice is to find a keyboard with an organ setting (or something with a long sustain) and try playing through fugues at a very very slow tempo. By sustaining certain notes for 1 beat too long, it will immediately show the interaction between certain notes. And the extreme dissonance that arises from sustaining one note for an extra beat is very demonstrative of how the separate voices interact. And I only mention fugues because they are the most complicated example of counterpoint, and thus the structure is more constrained and easier to break apart in my opinion, even if it requires a more demanding approach. But it's worth it!
Answer: There are two types of harmony: vertical and linear. The linear form would "imply" harmony depending on which tones are used in which sequence. The vertical harmony (chords) would be a more direct form, because more notes are used, therefore more context. But even this form of harmony does have a linear component, because harmony is not just the information that is contained in single chord - it also contains the information that is included in tones/chords that come before and after.
Counterpoint, because it is two or more tones played simultaneously, is a subset of vertical harmony.
My personal opinion is that modern concepts of Western harmony are more than adequate for explaining what's going on with traditional counterpoint as it occurred during the Baroque and Classical eras.
Answer: R. Descartes: "People would get rid of half of their problems if they could agree on the meaning of words."
It seems to me that the infinite discussion about counterpoint and harmony is basically such a case. The matter with counterpoint is clear: it is method of composition, which evolved from chorus singing when such music instruments as organ and clavichord appeared. The notion harmony is very old and had in its history many means. If we ask Wikpedia about harmony in music we can read: "The study of harmony involves chords and their construction and chord progressions and the principles of connection that govern them." That is essence of the notorious "harmony" is method of chord progression and it were useful to forget about  "harmony" and compare methods of counterpoint and chord progressions.  The former one lost the competition because it is difficult to use as it was here said by Wheat Williams. The both methods permit to obtain desired  timbre and smoothness of music, that is, make it harmonic in common sense of the word.
Answer: Harmony is always there whether it is single line music or multiline music. The difference is just the way it is presented. Harmony could be effectively used in all types of music. It is a fact that music is created basically with a melody. Of course, the melody has a sweetness. This abstract quality will be fortified  when it is harmonized, and again it will be sweetest when packed with counter parts. Harmony is an ocean over which the vessel of counter part glides with the sailor called melody.
Comment: What do you mean by "the melody has a sweetness"? Also, I don't agree that all music is created with a melody, perhaps you want to elaborate on that?