Need to know when user cancel, not just a shutdown notification

Question:
Possible Duplicate:
Detect browser connection closed in PHP

There are times where I must use something like this:
set_time_limit(0);

// User is uploading some large data
// ...

// Done uploading, from now on processing must be finished, no matter what
ignore_user_abort(true);

// Process the uploaded data...
while (reading the data) {
    // ...
}

The problem is, there are cases where I need to know when user cancel/close the browser window (while I'm still processing the data), so that I can do some db sync/cleanup
I read about register_shutdown_function, but it seems not the way to go, I need a reliable indicator that tells that user is gone, not just a shutdown notification, ie.
// Process the uploaded data...
while (reading the data) {
    if (user_closed_window == true) {
        CleanUp();
        break;
    }
}

Comment: use `if (connection_status()==1){` or `if (connection_aborted()) {`. check [`connection handling`](http://www.php.net/manual/en/features.connection-handling.php)
Answer: That seems kind of impossible to do with PHP. But I think you can do this:
In some way you don't commit your operation, you just do the operation saving it on a buffer for example. Next you check if the user cancelled/closed the window. This can be done with AJAX, you start the operation via XHTTP request and show an other page to the user (with a cancel button perhaps). I believe that you can detect if the user closed the window via JS+AJAX. If your operation has ended and the user didn't closed the window/cancelled the operation, then you commit the changes. If not, you simply discard.
If your operation is huge, you can divide in batch it and check if the user has cancelled/closed between the jobs.
I think that this is a feasible way to do that in PHP.
Hope it helps.


How to list comments of certain node in a PHP

Question: How to list comments of certain node in a PHP ( preferably in XML / RSS format )?
Answer: You can use the Views module to create custom lists of nodes, comments, users etc. In your case, you will need to:

create a view of comments
add a 'Feed' display
add a 'Contextual filter' for 'Comment: Nid' (this will allow you to specify the node ID in the url)
set the feed url to something like comments/%/rss, where the % is a placeholder for the Node ID
save the view

Now you can find an rss feed of comments for node 32 at http://example.com/comments/32/rss
Here is an export of such a view:
<code>$view = new view;
$view->name = 'comments_rss';
$view->description = '';
$view->tag = 'default';
$view->base_table = 'comment';
$view->human_name = 'comments rss';
$view->core = 7;
$view->api_version = '3.0-alpha1';
$view->disabled = FALSE; /* Edit this to true to make a default view disabled initially */

/* Display: Master */
$handler = $view->new_display('default', 'Master', 'default');
$handler->display->display_options['title'] = 'comments rss';
$handler->display->display_options['access']['type'] = 'perm';
$handler->display->display_options['cache']['type'] = 'none';
$handler->display->display_options['query']['type'] = 'views_query';
$handler->display->display_options['query']['options']['query_comment'] = FALSE;
$handler->display->display_options['exposed_form']['type'] = 'basic';
$handler->display->display_options['pager']['type'] = 'full';
$handler->display->display_options['pager']['options']['items_per_page'] = '10';
$handler->display->display_options['style_plugin'] = 'default';
$handler->display->display_options['row_plugin'] = 'comment';
/* Relationship: Comment: Content */
$handler->display->display_options['relationships']['nid']['id'] = 'nid';
$handler->display->display_options['relationships']['nid']['table'] = 'comment';
$handler->display->display_options['relationships']['nid']['field'] = 'nid';
$handler->display->display_options['relationships']['nid']['required'] = 1;
/* Field: Comment: Title */
$handler->display->display_options['fields']['subject']['id'] = 'subject';
$handler->display->display_options['fields']['subject']['table'] = 'comment';
$handler->display->display_options['fields']['subject']['field'] = 'subject';
$handler->display->display_options['fields']['subject']['label'] = '';
$handler->display->display_options['fields']['subject']['alter']['alter_text'] = 0;
$handler->display->display_options['fields']['subject']['alter']['make_link'] = 0;
$handler->display->display_options['fields']['subject']['alter']['absolute'] = 0;
$handler->display->display_options['fields']['subject']['alter']['word_boundary'] = 0;
$handler->display->display_options['fields']['subject']['alter']['ellipsis'] = 0;
$handler->display->display_options['fields']['subject']['alter']['strip_tags'] = 0;
$handler->display->display_options['fields']['subject']['alter']['trim'] = 0;
$handler->display->display_options['fields']['subject']['alter']['html'] = 0;
$handler->display->display_options['fields']['subject']['hide_empty'] = 0;
$handler->display->display_options['fields']['subject']['empty_zero'] = 0;
$handler->display->display_options['fields']['subject']['link_to_comment'] = 1;
/* Sort criterion: Comment: Post date */
$handler->display->display_options['sorts']['created']['id'] = 'created';
$handler->display->display_options['sorts']['created']['table'] = 'comment';
$handler->display->display_options['sorts']['created']['field'] = 'created';
$handler->display->display_options['sorts']['created']['order'] = 'DESC';
/* Contextual filter: Comment: Nid */
$handler->display->display_options['arguments']['nid']['id'] = 'nid';
$handler->display->display_options['arguments']['nid']['table'] = 'comment';
$handler->display->display_options['arguments']['nid']['field'] = 'nid';
$handler->display->display_options['arguments']['nid']['default_action'] = 'not found';
$handler->display->display_options['arguments']['nid']['default_argument_type'] = 'fixed';
$handler->display->display_options['arguments']['nid']['default_argument_skip_url'] = 0;
$handler->display->display_options['arguments']['nid']['summary']['number_of_records'] = '0';
$handler->display->display_options['arguments']['nid']['summary']['format'] = 'default_summary';
$handler->display->display_options['arguments']['nid']['summary_options']['items_per_page'] = '25';
$handler->display->display_options['arguments']['nid']['break_phrase'] = 0;
$handler->display->display_options['arguments']['nid']['not'] = 0;
/* Filter criterion: Comment: Approved */
$handler->display->display_options['filters']['status']['id'] = 'status';
$handler->display->display_options['filters']['status']['table'] = 'comment';
$handler->display->display_options['filters']['status']['field'] = 'status';
$handler->display->display_options['filters']['status']['value'] = 1;
$handler->display->display_options['filters']['status']['group'] = 0;
$handler->display->display_options['filters']['status']['expose']['operator'] = FALSE;
/* Filter criterion: Content: Published */
$handler->display->display_options['filters']['status_node']['id'] = 'status_node';
$handler->display->display_options['filters']['status_node']['table'] = 'node';
$handler->display->display_options['filters']['status_node']['field'] = 'status';
$handler->display->display_options['filters']['status_node']['relationship'] = 'nid';
$handler->display->display_options['filters']['status_node']['value'] = 1;
$handler->display->display_options['filters']['status_node']['group'] = 0;
$handler->display->display_options['filters']['status_node']['expose']['operator'] = FALSE;

/* Display: Page */
$handler = $view->new_display('page', 'Page', 'page');
$handler->display->display_options['path'] = 'comments/%';

/* Display: Feed */
$handler = $view->new_display('feed', 'Feed', 'feed');
$handler->display->display_options['pager']['type'] = 'some';
$handler->display->display_options['style_plugin'] = 'rss';
$handler->display->display_options['row_plugin'] = 'comment_rss';
$handler->display->display_options['path'] = 'comments/%/rss';
$handler->display->display_options['displays'] = array(
  'default' => 'default',
  'page' => 'page',
);
</code>
Comment: Is there a way to have the RSS icon appear somewhere on the node?


A psychology journal banned p-values and confidence intervals; is it indeed wise to stop using them?

Question: On 25 February 2015, the journal Basic and Applied Social Psychology issued an editorial banning $p$-values and confidence intervals from all future papers.
Specifically, they say (formatting and emphasis are mine):

[...] prior to publication, authors will have to remove all
  vestiges of the NHSTP [null hypothesis significance testing procedure] ($p$-values, $t$-values, $F$-values, statements about ''significant'' differences or lack thereof, and so on).
Analogous to how the NHSTP fails to provide the probability of the null hypothesis, which is needed to provide a strong case for rejecting it, confidence intervals do not
  provide a strong case for concluding that the population
  parameter of interest is likely to be within the stated
  interval. Therefore, confidence intervals also are banned
  from BASP.
[...] with respect to Bayesian procedures, we reserve the right to make case-by-case
  judgments, and thus Bayesian procedures are neither
  required nor banned from BASP.
[...] Are any inferential statistical procedures
  required? -- No [...] However, BASP will require strong
  descriptive statistics, including effect sizes.

Let us not discuss problems with and misuse of $p$-values here; there already are plenty of excellent discussions on CV that can be found by browsing the p-value tag. The critique of $p$-values often goes together with an advice to report confidence intervals for parameters of interest. For example, in this very well-argued answer @gung suggests to report effect sizes with confidence intervals around them. But this journal bans confidence intervals as well.
What are the advantages and disadvantages of such an approach to presenting data and experimental results as opposed to the "traditional" approach with $p$-values, confidence intervals, and significant/insignificant dichotomy? The reaction to this ban seems to be mostly negative; so what are the disadvantages then? American Statistical Association has even posted a brief discouraging comment on this ban, saying that "this policy may have its own negative consequences". What could these negative consequences be?
Or as @whuber suggested to put it, should this approach be advocated generally as a paradigm of quantitative research? And if not, why not?
PS. Note that my question is not about the ban itself; it is about the suggested approach. I am not asking about frequentist vs. Bayesian inference either. The Editorial is pretty negative about Bayesian methods too; so it is essentially about using statistics vs. not using statistics at all.

Other discussions: reddit, Gelman.
Comment: There is a one-to-one mapping between p-values and confidence intervals in linear regression models, so I don't see a strong reason why banning p-values but keeping confidence intervals would make much sense. But banning both p-values and confidence intervals leaves a gap in description of results... I wonder if they allow reporting standard errors (that would be another measure of the same one-to-one mapping group).
Comment: Great idea. Using statistics just hides the unscientific nature of this field.
Comment: **Everything** could be misused so banning stuff on this condition is, well... strange. I am not the fan of p-values but this seems as a pretty naive approach to the problem. One thing is encouraging to use proper stuff, but banning things does not sound like a proper way to deal with the problem...
Comment: The 4th item in your list suggests they're *not* requiring point estimates, which would be inference, but effect sizes reported merely as descriptive statistics. (Nevertheless, a few lines down in the editorial, "we encourage the use of larger sample sizes than is typical in much psychology research, because as the sample size increases, descriptive statistics become increasingly stable and sampling error is less of a problem". I look forward to the 2016 editorial's calling for research into formalizing this notion of stability & accounting quantitatively for the effects of sampling error.)
Comment: The American Statistical Association has just posted an official comment on this at http://community.amstat.org/blogs/ronald-wasserstein/2015/02/26/asa-comment-on-a-journals-ban-on-null-hypothesis-statistical-testing.  It ends, "The ASA encourages the editors of this journal ... not [to] discard the proper and appropriate use of statistical inference."
Comment: This seems like a complete overreaction to the frustration over the misuse of p values. I would be much happier with a ban on the misuse of p values rather than P values in general.
Comment: @RichardHardy Confidence intervals provide more information without the input of additional work.
Comment: Is this a wise decision? is the pivot here. For whom? In what way? At one level this is trivially easy: if you work in that field, the decision made public will help you decide whether to submit to that journal. While I am as negative as anyone here about abuses of inferential statistics, my wild guess is that this  decision will, overall, **seriously weaken that journal's reputation**. Banning is a bad idea, as others have emphasised. For most  researchers, the existence of a journal somewhere which is run strangely is of no consequence.
Comment: Reminds me of an English teacher at junior school who banned us from using the word "nice". So it could do some good; but if so that doesn't say much for the quality of research in the field.
Comment: @Scortchi: right, so NHST should by default be a mark against a paper approval (most certainly if it is applied incorrectly), but it shouldn't be banned outright. I mean, Gödel's incompleteness theorem leaves us in a position where *no statistical procedure will ever be valid* unless you accept at least one unprovable axiom somewhere. So banning something completely because it is somewhat flawed is just stupid.
Comment: @Aksakal, I caution you against generalizing the policies of one journal to all of psychology, lest you fall prey to the same errors in logic made by the editors of BASP.
Comment: To add to this question - what possible "strong descriptive statistics" would be available, aside from Bayesian ones?
Comment: It's a great topic.  I'm a little bothered by the phrasing, though, because "wise" seems a little too vague and broad to fit our framework and might not admit a unique or definite answer.  Could you perhaps modify it in a way that will make it clear what kind(s) of answers are required and indicate how to tell good answers from not so good ones?
Comment: @Lubo Many hundreds of descriptive statistics are available, depending on the nature of the data, and thousands of graphical representations.  I think a discussion of those would be tangential to the question rather than adding to it.
Comment: @whuber: Thanks for a suggestion. To tell the truth, I was hesitating to post this question, because I was afraid that it might get frowned upon or even closed as "opinion-based". I have now added a paragraph specifying what sort of answer I am mainly looking for.
Comment: If the question were closer to Would it be a wise decision to ban P-values (etc.) in publications? I think it becomes much more elusive. As another analogy, we have scope to ask people not to smoke in our house (and in any case almost no-one we know smokes any way and the others know without asking that it would be unwelcome) but whether smoking should be banned is a much more tangled question.
Comment: In the end this may well be positive. For starters, it gets some abuses out in the open (rather than the current practice for many journals of decrying problems in editorials and then allowing authors, referees and editors to go on sticking with the same ol' - some journals might hold better to their claimed ideals). Secondly, it's going to get people to actually *explain* what good statistical practice is, and why it's important. I'm glad I'm not trying to publish in that journal, but if it makes some researchers think a little harder about what they're doing, well, that may be a good thing.
Comment: @amoeba: Sorry, but there's clearly a context to all this of which I'm almost completely ignorant; any answer I could offer - along the lines of "What's the use of Statistics?" - would be as relevant as the views of a metrologist from the *Bureau international des poids et mesures* on the *Quebra–Quilos* revolt. (I noticed an advert the other day for a book aimed at psychologists: *Understanding the New Statistics: Effect sizes, Confidence Intervals, & Meta-Analysis*. It was published in 2012.)
Comment: @Scortchi: this was a great remark. By the way, I would encourage you to write a full answer here; this question seems to become quite popular, and I think it would be useful for the community if it had some answers directly discussing possible drawbacks/benefits of such a policy (see my added last paragraph). You are one of the people here who I am sure could present some valuable arguments.
Comment: @Nick, I don't care much about this journal, I was certainly asking about the policy itself. My question is whether it is better or worse to communicate the results of a study according to this suggested policy, as opposed to the traditional way with p-values and confidence intervals. What style of presentation would you, as a reader, prefer, and why? This is the real question. I hoped it would be obvious, but perhaps I should try to make it more clear.
Comment: @Nick, I see. The question is not (at least not so much!) whether *the ban* is a good idea, it is about whether the suggested policy is a good idea. I edited the question to clarify (see the bottom paragraph). I still have to think how to edit the title so that it stays concise and reasonably catchy, while representing what I am really asking. Any suggestions?
Comment: Sorry, no. My personal view is that while this decision raises numerous interesting and important questions, it's hard to see a focused question here suited to the style of this forum. I expect that some statistical blogs will pick this up.
Comment: In light of the edits, I (reluctantly) have to agree with @NickCox.  The question now asks explicitly for *opinions*. "What ... would you ... prefer?" and "would it do good for the scientific field?" are at once vague and speculative.  Given the potential exposure this thread will get, we need to make sure it fits our framework and invites clear answers that readers can evaluate objectively. As evidence of that we have begun to get garbage answers by drive-by (zero-rep) readers, so I have also protected the thread and made it CW pending edits to improve the question.
Comment: Gelmans response: http://andrewgelman.com/2015/02/26/psych-journal-bans-significance-tests-stat-blogger-inundated-with-emails/
Comment: http://www.smithsonianmag.com/science-nature/scientists-replicated-100-psychology-studies-and-fewer-half-got-same-results-180956426/ researchers from uva couldn't reproduce most papers published in top psychology Journals recently. What a surprise. Cargo cult science, what do you expect from it
Comment: @Aksakal: yes, I saw this paper when skimming through yesterday's Science issue. In the context of this thread, however, I am not sure what the lesson should be; it seems to me that not using (banning) p-values is unlikely to make published results more trustworthy.
Comment: @amoeba, this p-value debacle is a symptom of a larger issue in their field. They need to go back to a drawing board, and stop trying to look more science-y. There's a value in fields which are not quantitative, e.g. philosophy or theology
Comment: How long before this nonsense from JAMA Psychology is refuted? [Among men, low resting heart rate in late adolescence was associated with an increased risk for violent criminality, nonviolent criminality, exposure to assault, and unintentional injury in adulthood](http://archpsyc.jamanetwork.com/article.aspx?articleid=2436277) They have confidence intervals, of course.
Comment: Great analysis of the *consequences* of this ban in this very journal: http://daniellakens.blogspot.pt/2016/02/so-you-banned-p-values-hows-that.html. It would constitute an excellent answer here...
Answer: The first sentence of the current 2015 editorial to which the OP links, reads:

The Basic and Applied Social Psychology (BASP) 2014 Editorial
  *emphasized* that the null hypothesis significance testing procedure
  (NHSTP) is invalid...

(my emphasis)
In other words, for the editors it is an already proven scientific fact that "null hypothesis significance testing" is invalid, and the 2014 editorial only emphasized so, while the current 2015 editorial just implements this fact.
The misuse (even maliciously so) of NHSTP is indeed well discussed and documented. And it is not unheard of in human history that "things get banned" because it has been found that after all said and done, they were misused more than put to good use (but shouldn't we statistically test that?). It can be a "second-best" solution, to cut what on average (inferential statistics) has come to losses, rather than gains, and so we predict (inferential statistics) that it will be detrimental also in the future.
But the zeal revealed behind the wording of the above first sentence, makes this look -exactly, as a zealot approach rather than a cool-headed decision to cut the hand that tends to steal rather than offer.  If one reads the one-year older editorial mentioned in the above quote (DOI:10.1080/01973533.2014.865505), one will see that this is only part of a re-hauling of the Journal's policies by a new Editor.
Scrolling down the editorial, they write

...On the contrary, we believe that the p<.05 bar is too easy to pass and
  sometimes serves as an excuse for lower quality research.

So it appears that their conclusion related to their discipline is that null-hypotheses are rejected "too-often", and so alleged findings may acquire spurious statistical significance. This is not the same argument as the "invalid" dictum in the first sentence.
So, to answer to the question, it is obvious that for the editors of the journal, their decision is not only wise but already late in being implemented: they appear to think that they cut out what part of statistics has become harmful, keeping the beneficial parts -they don't seem to believe that there is anything here that needs replacing with something "equivalent".
Epistemologically, this is an instance where scholars of a social science  partially retract back from an attempt to make their discipline more objective in its methods and results by using quantitative methods, because they have arrived at the conclusion (how?) that, in the end, the attempt created "more bad than good". I would say that this is a very important matter, in principle possible to have happened, and one that would require years of work to demonstrate it "beyond reasonable doubt" and really help your discipline. But just one or two editorials and papers published will most probably (inferential statistics) just ignite a civil war.
The final sentence of the 2015 editorial reads:

We hope and anticipate that banning the NHSTP will have the effect of
  increasing the quality of submitted manuscripts by liberating authors
  from the stultified structure of NHSTP thinking thereby eliminating an
  important obstacle to creative thinking. The NHSTP has dominated
  psychology for decades; we hope that by instituting the first NHSTP
  ban, we demonstrate that psychology does not need the crutch of the
  NHSTP, and that other journals follow suit.
Comment: @naught101: If you notice that the workman can't handle the chainsaw properly, you might not blame the tool. But you would still take it away from the workman, to prevent further harm ;-)
Comment: Yes...we have to be careful when writing tongue-in-cheek or sardonic replies on this site: they might be (completely) misunderstood!
Comment: @naught101 ...that wouldn't be very diplomatic. Notice that the way the NHSTP is condemned, it spares the psychologists themselves that they have used it in all these decades. If it was written the way you propose, it would look much more like a direct attack on their colleagues as scientists. As it now stands essentially the text implies that psychologists full of good intentions have been unfortunately misled in using the approach, by "someone", which misused his "power of scientific authority" in the matter... Perhaps by evil statisticians driven by scientific imperialism?
Comment: A bad workman blames his tools.
Comment: @BrianDHall I would suggest to look up more authoritative resources on the issues surrounding NHSTP (this site included), rather than the specific author's works on the issue. The matter is difficult and subtle -already from your comment one should discuss first the semantics around "accept" and "assert"...
Comment: @Anoldmaninthesea. That the way _they_ write the editorial implies that _they_ think that this is so. I added something to clarify.
Comment: Ok. Thanks. For a moment there I got worried... =D
Comment: Perhaps one should not shy away from naming this new editor: David Trafimow, and the paper his 2014 editorial (under paywall) cites to support the claim that "The null hypothesis significance testing procedure has been shown to be logically invalid" (sic!) is his own 2003 paper [that can be found here](http://homepage.psy.utexas.edu/HomePage/class/psy391p/trafimow.nhst.21003.pdf). I am not sure I have the guts to actually read it.
Comment: @amoeba I deleted pat of my draft answer that raised doubts regarding this scholar's level of understanding of statistics (after I did some research and read some of his papers), because I wanted to focus on the editorial text itself. Try this, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3957210/, and also in the 2015 editorial, re-read the part on Bayesian Statistics ("The usual problem...")
Comment: @amoeba Mine was a bit of a meta-answer from the beginning, and naturally should not be considered for the green mark. There is no point in updating it, but if the other answers that I see start coming leave something to be said, I might contribute a second answer.
Comment: Alecos, what do you mean by «it is an already proven scientific fact that "null hypothesis significance testing" is invalid»?
Comment: Maybe that first quote should have read *"...the null hypothesis significance testing procedure (NHSTP) **as it is commonly used in this and many other journals** is invalid..."*.
Comment: @naught101 Now, the text says "the tool is flawed". Under your proposal, the text would argue "the tool may be ok, but the way it was used was wrong".
Comment: @naught101 I am under the impression that your assertion can be statistically tested as the alternative hypothesis -and then the null would be convincingly rejected.
Comment: What whuber said about sardoncism :P
Comment: In trying to figure out if there is merit to what the editor is saying, I looked at the 2003 paper regarding "surprising insights". I am left confused by the first page, as he seems to construct a straw man of NHSTP with stating that if one rejects the null then the alternative is true and accepted; but I was taught that one cannot possibly accept the alternative, merely *assert* it, because it was not ever tested - the null is assumed and tested, and so all you can do is reject it or fail to. This seems fundamentally different...is my understanding deficient here, or is this all a bit 'off'?
Comment: Thank you - after looking some more into Bayesian statistics, I think I get at least a better idea of things, and I can see that the issue is indeed more difficult and subtle a quick shot editorial would imply. Now I think I understand your answer much better, too!
Comment: *the attempt created "more good than bad"* - was this intended to be the reverse, more bad than good? Though I may have misread.
Comment: @Silverfish Thanks for reading my answer _that_ carefully! Fixed it.
Comment: @nikie: And then you would offer them the option to use it again, if they first learn how to use it properly.
Comment: Alecos, as you might have seen, I had to reformulate my question because it got closed as opinion-based. The main question is now about the suggested approach as such: namely, **what are the advantages and disadvantages of presenting data with plots and descriptive statistics *only***, as opposed to using p-values, significance tests, standard errors, and confidence intervals? See also the current formulation of my post. I like your answer (+1), but I will not be able to accept it unless you update it specifically addressing these issues. Would you consider such an update?
Answer: I feel that banning hypothesis tests is a great idea except for a select few "existence" hypotheses, e.g. testing the null hypothesis that there is not extra-sensory perception where all one would need to demonstrate to have evidence that ESP exists is non-randomness.  But I think the journal missed the point that the main driver of poor research in psychology is the use of a threshold on $P$-values.  It has been demonstrated in psychology and most other fields that a good deal of gaming goes on to arrive at $P < 0.05$.  This includes hypothesis substitution, removing of observations, and subsetting data.  It is thresholds that should be banned first.
The banning of confidence intervals is also overboard, but not for the reasons others have stated.  Confidence intervals are useful only if one misinterprets them as Bayesian credible intervals (for suitable non-information priors).  But they are still useful.  The fact that their exact frequentist interpretation leads to nothing but confusion implies that we need to "get out of Dodge" and go Bayesian or likelihood school.  But useful results can be obtained by misinterpreting good old confidence limits.
It is a shame that the editors of the journal misunderstood Bayesian statistics and don't know of the existence of pure likelihood inference.  What they are seeking can be easily provided by Bayesian posterior distributions using slightly skeptical priors.
Comment: I think of standard errors are oversimplified (because they assume symmetric distributions) but useful measures of precision, like mean squared error.  You can think of a precision interval based on root mean squared error without envisioning probability coverage.  So I don't see where any of this discussion implies de-emphasis of standard errors.  And I wasn't suggesting that we stop using CLs.  But the difficulty with CLs comes mainly from attempts at probability interpretations.
Comment: In symmetric situations, the standard error is a building block for a confidence interval.  But in many cases the correct confidence interval is asymmetric so can't be based on a standard error at all.  Some varieties of the bootstrap and back-transforming are two approaches of this type.  Profile likelihood confidence intervals especially come to mind here.
Comment: +1, thanks. Let me clarify regarding confidence intervals. Confidence intervals are related to standard errors, so the suggestion is probably to stop using those as well. Let's consider the simplest case: some value is measured across a group of $n$ subjects/objects; let's say the mean is 3. As far as I understand this journal suggests to report it simply as 3. But would you not want to see standard error as well, e.g. $3 \pm 0.5$? This of course means that 95% confidence interval is $3 \pm 1$, which also means that $p<0.05$, so it's all related. I am not sure how you suggest to report it.
Comment: Hmmm. Interesting. To me it seems like there is such a small step from standard error to CI (a constant factor!), that treating them differently would be weird. But perhaps it is a semantic point; I guess what you mean is that people *think* about standard errors and CIs differently and tend to get more confused about CIs. I wonder what this particular journal policy says about standard errors (the Editorial doesn't mention them explicitly).
Comment: @Frank Harrell - As for "pure likelihood inference" I agree that an emphasis toward summarization of the data's likelihood without embellishing it with thresholds appears to be the answer the editors were grasping for. A. W. F. Edwards' book "Likelihood" (1972) speaks directly to the editor's concern: "We may defer consideration of these arguments (e.g. significance testing) until later chapters, and pass immediately to the description of a procedure, based on Fisher's concept of Likelihood, which is open to none of these objects which may be levelled at significance tests."
Answer: I see this approach as an attempt to address the inability of social psychology to replicate many previously published 'significant findings.'
Its disadvantages are:

that it doesn't address many of the factors leading to spurious effects.  E.g.,

A) People can still peek at their data and stop running their studies when an effect size strikes them as being sufficiently large to be of interest.
B) Large effects sizes will still appear to have large power in retrospective assessments of power.
C) People will still fish for interesting and big effects (testing a bunch of hypotheses in an experiment and then reporting the one that popped up) or
D) pretend that an unexpected weird effect was expected all along.

Shouldn't efforts be made to address these issues first?
As a field going forwards it will make a review of past findings pretty awful.  There is no way to quantitatively assess the believability of different studies.  If every journal implemented this approach, you'll have a bunch of social scientists saying there is evidence for X when it is totally unclear how believable X is and scientists arguing about how to interpret a published effect or arguing about whether it is important or worth talking about.  Isn't this the point of having stats?  To provide a consistent way to assess numbers.  In my opinion, this new approach would cause a mess if it was widely implemented.
This change does not encourage researchers to submit the results of studies with small effect sizes so it doesn't really address the file-drawer effect (or are they going to publish findings with large n's regardless of effect size?).  If we published all results of carefully designed studies, then even though the believability of results of individual studies may be uncertain, meta-analyses and reviews of studies that supplied statistical analysis would do a much better job at identifying the truth.
Comment: @captain_ahab Regarding point 3, we must mention that the previous editorial (2014) of the Editor _explicitly_ encouraged the submission of "null-effect" studies.
Comment: What seems like a better solution is that all scientists must log the hypothesis, basic rational, power of and analytics approach of a study in a PUBLIC place BEFORE running the study.  And then being limited to publishing that study in the prescribe manner.  If an unexpected interesting effect is found, they should publicly log then run a new study that examines that effect.  This approach while controlling for false positives would also enable scientists to demonstrate their productivity without publishing new effects.
Comment: I can't seem to find a comment in the editorial discussing any criteria for publication except for the need have larger sample sizes than normal (how they are planning on identifying acceptable n's without inferential statistics is unclear to me).  To me there is no emphasis in this editorial that they don't care what the effect size is.  It seems to me that they will still be looking for interesting effects and interesting stories, which I think is the bigger problem in social science work (i.e., the post-hoc search for interesting effects and stories).
Answer: I came across a wonderful quote that almost argues for the same point, but not quite -- since it is an opening paragraph in a textbook that is mostly about frequentist statistics and hypothesis testing.

It is widely held by non-statisticians, like the author, that if you do good experiments statistics are not necessary. They are quite right. [...] The snag, of course, is that doing good experiments is difficult. Most people need all the help they can get to prevent them making fools of themselves by claiming that their favourite theory is substantiated by observations that do nothing of the sort. And the main function of that section of statistics that deals with tests of significance is to prevent people making fools of themselves. From this point of view, the function of significance tests is to prevent people publishing experiments, not to encourage them. Ideally, indeed, significance tests should never appear in print, having been used, if at all, in the preliminary stages to detect inadequate experiments, so that the final experiments are so clear that no justification is needed.
-- David Colquhoun, Lectures on biostatistics, 1971
Comment: Thanks for your comment, @whuber; I agree with what you are saying. Still, I must add that I do find it appealing to say that ideally experimental data should be so convincing as to render formal hypothesis tests redundant. This is not an unattainable ideal! In my field (where p-values are used a lot), I find that the best papers *are* convincing without them: e.g. because they present a sequence of several experiments supporting each other, which taken together, *obviously* cannot be a statistical fluke. Re comment: it was too long for a comment, and I figured it's okay as a CW answer.
Comment: Your post is really a comment, rather than an answer, so I am refraining from upvoting it, but I do wish to thank you for sharing the quotation.  There are so many misunderstandings evident in this passage that it would take extensive effort (not to say space) to point out and debunk them all. In one word, though, the counter to these assertions is "efficiency." If everybody had unlimited time and budget we could at least aspire to perform "good experiments." But when resources are limited, it would be foolhardy (as well as costly) to conduct only "final, ... clear" experiments.
Comment: Perhaps one way of reading that is as saying the initial significance test that suggested a substance stimulates a certain physiological response is no longer relevant by the time you're publishing your investigations into the effects of different kinds of inhibitors on the dose-response curve.
Comment: Yes, I understand why it had to be posted as an answer, and therefore did not vote to move it into a comment (which would cut off the last part of the quote). I agree that the ideal is not unattainable *in particular cases*.  I also agree it's a nice ideal to bear in mind.  But as a guide to how to design experiments (which is, overall, a discipline of allocating resources), it could be a terrible mistake. (This is certainly debatable.) The suggestion that a "good" experiment would never require statistical methods is, however, one that does not stand up even to cursory examination.


Stratified Random Sampling Implementation how to in R?

Question: I have a dataset of 20 million rows. it is organized into strata (groups), and I need to sample from them. I need to create a smaller sampled dataset on which I bulid a regression model.
I need to first determine total sample size and then the sample size for each stratum, and then choose the training and test indexes.
How does one implement all of this in R? can anyone provide some links to this?
Answer: For a stratified sample you can use caTools library. If your factor variable is strata and you want 70% of the data as train, the code is
library(caTools)
train_rows = sample.split(data$strata, SplitRatio=0.7)
train = data[ train_rows,]
test  = data[-train_rows,]

Ps: it should be:
library(caTools)
train_rows = sample.split(data$strata, SplitRatio=0.7)
train = data[ train_rows,]
test  = data[!train_rows,]

Now, it should be working
Comment: It is probably very slow with 20 million rows. If you want to check if the code runs at all, try it on a small subsample.
Comment: I tried using sample.split but the code gets stuck. It does not run at all.
Answer: In addition to @lanenok's answer, you seem to need to determine the sample size per stratum. One way to do so is to use Neyman's allocation, which is implemented in the stratification package :
library(stratification)

## Example dataset
N <- 1000
id <- c(1:N)
y <- 10*runif(N) # Quantitative variable from which strata are constructed
df <- data.frame(cbind(id,y))

n <- 50 ## Example of sample size

testStrata <- strata.bh(y, c(2,4,7), n, Ls=4)

allocation <- testStrata$nh

In my example, 4 strata are constructed using the quantitative variable y, which range is (0,11) :

Stratum 1 : $y \in [0,2]$
Stratum 2 : $y \in [2,4]$
Stratum 3 : $y \in [4,7]$
Stratum 4 : $y \in [7,11]$

In my example, this gives you an allocation computed with Neyman's formula :
allocation = c(9,7,16,18)

Then, you can draw your sample using the package sampling :
library(sampling)

## These lines construct the strata variable for our example
df$strataId <- sapply(df$y, function(x) {
  if (x<=2) {return(1)}
  if(x>=2 && x<=4) {return(2)}
  if (x>=4 && x<=7) {return(3)}
   if(x>=7) {return(4)}
  return(4)
                    })

sample <- strata(df, c("strataId"), size=allocation)

which finally gives you 50 units, drawn using stratified sampling and the Neyman allocation.
Comment: can you pl explain what this line of code does : testStrata <- strata.bh(y, c(2,4,7), n, Ls=4) ? and why do we provide c(2,4,7) as an argument? how do you get that?
Comment: It computes the [Neyman Allocation](http://stattrek.com/statistics/dictionary.aspx?definition=Neyman_allocation), assuming your strata were created regarding the values of a variable $y$. In this example, $y$ varies from $0$ to $11$, and 4 strata are defined, using 4 different value intervals of $y$.
Answer: I assume all the data which you have is in x
library(caret)
# considering response variable as strata
data_part <- createDataPartition(y = x$Response,
                                 p = 0.7, list = F)
test <- x[-data_part,] # 30% data goes here
train <- x[data_part,] # 70% here

Hope it helps.


What firewall ports do I need to open when using FTPS?

Question: I need to access an FTPS server (vsftpd) on a vendor's site. The vendor has a firewall in front of the ftps server. I have a firewall in front of my FTPS client.
I understand that ports 990, 991 and maybe 989 need to be opened up for control traffic.
I have some questions:

When looking at it from the vendor's firewall perspective, should these ports be opened up for both inbound and outbound traffic?
What about ports for the DATA channel?
Do I have to open all ports above 1000?
Should I do it for both inbound and outbound traffic ?
Answer: I believe the ports around 990 were for implicit SSL, which was an old non-standard way of doing FTP/SSL. The "right" way these days is explicit SSL, which means you still connect on port 21 and then negotiate SSL before sending your goodies. To support connections through a firewall, you need to use PASV mode and hard set the data ports to be used.
I believe you need at least one port per data connection you want to support. If it's just you, you're probably fine only opening a few extra ports. Specifically for me, I use 21000-21010.
In vsftpd.conf, I have these two lines (along with all the other stuff to support SSL):

pasv_min_port=21000
pasv_max_port=21010

On my firewall, I have a public static IP with one-to-one/static NAT to the internal IP and only tcp ports 21, 21000-21010 open.
Answer: My understanding of FTP over SSL (ftps) is that it doesn't work well with firewalls and NAT. In an ordinary FTP session, the information about data connections is read, and for NAT modified, by the firewall in order for the firewall to dynamically open the needed ports. If that information is secured by SSL, the firewall can't read it or change it.
Using SFTP, or scp, makes the network administrator's job a lot easier - everything happens on the server's port 22, and the transaction follows the normal client/server model.
One thing not mentioned is whether or not your firewall is performing NAT and whether or not it is static NAT or dynamic NAT. If your client machine has a static address or is being statically NATed, you may not need to make any firewall changes, assuming you allow all outbound traffic and the server operates only in Passive mode (PASV).
To know exactly what ports you will need to open, you will need to either:
a) talk to the vendor to get specifics about how their system has been configured.
b) Use a protocol analyzer, such as tcpdump or wireshark, to look at the traffic, both from outside your firewall and inside your firewall
You need to find out which port is the Control Connection. You list 3, which seems odd to me. Assuming the server only works in PASV (passive) mode, you need to figure out how the server is configured to allocated DATA ports.  Have they locked down the DATA channel to a single inbound port? Have they locked down the DATA channel to a small range or ports?
With these answers, you can start configuring your firewall.
Comment: passive mode FTPS would use a control port over port# 1024 and so it would work better with a firewall than non-passive.  the client tells port 21 what upper-bound port to open and so you can configure the client to say "control is on port 2000 or 2001" and then the server will open outbound port 2000 or 2001.  most FTP clients support defining a specific range of ports for "control" to be on and that makes firewall definitions easy.
Answer: I know this is an extremely old thread, however..
Please note that SFTP is completely different from FTPS. ( SSH vs SSL )
FTPS worsk in 2 way. Explicit and Implicit. Explicit is less secure because after the initial handshake skips encryption during data transfers [if data encryption is maintained is configurable on server side with PROT P], while the Implicit keeps the encryption of the data after handshake too.
The default Explicit FTPS port is 21. The default Implicit port is 990 ( after handshake it will switch automatically to 989 for data transmission, if not configured differently).
While port 21 is generally accepted as EXPLICIT FTPS and 990 as IMPLICIT FTPS, in reality whichever port you will configure, except 990/989, will lead to EXPLICIT FTPS while ONLY 990/989 will be accepted as IMPLICIT FTPS.
So, to answer your question:
- depending on the FTPS Server configuration, you'll need to open port 21 or 990/989. However, just to be sure, you should contact the FTPS Server admin and ask for directions. Also, keep in mind that for passive mode, as with every other FTP software, you'll have to open additional ports (TCP/UDP) usually something from the range 64000-65000.
Answer: Basically ftps is almost useless, because you must make embarrassing requests to firewall admins. The advice to restrict ports to 10 is good. Much more, it gets pathetic.
sftp is a lot better in theory. But you need a viable sftp server, e.g. one that restricts clients to their own home directory.
Depending on the application, consider HTTPS. A file upload is really simple, and a download obviously is as well. If you're scripting the FTP anyway, it's probably going to be easier altogether to script an HTTPS file upload.
Automated FTP is a sign of a design problem. I noticed this when dealing with a total of about a dozen vendors that 'required' a place I worked to do automated FTP (for VERY important things), and when making dozens of customers do it with that same shop (a design failure for about 20 distinct uses I witnessed). It was easy to convince most app guys to use HTTPS (usually at the mention, they said "wait, there's no reason we're not just having them get it with HTTPS from the web server we're already serving them data on?"), except a few that gave responses like "well, we already have these scripts that seem to work, and no one on our team is really good with scripting so we cant really make any changes" (a team of 5-10 programmers, pretending to not understand that they can write it in a language of their choice, because they don't know how to write a trivial program from scratch.).
Answer: The vendor may be able to configure a narrow port range for the DATA connection ports, if they haven't already.  Then you can open the same range on your end, for the hosts that need such access.  PASV mode should be used.
Answer: Port 22 is standard since  the SSH daemon on UNIX has a SFTP module that you can enable to basically make a explicit SFTP server.  If you want to run a implicit FTP server with Filezilla then you can run it on any port you want but there is a catch:  if you use FileZilla client you need to specify the ftp site URL as  ftps://mysite.com:8086  rather than putting the port in the separate port field that the FileZilla client provides.
For the explicit option you only need ONE port: 22.  For the implicit option you only need to have the firewall open for the control port: 8086 (which forwards internally to port 21 on your filezilla server).
Comment: The question asked about FTPS, not SFTP.
Answer: if ftps is the same as sftp, then you only need to be able to access port 22 on the vendor's site.
On your end you should configure your firewall to allow port 22 outgoing, and related incoming traffic.  This will allow communication on any incoming port that is related to the initial outgoing connection on port 22.
Comment: SFTP is not the same as ftps (http://www.codeguru.com/csharp/.net/net_general/internet/article.php/c14329). SFTP is the file transfer protocol used with SSH. FTPS is FTP with SSL,

FTPS starts a new DATA connection on a new random port, which makes it tough to deploy behind firewalls, but I can't remove the firewall in this situation.
Comment: Sorry then.  I will leave this post though, in case others have the same confusion.
Comment: It's a hugely common confusion. From the firewall side, sftp is way easier to allow, except that it's built on a protocol that is intended for login access. Negligent for independent security groups to allow without significant independent validation. Difficult for sysadmins to find/configure a reasonable sftp server for untrusted clients.

Any discussion of sftp or ftps must mention the other protocol becauuse confusion is SO common.


wrong number of factors

Question: I am supposed, based on literarture, to get 3 factors from a scale I applied. However, the exploratory factor analysis leads me to a 5 factor solution. Is there anything wrong? What should I check?
Answer: If you were intending to test whether the previously proposed factor structure is correct, you want confirmatory factor analysis, not exploratory factor analysis.
If you really did you want exploratory factor analysis, and you're using new data, you shouldn't be too surprised to get a different factor structure. Exploratory factor analysis comes with no guarantee of consistency.

wrong number of factors


Extending a field homomorphism on a finite field

Question: Let $p$ be a prime number. Let $a,b,c$ be positive integers, $a$ divides $b$ and $b$ divides $c$.
Fix homomorphisms $f\colon\mathbb{F}_{p^{a}}\to\mathbb{F}_{p^{b}}$ and $g\colon\mathbb{F}_{p^{a}}\to\mathbb{F}_{p^{c}}$.
I want to show that there is a field homomorphism $h\colon\mathbb{F}_{p^{b}}\to\mathbb{F}_{p^{c}}$ such that $h\circ f=g$.
I don't want to use the algebraic closure of $\mathbb{F}_p$ (my question arises from trying to understand the construction of this algebraic closure via direct limits).
Comment: The standard one is: Let $E$ be a field, $p(x)$ an irreducible polynomial with coefficients in $E$, and let $F$ be an extension of $E$ with $F=E(\alpha)$, where $\alpha$ is a root of $p(x)$. If $\sigma\colon E\to K$ is a field homomorphism such that $\sigma(p)$ has a root $\beta$ in $K$,then there exists a homomorphism $\tau\colon F\to K$ such that $\tau(\alpha)=\beta$ and $\tau|_{E} = \sigma$.
Comment: Do you know that $g$ extends to a homomorphism $\mathbb{F}_{p^b}\to\mathbb{F}_{p^c}$?
Comment: @ArturoMagidin: I don't know that. Extends in what sense? By looking at $\mathbb{F}_{p^a}$ as a subfield of $\mathbb{F}_{p^b}$ via $f$?
Comment: If you use $f$ then the extension is exactly what you are asking about. Do you not know any theorems about extending morphisms to field extensions?
Comment: @ArturoMagidin: The theorems that I know discuss the situation where the range is algebraically closed. If you can state the general useful theorem that can help me here, I'll appreciate it. Even with no proof and no reference it will be useful.
Comment: @ArturoMagidin: If you do state such a theorem, it will be useful to state a general one. One thing I am confused about is what is special about the finite field situation and what is general.
Comment: @ArturoMagidin: OK. I proved the theorem you stated. Now I will think how to use it.
Answer: Let $K,L,M$ be three finite fields with two homomorphisms
$$\require{AMScd}
\begin{CD}
K @>{\alpha}>> M \hookrightarrow L \\
@V{\beta}VV \\
L
\end{CD}$$
We need to find a homomorphism $\gamma : M \to L$ that makes the triangle commutative. It suffices to find a homomorphism $L \to L$ with that property (because this can then be restricted to $L$). So the task simplifies to completing diagrams of the form
$$\require{AMScd}
\begin{CD}
K @>{\alpha}>> L  \\
@V{\beta}VV \\
L
\end{CD}$$
to a homomorphism $\gamma : L \to L$ with $\gamma \circ \alpha = \beta$. In fact, $\gamma$ will be an automorphism.
Notice that $\mathrm{im}(\alpha)$ and $\mathrm{im}(\beta)$ are two subfields of $L$ with the same order. Hence, they are equal. In fact, if $K$ has order $q$ and $L$ has order $q^m$, then $L$ has a unique subfield of order $q$, namely $\{x \in L : x^q = x\}$.
It follows that there is some automorphism $\pi : K \to K$ with $\beta = \alpha \circ \pi$ (we define $\pi$ by $\beta(x) = \alpha(\pi(x))$, it exists since $\beta(x) \in \mathrm{im}(\alpha)$ etc.). So we can completely remove $\beta$ from the task, we only need to find $\gamma$ with $\gamma \circ \alpha = \alpha \circ \pi$.
$$\require{AMScd}
\begin{CD}
K @>{\alpha}>> L  \\
@V{\pi}VV @VV{\gamma}V \\
K @>>{\alpha}> L
\end{CD}$$
This can be proven in at least two ways. One proof uses that $\alpha : K \to L$ is a normal extension. Another proof just uses that we actually know all the automorphisms of $K$, they are all powers of the Frobenius $x \mapsto x^p$, where $p = \mathrm{char}(K)$. On $L$, we can then take the same power of the Frobenius, and that's it.


How do I build a 2d matrix using STDIN in Perl?

Question: How do I build a 2d matrix using STDIN?
If I input a matrix like so:
1 2 3
4 5 6
7 5 6

7 8 9
4 5 6
3 3 3

how do I input this and create two matrices out of this?
Here's my code so far
while (defined ($a=<STDIN>)) {
    chomp ($a);
    push @a,($a);
    }

This is just for the input.
My understanding is I can just add each row to a stack.  When the matrices are all put in I can take each line, break by space to create an array.  I then need to create an array reference and push this reference into an array to create my matrix.  How the heck do I do this?  Is there an easier way?  I've been bashing my head on this for 3 days now.  I feel pretty damn stupid right now...
Answer: Let's make that code you have a little more Perl-y, and we'll do everything you need done in one pass:
my @a = ();
while(<>) {
  push @a, [ split ];
}

This is taking a lot out of your answer, so I'll opt to explain it, rather than aiming for John Wayne-like answering reflexes. We'll start with your line here:
while(defined(my $a = <STDIN>))

Perl users know that many loops will implicitly use the $_ variable. If you need lots of nested loops, you should avoid using that variable, and use well-named variables for each level of looping, but in this case we only have one level, so let's go ahead and use it:
while(defined($_ = <STDIN>))

Now, Perl is kind enough to understand that we want to test for defined()ness a lot, so it will allow us to shorten that to this:
while(<STDIN>)

This is implicitly translated by Perl as assigning the line read to $_ and returning true as long as the result is defined (and therefore until end-of-file occurs). However, Perl gives us one more trick:
while(<>)

This will loop over STDIN or, if arguments are given on the command line, it will open those as files and loop over them. So this still reads from STDIN:
./myscript.pl

But we can also read from one or more files:
./myscript.pl myfile [myfile2 [myfile3 ...]]

It's easier and more intuitive than using the shell to do the same (though this will still work):
cat myfile [myfile2 [myfile3 ...]] | ./myscript.pl

If you don't want this behavior, you can change it back to <STDIN>, but consider keeping it.
The loop is:
push @a, [ split ];

First, split() with no arguments is identical to split /\s+/, $_ (i.e. it splits the $_ string on occurrences of whitespace characters), and due to the subtleties of split empty trailing fields are removed, so a chomp() is unnecessary. Then, [] creates an anonymous array reference (which, in this case, contains the contents of our split $_ string). Then, we push that array reference onto @a. Simple as pie, you now have a two-dimensional matrix from your standard input.
Comment: More Perl-y! Me want  `@a=map{[split]}<>` !
Comment: Note that unlike @Ether's similar (but more explicit) answer, this will tolerate irregular input and jagged arrays. See his answer if you don't want that (although his answer may cause problems if someone tries to enter a jagged array).
Comment: Sorry, got code golf on the brain. But the `chomp` is superfluous with a bare `split`.
Comment: @mobrule - If you can show me that `my @a = map { [ split ] } <>` gets translated down to the same bytecode as what I posted, rather than reading in the entire `<>` filehandle _and then_ `map()` ing it, I'll add that to my answer.
Comment: I forgot about the subtleties of bare `split` - Thanks.
Comment: @Chris Lutz - Thank You so much!  Thank you for explaining it to a n00b like me.
Answer: Try this:
use strict;
use warnings;
use Data::Dumper;

my @matrix;

while (my $line = <>) {
    chomp $line;
    my @row = split /\s+/, $line, 3;
    push @matrix, \@row;
}
print Dumper(\@matrix);

Instead of using <STDIN> explicitly, you can read from either stdin or a piped file with <>.
Inputting one matrix gives the result:
$VAR1 = [
          [
            '1',
            '2',
            '3'
          ],
          [
            '4',
            '5',
            '6'
          ],
          [
            '7',
            '8',
            '9'
          ]
        ];

From here you should be able to see what you need to do to read in two matrices.
Answer: The other answers seem to be missing the requirement to read multiple matrices from the same input, breaking on a blank line. There are a few different ways to go about this, including frobbing $/, but here's one that appeals to me.
# Read a matrix from a handle, with columns delimited by whitespace
# and rows delimited by newlines. A matrix ends at a blank line
# (which is consumed) or EOF.
sub read_matrix_from {
  my ($handle) = @_;
  my @out;
  while (<$handle>) {
    last unless /\S/;
    push @out, [ split ];
  }
  return \@out;
}

my @matrices;
push @matrices, read_matrix_from(\*ARGV) until eof();

Season the last part to taste, of course -- you might be using an explicitly opened handle instead of ARGV magic, and you might know in advance how many things you're reading instead of going to EOF, etc.
Comment: +1 I like that. If only there were a nicer way to pass around old-style filehandles.
Comment: how do I go about handling multiple matrices with the above code? Say I have
1 2 3
4 5 6
4 5 6

1 2 2
1 2 2
8 9 9

7 8 1
5 6 3
9 4 8

how would I multiply them all together?  wouldn't I have to count how many matrices where input then create a new matrix for each?  I'm lost.


Find $n$ such that $209$ divides $n^{180}-n^{20}-n^{36}+1$

Question: Finding $n\in \mathbb{N}$ (with $n > 1$) such that $209$ divides $n^{180}-n^{20}-n^{36}+1$ is equivalent to solving $$ n^{180} - n^{20} - n^{36} + 1 \equiv 0 \mod 11 \quad \text{ and } \quad n^{180} - n^{20} - n^{36} + 1 \equiv 0 \mod 19. $$
Using Fermat Little Theorem one can further reduce the problem to $$ n^{6}    -1\equiv 0 \mod 11 \quad \text{ and } \quad  (n + 1)(n-1) \equiv 0 \mod 19$$
How does one solve this problem (specifically, $n^{6} -1  \equiv 0 \mod 11$) in a straightforward way?
Comment: Any restriction on $n$? Because $n=1$ works.
Answer: $n^{6} \equiv 1 \bmod 11$ and $n^{10} \equiv 1 \bmod 11$ imply $n^{2} \equiv 1 \bmod 11$ because $\gcd(6,10)=2$.
So $n^{180}-n^{20}-n^{36}+1 \equiv 0\bmod 209$ is equivalent to $n^{2} \equiv 1 \bmod 209$, which is much easier to solve, giving $n=\pm 1, \pm 56$.
Comment: @Jai, Fermat for $p=11$ and $10=2+2\cdot 6 $.


J-Meter performance test results varies a lot for Home Wifi & Office Lan Networks. Why?

Question: I have recorded a Test over VPN and run it successfully in J-Meter 4.0. I have tested it with different loads in my office via the Office Lan Network and also from my home over my home wifi. BUT My confusion is that for the same test with identical load and everything, I am getting very different performance testing results.
I will say the performance tests done via by Office LAN network is more accurate as I can check that manually as well. But the tests done over my home wifi when I am working from home, for the exact same test with same load, gives much better and improved performance test results which is inaccurate. When I gave 300 users load in Office LAN, the application broke & gave horrible results. In my Home wifi, the same test was giving good results. Can please someone explain why this happening and how I can fix this??
Office LAN Network: [Average response times (ART)]
20  User test for 2 mins: 2.2 seconds, 0% Error
300 User test for 3 mins: 45.1 seconds, 25% Error
Home Wifi: [Average response times (ART)]
20  User test for 2 mins: 1.1 seconds, 0% Error
300 User test for 3 mins: 1.4 seconds, 0% Error
Answer: System and network resource play a very crucial role in performance testing. Different system and network are bound to produce different results. You need to check the error for more info and need to find out the reason. Internet bandwidth and network congestion can also be the possible causes of the errors.
Check the servers log, event log to get more information on the errors, if it is not clear from the jmeter csv report.
Hope it helps.


keeping positionning after selecting an item from comboboc

Question: I have a form with 2 combobox: statutbox and categorybox, the statutbox is on the top of my page, but categorybox is in the middle.my problem is when i select an item from the categorybox which is in the middle , the page is refreshed and returns to the top.
so how to how to keep the same positioning.
here is my code:
   <form id = "formbox" name="form" method = "POST" action="parproj.php">
<select id="statutbox" name="statut" onChange= "this.form.submit()" style= "width:300px;padding:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow: 0 3px 0;cursor:pointer;">

                <?php
                $reponse2 = $bdd->query('select name from issue_statuses order by name asc;');
                while ($donnees2 = $reponse2->fetch())
                {
                    $selected2 = (isset($_POST['statut']) and $_POST['statut'] == $donnees2["name"])?'selected="selected"':'';
                    echo '<option value="'.$donnees2['name'].'" '.$selected2.'>'.$donnees2['name'].'</option>';

                }

                $reponse2->closeCursor(); // Termine le traitement de la requête

                ?>
              </select>

               </form>

<select id="categorybox" form="formbox" name="category" onChange= "this.form.submit()" style= "width:300px;padding:2px;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px;-webkit-box-shadow: 0 3px 0;cursor:pointer;">

                <?php
                while ($donnees3 = $reponsecat->fetch())
                {
                    $selectedcat = (isset($_POST['category']) and $_POST['category'] == $donnees3["category"])?'selected="selected"':'';
                    echo '<option value="'.$donnees3['category'].'" '.$selectedcat.'>'.$donnees3['category'].'</option>';
                }

                $reponsecat->closeCursor(); // Termine le traitement de la requête
               ?>
               </select>

Comment: your second `select` is out of the form tag.
Answer: There are two solutions. Check the example below.
<!DOCTYPE html>

<meta charset="utf-8">
<title>Scroll to an element</title>

<form method="post" action="<?php echo htmlentities($_SERVER['PHP_SELF']);?>">
<select name="status" id="status" onchange="this.form.submit();">
    <option value="0">0</option>
    <option value="1">1</option>
    <option value="2">2</option>
</select>

<select name="category" id="category" onchange="this.form.submit();">
    <option value="0">0</option>
    <option value="1">1</option>
    <option value="2">2</option>
</select>
<input type="submit" name="button" value="button" />
</form>

Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.

<?php
    if(isset($_POST["status"]) && $_POST["status"] != 0){

        // 1st solution
        header("Location: http://" . $_SERVER["HTTP_HOST"] . $_SERVER["PHP_SELF"] . "#category");

        // or

        // 2nd solution
        echo '';
    }
?>

Don't forget to comment the solution you won't use. this.form.submit() is optional, you may submit the form by a regular submit button.
Comment: it works for the two combobox, but i want that works only when i select an item from categorybox
Comment: @Salma I wrote an entire example for you so I hope you'll find it helpful. Give it a try :)
Comment: thank you so much, i tried the both of them , i've always been taken to the top of the page, i don't want to use the submit button :(
Comment: @Salma So, since this is not quite clear, did you solve your issue or not?
Comment: not yet, i tried your solution, but it has no effect, i can't see where is the problem :(
Comment: @Salma If you save the example above as a .php file it will work just fine - I tested it before pasted here. Otherwise, paste your code within your question, drop me a note here and I'll check it in the meantime.
Comment: I tested your example alone an it works, but im my case i test category , i'm not interested in testing status, i want when i click on category that i still in the same positionning
Comment: @Salma Well, change this `if(isset($_POST["status"]) && $_POST["status"] != 0){` to this `if(isset($_POST["category"]) && $_POST["category"] != 0){`
Comment: I found the error, in fact the problem come from this part : $_POST["category"] != 0) I change it by string and it works
Comment: @Salma By the way, that's not an error; that's the right way dealing with select list. Your default option on a select list should be `value = 0` and not `value = ''`. However, this `$_POST["status"] != 0` could be written also like this `$_POST["status"] != FALSE` and you (if you want) may use `0` or an empty string
Comment: @Salma Could you please tell me why you're not up-voting and accepting my answer since it helped you to solve the issue :)


Using a Layout with a Canvas or specifying a background color with a background drawable on a canvas

Question: I'm creating a small game for a programming assignment. I'm trying to figure out how to set the background properly, but I can't seem to figure out.
I have a two linear layouts in my main.xml; one has a background color, and another layout that's inside it has a background image (transparent PNG). It shows up fine when I start up the app:
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:background="#aacceeff"
    >

 <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/background"
    android:orientation="vertical"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent">
    >

 </LinearLayout>

</LinearLayout>

However, I now have to add the animation. To do that I created a View object that loads up images and animates them. However, I now have to set the content view from the main activity to an instance of this new view object. When I do that, I get a null pointer exception because it cannot find the linear layout with id background:
BitmapDrawable tiledBackground = new BitmapDrawable(BitmapFactory.decodeResource(getResources(), R.drawable.background));
LinearLayout backgroundLayout = (LinearLayout) findViewById(R.id.background);
tiledBackground.setTileModeX(Shader.TileMode.REPEAT);
backgroundLayout.setBackgroundDrawable(tiledBackground); //backgroundLayout is null

I figure this is because the view is no longer the view that's specified in main.xml. If I remove this code, the code runs fine but I get a black background. I tried moving this specific snippet to the custom view and set the background to the image, but then I get the image with a black background; what I actually want is a transparent image with the background color showing through.
I guess what I'm trying to figure out is how to tell the custom view to use the layout I've defined in main.xml, or figure out a way to specify a background color and a background drawable for the custom view.
Answer: I was able to solve this by using a color filter:
super(context);

BitmapDrawable tiledBackground = new BitmapDrawable(BitmapFactory.decodeResource(getResources(), R.drawable.background));
tiledBackground.setTileModeX(Shader.TileMode.REPEAT);

//I determined the PorterDuff.Mode value mainly through trial and error
tiledBackground.setColorFilter(0xaacceeff, PorterDuff.Mode.DARKEN);

this.setBackgroundDrawable(tiledBackground)

Answer: When you use findViewById method, you are looking for a layout or view from the layout.xml you specify in the setContentView method.
So if you call setContentView and set another xml file as the layout, then you will not be able to find a view from the previous layout.
In ideal cases setContentView is only called once for an activity. New activities are called for different screens. But you call also use the ViewFlipper class, you can add both your layouts as its childs. It will only show 1 child at a time. To show the next child you can use
viewflipper.setDisplayChild(1);

Comment: Right, I figured that part out. I'm trying to figure out how to get around that.
Comment: I guess I'm not seeing how this will fix my issue. I'm basically trying to use the layout I've already defined, along with the custom view that I have. So I want the background in the main layout to show up with the custom view.


Passing arguments to non interactive script interactively

Question: Does anyone have an idea how to pass something interactively to a script which does not support interactive mode? Is there any shell feature / application etc. or any other way to wrap the script / application by something in such a way that I could pass arguments and then hide them? I don't want my sensitive data such as password were visible in keyboard buffer, on the process list etc. Is it possible?
Comment: Can you ask a more specific question (or at least provide an *actual* example)?  There are too many hypotheticals here to figure out what problem you're actually trying to solve.
Comment: After re-reading your question and a little coffee, I believe this link should help.
http://stackoverflow.com/questions/3830823/hiding-secret-from-command-line-parameter-on-unix
Comment: And this one.
http://serverfault.com/questions/592744/how-to-hide-a-password-passed-as-command-line-argument
Answer:
First, check if this is really necessary. Some software rewrites its command line after startup.
If your executable is a binary, there are ways to dynamically load your program into an existing process.
You were talking about a script. Assuming it is Bash or compatible, you can easily write a wrapper that reads your secret, then sources the existing script.

Because no new process is generated, the secret will not be part of any command line, but it will be available as shell parameter (here $1) while yourscript is executing:
#!/bin/bash
read secret
. <yourscript> "$secret"


Android NDK: How to solve "E/NdkMediaExtractor: setDataSource(path) must be called from Java thread"

Question: I am trying to use Androids NDKMediaExtractor in C++ code like this:
const char *storageFile = "/storage/emulated/0/mytestfile.mp3";
AMediaExtractor *extractor = AMediaExtractor_new();
media_status_t amresult = AMediaExtractor_setDataSource(extractor, storageFile);

This fails with the error message E/NdkMediaExtractor: setDataSource(path) must be called from Java thread.
Searching online did not bring me any hints how I can solve this, what I did find though was the source code, the responsible part looks like this:
EXPORT
media_status_t AMediaExtractor_setDataSource(AMediaExtractor *mData, const char *location) {
    ALOGV("setDataSource(%s)", location);
    // TODO: add header support
    JNIEnv *env = AndroidRuntime::getJNIEnv();
    jobject service = NULL;
    if (env == NULL) {
        ALOGE("setDataSource(path) must be called from Java thread");
        env->ExceptionClear();
        return AMEDIA_ERROR_UNSUPPORTED;
    }

    // rest of method omitted for brevity
}

I hand in a JNIEnv when I call my native methods from Java/Kotlin, but NDKMediaExtractordoes not seem to know about it. How can I solve this?
Comment: Did you switch to another (native) thread before calling `AMediaExtractor_setDataSource`? AFAIK, `AndroidRuntime::getJNIEnv` will not to attempt to attach the current thread to the VM.
Comment: Not that I know of - but I am still learning my ways around the NDK. I have noticed though that if I create and use the extractor directly inside the method that is called from the JAVA side, this problem doesn't occur. But if I instantiate a class, that instantiates another class that builds the extractor, it fails.
Answer: you cant just pass JNIenv* between threads you should get it(every time in a native thread) by calling AttachCurrentThread(before calling any jni methods)in the current native thread so that it can be managed as java Thread.Once you do this it should be working.
i.e
 JNIEnv* env;
virtualMachine->AttachCurrentThread(&env,NULL);

you should obtain env by calling  AttachCurrentThread and now your native thread is managed as Java thread after attaching.So should resolve ur error.
Comment: Please provide more information for further detail.


Notice: Undefined index: main_profile in include() (line 54

Question: global $user;
$user->uid ;
print_r($user->uid);
print "print out the user value" . $user->uid;

$uid =3;

$account = user_load($user->uid);

$profile = profile2_load_by_user($account);

$print_first_name =  drupal_render(field_view_field('profile2',$profile['main_profile'], 'field_first_name'));
print $print_first_name;

What I have attempted to do is to pass the $user->uid to the user_load function.
When I pass $uid = 3 to the user_load function it seems to work fine. I am wondering what approach I should try to fix this problem.
error message:
EntityMalformedException: Missing bundle property on entity of type profile2. in entity_extract_ids() (line 7562 of /home/td6/public_html//includes/common.inc).
Comment: This is poorly titled as it seems to be cut off and the error message in the title is different to the one you mention at the end of your post.
Answer: Your mistakes is here main_profile , profile type always start with profile_name . just need change main_profile to profile_main
global $user;
$user->uid ;
print_r($user->uid);
print "print out the user value" . $user->uid;

$uid =3;

$account = user_load($user->uid);
$profile = profile2_load_by_user($account);

$print_first_name =  drupal_render(field_view_field('profile2',$profile['profile_main'], 'field_first_name'));
print $print_first_name;

If you want see all your profile type and correct name and it's contents print_r or var_dump $profile
$profile = profile2_load_by_user($account);
print_r($profile);

Comment: profile2 profiles do not always start with profile_, but like you say, the issue is that main_profile is not the name of the profile. Also, drupal_render takes an element by reference so you really should pass in a variable, not a function.  For example, set a variable to the result of field_view_field and then pass the variable into drupal_render().


Automatically add users to the same group as their group admin

Question: I am using the Organic group module. I created 2 groups on my site. To each group I assigned 1 user as group admin. Group admins have the permission to create users.
My question is how to automatically assign the users which got created by a group admin to the group admin's group?
Answer: In Drupal 7, you can achieve it using hook_user_insert and OG function og_group.

hook_user_insert : Called when a user account is created
og_group : Set an association (e.g. subscribe) an entity to a group

<code> /**
     * Implemention hook_user_insert
     * @global type $user
     * @param type $edit
     * @param type $account
     * @param type $category
     */
    function HOOK_user_insert(&$edit, $account, $category) {
        global $user;
        $usr_grps = og_get_entity_groups(); //get all user groups
        if (count($usr_grps) > 0) {//if any groups
            $grp_mgr_grps = array();
            foreach ($usr_grps['node'] as $key => $gid) {
                $group_type = 'node';
                $group = entity_load_single($group_type, $gid);
                if (is_object($group)) {
                    if (!empty($group->uid) && ($group->uid == $user->uid)) {//subscribe only the groups where the user is group manager
                        $grp_mgr_grps[] = $gid;
                    }
                }
            }
            if (count($grp_mgr_grps) > 0 && is_object($account)) {
                HOOK_subscribe_grpmgr_groups($grp_mgr_grps, $account); //subscribe the group
            }
        }
    }

    /**
     *
     * @param type $grp_mgr_grps : array of group ids
     * @param type $account : Object of the user we want to add to the group
     */
    function HOOK_subscribe_grpmgr_groups($grp_mgr_grps, $account) {
        foreach ($grp_mgr_grps as $key => $gid) {
            $group_type = 'node'; //All groups are of type node
            $values = array(
                    'entity_type' => 'user',
                    'entity' => $account->uid,
                    'field_name' => FALSE,
                    'state' => 1,
            );
            og_group($group_type, $gid, $values);
        }
    }</code>
Comment: Please post your answer so that I and others can also get to know, another approach to doing it using RULES.
Comment: I have tried your code but after group admin submit user creation form, I didn't see the user is created appear in group members list. Could you take a look again please?
Comment: I have updated my answer. You can check it now.
Comment: Thank you for updating, I have tried another way by Rules and it works now. But I will check your code.


Auto populate a node add with values of another node

Question: Is there any module that auto-populates the fields of a node/add form with the values of an already existing node? I would like to pass the node ID in the URL and get the form prepopulated with the values taken from that node.
I know it is possible to do; I am just wondering if someone has already done it in a module I could use.
Comment: There is the Prepopulate Module (http://drupal.org/project/prepopulate) that might help you along the way with respect to fields, but I'm not aware of any that does it for all of them.
Comment: I'm voting to close this question because it's asking for recommendations or suggestions on hosting, books, tools, modules, themes, distributions, tutorials, or other off-site resources.
Answer: Did you mean node clone module?

The clone module allows users to make a copy of an existing item of site content (a node) and then edit that copy. The authorship is set to the current user, the menu and url aliases are reset, and the words "Clone of" are inserted into the title to remind you that you are not editing the original content.
Comment: Consider filing a bug report. It seems it should be easy to fix, so maybe it will get fixed before you will find a better way.
Comment: Do you know if this module works fine with field collections?
Comment: As far as I remember, yes, but you need to test it yourself - there are so many add-ons not compatible with each other it's hard to tell what will and what won't work on any given setup.
Comment: Just tested it. It does not work as I expected. It copies the reference for the collection, so if you change a value of a field of the field collection in the new copy, it will also change the value on the old one.


Anyone know of a script, program, etc... to check folder/file permissions and repair/restore?

Question: I am not a newb. I am a professional imploring to other professionals over their discoveries of a quick solution to a common problem, and I know what chmod -R is.
This is in response to the -1 rating instantly given to my question, for those who might shrug off the seriousness of my inquiry, or dismiss its validity.

Experimenting with sshfs between my desktop and a remote server running Kali linux as root (Default for Kali). Managed to bugger up all file & folder permissions/ownership on the system. I have managed to repair some, with a simple chmod -R 0755 where appropriate, but have noticed many are still not fixed.
Wondering if there is a bash script or other script or program that could help to restore the correct owner & permissions?
I have found a similar script, but it is primarily aimed at repairing home directories.
Script, Below:
#!/bin/bash
read -r -p "Correct file and folder permissions? [y/N] " chse
if [[ "$chse" =~ ^([yY][eE][sS]|[yY])+$ ]]; then
  echo "Processing ..."
  find -H $(pwd) -type d -exec chmod 0755 {} \;
  # set dirs to 755
  find -H $(pwd) -type f \( -iname '*.so.*' -o -iname '*.so' \) -exec chmod 0644 {} \;
  # libs
  IFS=$'\n'
  for value in $(find -H $(pwd) -type f ! \( -iname '*.so.*' -o -iname '*.so' -o -iname '*.bak' \) -printf '%p\n'); do
    tstbin=$(readelf -l "$value" 2>/dev/null | grep -Pio 'executable|shared')
    if [ -z "$tstbin" ]; then
      tstbat=$(cat "$value" | head -c2 | grep -io '#!')
      if [ -n "$tstbat" ]; then
        perm=$(stat -c '%a' "$value")
        if [ "$perm" != "755" ]; then
          chmod 755 $value
          echo "Set script  755 $value"
          # set batch to 755
        fi
      else
        perm=$(stat -c '%a' "$value")
        if [ "$perm" != "644" ]; then
          chmod 644 $value
          echo "Set regular 644 $value"
          # set regular files to 644
        fi
      fi
      # above aren't elf binary
    else
      perm=$(stat -c '%a' "$value")
      if [ "$perm" != "755" ]; then
        chmod 755 $value
        echo "Set binary  755 $value"
        # set elf binaries to 755
      fi
    fi
  done
  unset IFS
  # process linux permissions for files and folders
else
  echo "Aborted."
fi

Anyone found anything more substantial that can fix permissions and ownership for a file system?

--> Update: <--
Not discovering a desirable solution, I have taken to modifying the above script and making it facilitate my desired resolution:
permafix.sh - Gist on github
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

# ====================================================================
# --> Documentation <--
# ---------------------
#
# 0755 21 root root  .
# 0755 21 root root  ..
# 0755  2 root root  bin
# 0755  4 root root  boot
# 0755 15 root root  dev
# 0755 53 root root  etc
# 0755  4 root root  home
# 0755  7 root root  lib
# 0700  2 root root  lost+found
# 0755  6 root root  media
# 0755  2 root root  mnt
# 0755  4 root root  opt
# dr-xr-xr-x 87 root root  proc # Not touching this.
# 0744  8 root root  root
# 0755  2 root root  sbin
# 0755  3 root root  share
# 0755  4 root root  srv
# 0755 12 root root  sys
# 1777  7 root root  tmp
# 0755 12 root root  usr
# 0755 13 root root  var
#
# ========================================================================

read -r -p "Correct file and folder permissions? [y/N] " chse
if [[ "$chse" =~ ^([yY][eE][sS]|[yY])+$ ]]; then
    echo "Processing ..."
    #################
    # Special Cases #
    #################

    SDIR=("/lost+found" "/root" "/tmp")
    for sd in ${SDIR[-1]}; do
        perm=$(stat -c '%a' "$sd")
        user=$(stat -c '%U' "$sd")
        group=$(stat -c '%G' "$sd")
        if [ $sd = "/tmp" ]; then
            if [ "$perm" != 1777 ]; then
                chmod 1777 $sd
                echo "Set directory to 177 $sd"
            fi
        elif [ $sd = "/lost+found" ]; then
            if [ "$perm" != 0700 ]; then
                chmod 0700 $sd
                echo "Set directory to 0700 $sd"
            fi
        elif [ $sd = "/root" ]; then
            if [ "$perm" != 744 ];then
                chmod 744 $sd
                echo "Set directory to 744 $sd"
            fi
        else
            echo "Abort!"
        fi
        # Do change in ownership
        if [ "$user" != root ]; then
            chown root $sd
            echo "Set user to root $sd"
        fi
        if [ "$group" != root ]; then
            chgrp root $sd
            echo "Set group to root $sd"
        fi
    done
    ###############
    # Directories #
    ###############
    DIR=("/bin" "/boot" "/dev" "/etc" "/home" "/lib" "/media" "/mnt" "/opt" "/sbin" "/share" "/srv" "/sys" "/usr" "/var")
    for pd in ${DIR[-1]}; do
        perm=$(stat -c '%a' "$pd")
        user=$(stat -c '%U' "$pd")
        group=$(stat -c '%G' "$pd")
        if [ "$perm" != 755 ]; then
            chmod 755 $pd
            echo "Set directory to 755 $pd"
        fi
        if [ "$user" != root ]; then
            chown root $pd
            echo "Set user to root $pd"
        fi
        if [ "$group" != root ]; then
            chgrp root $pd
            echo "Set group to root $pd"
        fi

        ############################
        # Subdirectories and files #
        ############################
        # chmod directories to 755
        find -H $pd -type d -exec chmod 0755 {} \;
        # Check library files
        find -H $pd -type f \( -iname '*.so.*' -o -iname '*.so' \) -exec chmod 0644 {} \;
    done

    #------#
    # libs #
    #------#
    # Assign Variables
    LIBFILES=$(find -H "$(pwd)" -type f ! \( -iname '*.so.*' -o -iname '*.so' -o -iname '*.bak' \) -printf '%p\n')
    # Now do the hustle
    for PLF in $LIBFILES; do
        tstbin=$(readelf -l "$PLF" 2>/dev/null | grep -Pio 'executable|shared')
        if [ -z "$tstbin" ]; then
            tstbat=$(cat "$PLF" | head -c2 | grep -io '#!')
            if [ -n "$tstbat" ]; then
                perm=$(stat -c '%a' "$PLF")
                if [ "$perm" != "755" ]; then
                    chmod 755 $PLF
                    echo "Set script 755 $PLF"
                    # set batch to 755
                fi
            else
                perm=$(stat -c '%a' "$PLF")
                if [ "$perm" != "644" ]; then
                    chmod 644 $PLF
                    echo "Set regular 644 $PLF"
                    # set regular files to 644
                fi
            fi
            # above aren't elf binary
        else
            perm=$(stat -c '%a' "$PLF")
            if [ "$perm" != "755" ]; then
                chmod 755 $PLF
                echo "Set binary 755 $PLF"
                # set elf binaries to 755
            fi
        fi
    done
    unset IFS
    # process linux permissions for files and folders
else
    # When shit goes pear shaped
    echo "Aborted."
fi

There are other ways of doing this, and also better ways to write the code. But, it works for now.
--->Yet another update<---
I fixed a careless error in the script correctly relocation the position of several variables that were formerly inaccessible due to script structure.
Please use this link to acquire the most recent version of this script.
Comment: The problem is that no script or solution will be completely conclusive. You performed a "destructive action" that can't be reverted without a backup. Partial fixes include querying the package management system and using the default permissions stored from installed packages and forcing the correct default permissions on the main directory tree but that won't restore the rest of your permissions, which were either changed from their default, set by different ways such manually by admins and owners, by scripts/tools, post-install scripts in packages, systemd unit files, UMASK or inherited.
Comment: Copy off any unique data and re-install the system if you don't have backups.
Comment: Nope, not doing that. The system is too far developed. A simple script such as `sudo chmod 755 chmod -R 755 /bin /boot /dev /etc/ /home /lib /lib64 /media /mnt /opt /run /sbin /srv /usr /var && sudo chmod -R 777 /initrd.img /vmlinuz && sudo chmod -R 1777 /tmp && sudo chmod -R 555 /proc && sudo chmod -R 555 /sys && sudo chmod -R 744 /root`, would be better. Not the solution I am asking for, but still better.
Comment: You are damn right, I upvoted to bring in interest. I've seen servers been destroyed by `chmod -R`. It would be nice for each distro to have such a RevertTodefaultPermissions.sh; MacOSX had something similar in the past, don't know if it still exists. A friendly advice (but not a cure), use ACLs wherever possible. They can be removed and start over without impacting the system. They helped me mitigate the problem you describe.
Answer: A simple script to fix arbitrary broken permissions on a general purpose system does not exist. You, your users, and the software you use can set whatever permissions you like to meet your requirements. Overly broad permission changes are a loss of that metadata.
First, determine how permissions were broken, for example chmod chown setfacl or chcon commands. If ownership is wrong, you'll need to fix that, such as giving a file in a home directory back to its owner.  Beware there are subtle things here, like chown will clear setuid flags. For example, /usr/bin/ping might not work, if it loses setuid root. Do you know which other of your programs need setuid? More complex ACLs or selinux labels are not in your solution, but could complicate things if they also are wrong.
You can attempt to fix permissions to what they were on package install. On Debian based systems, you might feed dpkg --contents output into a chmod chown script.  Downloading .debs for all installed packages for dpkg to query is an exercise for the reader. This does nothing for user data, or software not installed via deb.
Identify sensitive information previously protected by file permissions. Including but not limited to ssh and gpg private keys. Consider changing these credentials out of an abundance of caution.
Regarding the rest of user data, hard to say. Users generally, but not always, own files in their home directories. Multiple user shared directories could get tricky, as the correct owner and mode might no longer be known.
Backups should restore with correct permissions, if they existed.
Tedious work to fix correctly. Document by writing automation scripting that fixes permissions on your data directories.  Hopefully backups will fix things next time, but might as well have policy on what the permissions should be.
Comment: This is in theory what resolved the issue that I was experiencing, except I went about accomplishing it differently. I discovered this wiki entry searching for a resolution: https://wiki.debian.org/Permissions/ResetPackagePermissions  Apparently, a simple `sudo apt install --reinstall ~i` did the trick.  Since then, I have only experienced one issue that was corrected with `chmod -R 644` quite easily.


Center of mass of a solid

Question: The problem statement, all variables and given/known data
Find the center of mass of the solid of uniform density bounded by the graphs of the equations: Wedge: $x^2+y^2=a^2$ $z=cy\qquad(c>0),\;y \geq 0, z \geq 0$.
Relevant equations
$$M_x= \int y \cdot \rho(x,y) dA$$
$$M_y= \int x \cdot \rho(x,y) dA$$
$$\overline{x} = \frac{M_y}{m}$$
$$\overline{y} = \frac{M_x}{m}$$
The attempt at a solution
I set up all the equations for $M_x, M_y, \overline{x}, \overline{y}$ but I cant seem to realize what the limits of integration are. I can't see how the $z=cy$ comes into play at all. Does it imply a 3 dimensional figure? Do I just integrate with respect to the limits of $x^2+y^2=a^2$?
Comment: Why are $M_x$ and $M_y$ not called $M_y$ and $M_x$, respectively? And why doesn't $\overline{z}$ appear? Finding that would also be part of finding the centre of mass.
Comment: Ah, I think see now -- it seems you're thinking in terms of a two-dimensional problem. A "solid" is usually a three-dimensional solid. The answer to your first question is, yes, $z=cy$ bounds the figure in the third dimension (since the other equation describes a cylinder, which is unbounded in $z$). Your symbols for the moments $M_x$ and $M_y$ make sense in a two-dimensional setting, but not in three dimensions, where $x$ and $y$ are not distinguished with respect to $z$, so I'd switch them around to avoid confusion, and add a corresponding $M_z$ and $\overline{z}$.
Comment: understood. But still I would like to know how to find the limits of the integration. Is 'c' the same as 'a' ? please attempt to answer if time permits.
Answer: No, there's no reason for $c$ to be the same as $a$. In a sense, the limits of integration are exactly the equations as written, though some of them are in appropriate form for Cartesian coordinates and other for cylindrical coordinates, so you need to rewrite some of them depending on which coordinates you want to use -- the integration can be performed in either.
If you want to use Cartesian coordinates, you need to solve $x^2+y^2=a^2$ for $x$ or $y$. In this case $y$ is perhaps slightly easier since you have $y\ge0$ but not $x\ge0$. If you want to use cylindrical coordinates $(x,y,z)=(\rho\cos\phi,\rho\sin\phi,z)$, you need to rewrite $z \le cy$ as $z\le c\rho\sin\phi$ and $y\ge0$ as $0\le\phi\le\pi$.
In the Cartesian case you also need limits for $x$; you can get those by considering for which values of $x$ the limits for $y$ make sense.
Comment: in other words this is my integral for solving Mx...  http://tinyurl.com/3vu92gn
Comment: a) You forgot the factor of $x$ (or $y$ in your notation). b) You omitted the restriction $y\ge0$.
Comment: Corrected: http://tinyurl.com/42t78q6
Comment: If $k$ is the constant mass density, that looks OK now.

Center of mass of a solid


Which classifiers do consider the order of the features?

Question: In case the order of features can make a difference in the results of a classification approach, which classifier algorithms perform better? I know Naive Bayes/KNN use bag of words and ignore the order. Does SVM do the same (I understand it highly depends on its kernel)? And does feature selection approach do the same thing as the bag of words-based techniques?
Answer: Bag of words: it's a model to represent you text as a bag of words, which ignore the order of the original text (If you only include unigram words).
Naive Bayes/KNN/SVM are all machine learning classification approach by using the features(words) from bag of words model to predict. I think none of these methods take the order into consideration if you only include unigram words.
If you incorporate N-gram words (bigram, trigram..) into your bag of words model, the order of these features/words will be taken care of in all machine learning models
Comment: +1 The OP says "use bag of words and ignore order", when in fact its "are often used with bag of words, which ignores order".
Comment: Thanks. So if we consider n-grams we would have a bag-of-n_grams which still does not consider the order of the n-grams. The reason I am asking this follow-up question is that I am comparing SVM with a bag-of-n_grams and a traditional kernel with other text classification methods such as SVM with string kernels/word sequence kernels.